{"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59094,"databundleVersionId":7010844,"sourceType":"competition"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\nimport torch\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nprint(f'Using{device}')\n\n\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.status.busy":"2024-07-14T18:39:53.518733Z","iopub.execute_input":"2024-07-14T18:39:53.519189Z","iopub.status.idle":"2024-07-14T18:39:58.206802Z","shell.execute_reply.started":"2024-07-14T18:39:53.519155Z","shell.execute_reply":"2024-07-14T18:39:58.205758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Processing","metadata":{}},{"cell_type":"code","source":"!pip install rdkit","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:40:07.758023Z","iopub.execute_input":"2024-07-14T18:40:07.758896Z","iopub.status.idle":"2024-07-14T18:40:24.627298Z","shell.execute_reply.started":"2024-07-14T18:40:07.758861Z","shell.execute_reply":"2024-07-14T18:40:24.626042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df = pd.read_parquet('../input/open-problems-single-cell-perturbations/de_train.parquet')\ndf.tail()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:40:44.328889Z","iopub.execute_input":"2024-07-14T18:40:44.329711Z","iopub.status.idle":"2024-07-14T18:40:47.115783Z","shell.execute_reply.started":"2024-07-14T18:40:44.329667Z","shell.execute_reply":"2024-07-14T18:40:47.114792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.columns","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:27:03.362829Z","iopub.execute_input":"2024-07-14T19:27:03.363268Z","iopub.status.idle":"2024-07-14T19:27:03.370664Z","shell.execute_reply.started":"2024-07-14T19:27:03.363232Z","shell.execute_reply":"2024-07-14T19:27:03.369428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Loader","metadata":{}},{"cell_type":"markdown","source":"# Building Experts","metadata":{}},{"cell_type":"markdown","source":"# final trial","metadata":{}},{"cell_type":"code","source":"feature_cols =['cell_type','sm_name']\ntarget_cols = ['cell_type','sm_name','sm_lincs_id','SMILES','control']\ntargets = df.drop(columns=target_cols)\nfeatures = pd.DataFrame(df,columns=feature_cols)\nfeatures","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:40:51.617274Z","iopub.execute_input":"2024-07-14T18:40:51.617669Z","iopub.status.idle":"2024-07-14T18:40:51.667228Z","shell.execute_reply.started":"2024-07-14T18:40:51.617636Z","shell.execute_reply":"2024-07-14T18:40:51.666012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import OneHotEncoder\none_hot = OneHotEncoder()\nfeatures_array = one_hot.fit_transform(features)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:40:54.007164Z","iopub.execute_input":"2024-07-14T18:40:54.007569Z","iopub.status.idle":"2024-07-14T18:40:54.532402Z","shell.execute_reply.started":"2024-07-14T18:40:54.007529Z","shell.execute_reply":"2024-07-14T18:40:54.531310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:40:57.250919Z","iopub.execute_input":"2024-07-14T18:40:57.251772Z","iopub.status.idle":"2024-07-14T18:40:57.264477Z","shell.execute_reply.started":"2024-07-14T18:40:57.251736Z","shell.execute_reply":"2024-07-14T18:40:57.263229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features_one_hot=features_array\nprint(features_one_hot.toarray().shape)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:40:59.861557Z","iopub.execute_input":"2024-07-14T18:40:59.861962Z","iopub.status.idle":"2024-07-14T18:40:59.869616Z","shell.execute_reply.started":"2024-07-14T18:40:59.861932Z","shell.execute_reply":"2024-07-14T18:40:59.868409Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nfrom rdkit import Chem\nfrom rdkit.Chem import Descriptors, rdMolDescriptors, QED\nfrom rdkit.Chem.rdMolDescriptors import CalcTPSA, CalcNumRotatableBonds, CalcNumHBA, CalcNumHBD, CalcFractionCSP3\nfrom rdkit.Chem import BRICS, Recap\n\n\ndef extract_smiles_info(smiles):\n    if smiles is None or smiles == '':\n        return None\n    mol = Chem.MolFromSmiles(smiles)\n    if mol is None:\n        return None\n    info = {}\n    info['Molecular Formula'] = Chem.rdMolDescriptors.CalcMolFormula(mol)\n    info['Molecular Weight'] = Descriptors.MolWt(mol)\n    info['LogP'] = Descriptors.MolLogP(mol)\n    info['Number of Atoms'] = mol.GetNumAtoms()\n    info['Number of Bonds'] = mol.GetNumBonds()\n    return info","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:43:32.338120Z","iopub.execute_input":"2024-07-14T19:43:32.338813Z","iopub.status.idle":"2024-07-14T19:43:32.346670Z","shell.execute_reply.started":"2024-07-14T19:43:32.338776Z","shell.execute_reply":"2024-07-14T19:43:32.345629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"smiles_info_list = df['SMILES'].apply(extract_smiles_info)\n\n# Convert list of dictionaries to a dataframe\nsmiles_info_df = pd.DataFrame(smiles_info_list.tolist())\n\n# Concatenate the original dataframe with the new dataframe containing SMILES information\ndf_extended = pd.concat([df, smiles_info_df], axis=1)\ndf_extended.head()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:03.680074Z","iopub.execute_input":"2024-07-14T18:41:03.680958Z","iopub.status.idle":"2024-07-14T18:41:04.593026Z","shell.execute_reply.started":"2024-07-14T18:41:03.680923Z","shell.execute_reply":"2024-07-14T18:41:04.592182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import OneHotEncoder, StandardScaler\n# Normalize the specified numerical columns\nscaler = StandardScaler()\ndf_extended[['Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']] = scaler.fit_transform(df_extended[['Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']])\n# Display the updated dataframe\ndf_extended","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:06.718801Z","iopub.execute_input":"2024-07-14T18:41:06.719452Z","iopub.status.idle":"2024-07-14T18:41:06.848645Z","shell.execute_reply.started":"2024-07-14T18:41:06.719421Z","shell.execute_reply":"2024-07-14T18:41:06.847692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_features = ['cell_type', 'sm_name','Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']\ndf_train = df_extended[train_features]","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:10.096207Z","iopub.execute_input":"2024-07-14T18:41:10.096983Z","iopub.status.idle":"2024-07-14T18:41:10.103613Z","shell.execute_reply.started":"2024-07-14T18:41:10.096931Z","shell.execute_reply":"2024-07-14T18:41:10.102591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_norm = df_train[['Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']]","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:11.685088Z","iopub.execute_input":"2024-07-14T18:41:11.685470Z","iopub.status.idle":"2024-07-14T18:41:11.691481Z","shell.execute_reply.started":"2024-07-14T18:41:11.685440Z","shell.execute_reply":"2024-07-14T18:41:11.690372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_hot = df_train[['cell_type', 'sm_name']]","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:12.402653Z","iopub.execute_input":"2024-07-14T18:41:12.403100Z","iopub.status.idle":"2024-07-14T18:41:12.408741Z","shell.execute_reply.started":"2024-07-14T18:41:12.403067Z","shell.execute_reply":"2024-07-14T18:41:12.407656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"encoder = OneHotEncoder(sparse=False)\nencoded_features = encoder.fit_transform(df_hot[['cell_type', 'sm_name']])\nencoded_df = pd.DataFrame(encoded_features, columns=encoder.get_feature_names_out(['cell_type', 'sm_name']))","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:13.307935Z","iopub.execute_input":"2024-07-14T18:41:13.308863Z","iopub.status.idle":"2024-07-14T18:41:13.320372Z","shell.execute_reply.started":"2024-07-14T18:41:13.308827Z","shell.execute_reply":"2024-07-14T18:41:13.319327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Combining the encoded and normalized features\nfinal_df = pd.concat([encoded_df, df_norm], axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:14.378917Z","iopub.execute_input":"2024-07-14T18:41:14.379323Z","iopub.status.idle":"2024-07-14T18:41:14.385298Z","shell.execute_reply.started":"2024-07-14T18:41:14.379292Z","shell.execute_reply":"2024-07-14T18:41:14.384357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_df","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:17.901675Z","iopub.execute_input":"2024-07-14T18:41:17.902592Z","iopub.status.idle":"2024-07-14T18:41:17.952628Z","shell.execute_reply.started":"2024-07-14T18:41:17.902546Z","shell.execute_reply":"2024-07-14T18:41:17.951505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"array = final_df.to_numpy()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:19.878758Z","iopub.execute_input":"2024-07-14T18:41:19.879683Z","iopub.status.idle":"2024-07-14T18:41:19.884548Z","shell.execute_reply.started":"2024-07-14T18:41:19.879646Z","shell.execute_reply":"2024-07-14T18:41:19.883364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"array.shape","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:20.972186Z","iopub.execute_input":"2024-07-14T18:41:20.973113Z","iopub.status.idle":"2024-07-14T18:41:20.979242Z","shell.execute_reply.started":"2024-07-14T18:41:20.973076Z","shell.execute_reply":"2024-07-14T18:41:20.978186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\n# Split the data into 70% training, 15% validation, and 15% testing\nX_train, X_temp, y_train, y_temp = train_test_split(array, targets.values, test_size=0.1, shuffle=False)\nX_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:21.922610Z","iopub.execute_input":"2024-07-14T18:41:21.923464Z","iopub.status.idle":"2024-07-14T18:41:22.084330Z","shell.execute_reply.started":"2024-07-14T18:41:21.923430Z","shell.execute_reply":"2024-07-14T18:41:22.083461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"featurespace = array\ntargetsspace = targets.values\ntargetsspace.shape","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:23.776080Z","iopub.execute_input":"2024-07-14T18:41:23.777152Z","iopub.status.idle":"2024-07-14T18:41:23.783610Z","shell.execute_reply.started":"2024-07-14T18:41:23.777104Z","shell.execute_reply":"2024-07-14T18:41:23.782602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from torch.utils.data import DataLoader,Dataset\nclass dataset(Dataset):\n    def __init__(self,X_train,y_train):\n        self.X_train=X_train\n        self.y_train=y_train\n    def __len__(self):\n        return len(self.X_train)\n    def __getitem__(self,idx):\n        return self.X_train[idx], self.y_train[idx]","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:24.637840Z","iopub.execute_input":"2024-07-14T18:41:24.638711Z","iopub.status.idle":"2024-07-14T18:41:24.645406Z","shell.execute_reply.started":"2024-07-14T18:41:24.638676Z","shell.execute_reply":"2024-07-14T18:41:24.644407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\n\nfeatures_t= torch.tensor(featurespace).to(device)\ntarget_t=torch.tensor(targetsspace).to(device)\ndata=dataset(features_t,target_t)\ntarget_t.shape","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:26.271080Z","iopub.execute_input":"2024-07-14T18:41:26.271548Z","iopub.status.idle":"2024-07-14T18:41:26.545879Z","shell.execute_reply.started":"2024-07-14T18:41:26.271517Z","shell.execute_reply":"2024-07-14T18:41:26.544830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batchsize=64\ndataloader = torch.utils.data.DataLoader(data,batch_size=batchsize,shuffle=True)\nfor batch,(x,y) in enumerate(dataloader):\n    print('batch',batch,x.shape,y.shape)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:28.716722Z","iopub.execute_input":"2024-07-14T18:41:28.717227Z","iopub.status.idle":"2024-07-14T18:41:28.778566Z","shell.execute_reply.started":"2024-07-14T18:41:28.717191Z","shell.execute_reply":"2024-07-14T18:41:28.777399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import mean_absolute_error\nimport numpy as np\nfrom scipy.sparse import issparse\n\n# Define custom loss function\nclass LogCoshLoss(nn.Module):\n    def __init__(self):\n        super().__init__()\n    \n    def forward(self, y_prime_t, y_t):\n        ey_t = (y_t - y_prime_t) / 3  # divide by 3 to avoid numerical overflow in cosh\n        return torch.mean(torch.log(torch.cosh(ey_t + 1e-12)))\n\n# Define expert models\nclass Conv(nn.Module):\n    def __init__(self, scheme, input_channels=1):\n        super(Conv, self).__init__()\n        self.name = 'Conv'\n        self.conv_block = nn.Sequential(\n            nn.Conv1d(input_channels, 32, kernel_size=5, stride=1, padding=2),\n            nn.ReLU(),\n            nn.Dropout(0.3),\n            nn.Conv1d(32, 64, kernel_size=5, stride=2, padding=2),\n            nn.ReLU(),\n            nn.Dropout(0.3),\n            nn.Conv1d(64, 128, kernel_size=5, stride=2, padding=2),\n            nn.ReLU(),\n            nn.Dropout(0.3),\n            nn.AdaptiveAvgPool1d(1),\n            nn.Flatten())\n        self.linear = nn.Sequential(\n            nn.Linear(128, 512),\n            nn.ReLU(),\n            nn.Dropout(0.3),\n            nn.Linear(512, 512),\n            nn.ReLU(),\n            nn.Dropout(0.3)\n        )\n        self.head1 = nn.Linear(512, output_size)\n        \n    def forward(self, x):\n        # Ensure input is 3D: (batch_size, channels, sequence_length)\n        if x.dim() == 2:\n            x = x.unsqueeze(1)\n        out = self.conv_block(x)\n        out = self.linear(out)\n        out = self.head1(out)\n        return out\n\nclass LSTM(nn.Module):\n    def __init__(self, scheme, input_size, output_size):\n        super(LSTM, self).__init__()\n        self.name = 'LSTM'\n        self.lstm = nn.LSTM(input_size, 128, num_layers=2, batch_first=True)\n        self.linear = nn.Sequential(\n            nn.Linear(128, 512),\n            nn.ReLU(),\n            nn.Dropout(0.3),\n            nn.Linear(512, 512),\n            nn.ReLU(),\n            nn.Dropout(0.3)\n        )\n        self.head1 = nn.Linear(512, output_size)\n        \n    def forward(self, x):\n        # Ensure input is 3D: (batch_size, sequence_length, input_size)\n        if x.dim() == 2:\n            x = x.unsqueeze(1)  # Add sequence dimension\n        \n        batch_size, seq_len, _ = x.size()\n        \n        out, (hn, cn) = self.lstm(x)\n        \n        if seq_len == 1:\n            out = out.squeeze(1)  # Remove sequence dimension if it's 1\n        else:\n            out = out[:, -1, :]  # Take the last time step\n        \n        out = self.linear(out)\n        out = self.head1(out)\n        return out\n\nclass GRU(nn.Module):\n    def __init__(self, scheme, input_size, output_size):\n        super(GRU, self).__init__()\n        self.name = 'GRU'\n        self.gru = nn.GRU(input_size, 128, num_layers=2, batch_first=True)\n        self.linear = nn.Sequential(\n            nn.Linear(128, 512),\n            nn.ReLU(),\n            nn.Dropout(0.3),\n            nn.Linear(512, 512),\n            nn.ReLU(),\n            nn.Dropout(0.3)\n        )\n        self.head1 = nn.Linear(512, output_size)\n        \n    def forward(self, x):\n        # Ensure input is 3D: (batch_size, sequence_length, input_size)\n        if x.dim() == 2:\n            x = x.unsqueeze(1)  # Add sequence dimension\n        \n        batch_size, seq_len, _ = x.size()\n        \n        out, hn = self.gru(x)\n        \n        if seq_len == 1:\n            out = out.squeeze(1)  # Remove sequence dimension if it's 1\n        else:\n            out = out[:, -1, :]  # Take the last time step\n        \n        out = self.linear(out)\n        out = self.head1(out)\n        return out\n    \n    \n# Define Gating Network\nclass GatingNetwork(nn.Module):\n    def __init__(self, input_size, num_experts):\n        super(GatingNetwork, self).__init__()\n        self.linear1 = nn.Linear(input_size, 64)\n        self.relu = nn.ReLU()\n        self.linear2 = nn.Linear(64, num_experts)\n        self.softmax = nn.Softmax(dim=-1)\n    \n    def forward(self, data):\n        x = self.linear1(data)\n        x = self.relu(x)\n        x = self.linear2(x)\n        x = self.softmax(x)\n        return x\n\n# Define Mixture of Experts model\nclass MixtureOfExperts(nn.Module):\n    def __init__(self, input_size, output_size, scheme):\n        super(MixtureOfExperts, self).__init__()\n        self.experts = nn.ModuleList([\n            Conv(scheme, input_channels=1),\n            LSTM(scheme, input_size, output_size),\n            GRU(scheme, input_size, output_size)\n        ])\n        self.gating_network = GatingNetwork(input_size, len(self.experts))\n        self.loss1 = nn.MSELoss()\n        self.loss2 = LogCoshLoss()\n        self.loss3 = nn.L1Loss()\n        self.loss4 = nn.BCELoss()\n    \n    def forward(self, data, y=None):\n        data = data.float()\n        \n        gating_weights = self.gating_network(data)\n        \n        expert_outputs = []\n        for i, expert in enumerate(self.experts):\n            if i == 0:  # Conv\n                expert_input = data.unsqueeze(1) if data.dim() == 2 else data\n            else:  # LSTM and GRU\n                expert_input = data if data.dim() == 3 else data.unsqueeze(1)\n            \n            output = expert(expert_input)\n            expert_outputs.append(output)\n        \n        expert_outputs = torch.stack(expert_outputs, dim=1)\n        combined_output = torch.sum(expert_outputs * gating_weights.unsqueeze(-1), dim=1)\n        \n        if y is None:\n            return combined_output\n        else:\n            y = y.float()\n            loss1 = 0.4 * self.loss1(combined_output, y) + 0.3 * self.loss2(combined_output, y) + 0.3 * self.loss3(combined_output, y)\n            yhat = torch.sigmoid(combined_output)\n            yy = torch.sigmoid(y)\n            loss2 = self.loss4(yhat, yy)\n            return 0.8 * loss1 + 0.2 * loss2\n# Define RMSE rowwise loss\ndef RMSE_rowwise_loss(model, data, y_true):\n    model.eval()\n    with torch.no_grad():\n        y_pred_original = model(data.float())\n    mae = mean_absolute_error(y_true.numpy(), y_pred_original.numpy())\n    rowwise_rmse = np.sqrt(np.mean(np.square(y_true.numpy() - y_pred_original.numpy()), axis=1))\n    mrrmse_score = np.mean(rowwise_rmse)\n    return mrrmse_score\n\n# Usage\ninput_size = 156\noutput_size = 18211\nscheme = 'initial'\nepochs = 2000\n\nmodel = MixtureOfExperts(input_size, output_size, scheme)\nmodel = model.float()\nmodel = model.to(device)\n\n\noptimizer = torch.optim.Adam(model.parameters(), lr=0.001)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:30.599487Z","iopub.execute_input":"2024-07-14T18:41:30.599842Z","iopub.status.idle":"2024-07-14T18:41:32.527289Z","shell.execute_reply.started":"2024-07-14T18:41:30.599813Z","shell.execute_reply":"2024-07-14T18:41:32.526433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import logging\nlogging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')\n\ndef train_model(model, data, optimizer, epochs, batch_size, device):\n    dataloader = DataLoader(data, batch_size=batch_size, shuffle=True)\n    \n    for epoch in range(epochs):\n        model.train()\n        epoch_loss = 0\n        total_batches = 0\n        \n        for batch, (x, y) in enumerate(dataloader):\n            x = x.float().to(device)\n            y = y.float().to(device)\n            \n            optimizer.zero_grad()\n            try:\n                loss = model(x, y)\n                loss.backward()\n                optimizer.step()\n                \n                epoch_loss += loss.item()\n                total_batches += 1\n            except RuntimeError as e:\n                print(f\"Error in epoch {epoch+1}, batch {batch+1}: {str(e)}\")\n                print(f\"Input shapes: x: {x.shape}, y: {y.shape}\")\n                continue\n        \n        avg_loss = epoch_loss / total_batches\n        print(f'Epoch {epoch+1}/{epochs} completed. Average loss: {avg_loss:.4f}')\n\n# Usage\nepochs = 2000\nbatch_size = 64\ntrain_model(model, data, optimizer, epochs, batch_size, device)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:41:33.568038Z","iopub.execute_input":"2024-07-14T18:41:33.568912Z","iopub.status.idle":"2024-07-14T18:47:34.338091Z","shell.execute_reply.started":"2024-07-14T18:41:33.568876Z","shell.execute_reply":"2024-07-14T18:47:34.336999Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"id_map = pd.read_csv(\"/kaggle/input/open-problems-single-cell-perturbations/id_map.csv\")","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:33:23.138785Z","iopub.execute_input":"2024-07-14T19:33:23.139208Z","iopub.status.idle":"2024-07-14T19:33:23.146942Z","shell.execute_reply.started":"2024-07-14T19:33:23.139175Z","shell.execute_reply":"2024-07-14T19:33:23.145739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"id_map","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:33:25.575006Z","iopub.execute_input":"2024-07-14T19:33:25.575807Z","iopub.status.idle":"2024-07-14T19:33:25.588279Z","shell.execute_reply.started":"2024-07-14T19:33:25.575772Z","shell.execute_reply":"2024-07-14T19:33:25.587288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"id_map.columns","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:33:29.052060Z","iopub.execute_input":"2024-07-14T19:33:29.053126Z","iopub.status.idle":"2024-07-14T19:33:29.059666Z","shell.execute_reply.started":"2024-07-14T19:33:29.053090Z","shell.execute_reply":"2024-07-14T19:33:29.058611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Creating a dictionary to map sm_name to SMILES\nsm_name_to_smiles = df.set_index('sm_name')['SMILES'].to_dict()\nlen(sm_name_to_smiles)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:33:32.547591Z","iopub.execute_input":"2024-07-14T19:33:32.548112Z","iopub.status.idle":"2024-07-14T19:33:32.596094Z","shell.execute_reply.started":"2024-07-14T19:33:32.548068Z","shell.execute_reply":"2024-07-14T19:33:32.595030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Using the dictionary to map SMILES to the id_map DataFrame\nid_map['SMILES'] = id_map['sm_name'].map(sm_name_to_smiles)\nid_map.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:34:49.674004Z","iopub.execute_input":"2024-07-14T19:34:49.675040Z","iopub.status.idle":"2024-07-14T19:34:49.687531Z","shell.execute_reply.started":"2024-07-14T19:34:49.674991Z","shell.execute_reply":"2024-07-14T19:34:49.686218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"id_map","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:35:57.571133Z","iopub.execute_input":"2024-07-14T19:35:57.571555Z","iopub.status.idle":"2024-07-14T19:35:57.585903Z","shell.execute_reply.started":"2024-07-14T19:35:57.571521Z","shell.execute_reply":"2024-07-14T19:35:57.584810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"id_map.to_csv('id_map_with_smiles.csv', index=False)\nid_map.head()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T18:57:27.593397Z","iopub.execute_input":"2024-07-14T18:57:27.593794Z","iopub.status.idle":"2024-07-14T18:57:27.613254Z","shell.execute_reply.started":"2024-07-14T18:57:27.593762Z","shell.execute_reply":"2024-07-14T18:57:27.612323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create Map of smiles and do feature extraction here and run ","metadata":{}},{"cell_type":"code","source":"# Load the test data\ntestdata = pd.DataFrame(id_map, columns=['cell_type', 'sm_name', 'SMILES'])\ntestdata","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:38:09.579577Z","iopub.execute_input":"2024-07-14T19:38:09.580455Z","iopub.status.idle":"2024-07-14T19:38:09.594441Z","shell.execute_reply.started":"2024-07-14T19:38:09.580417Z","shell.execute_reply":"2024-07-14T19:38:09.593240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_smiles_info_list = testdata['SMILES'].apply(extract_smiles_info)\n# smiles_info_list = df['SMILES'].apply(extract_smiles_info)\n\n# Convert list of dictionaries to a dataframe\ntest_smiles_info_list_df = pd.DataFrame(test_smiles_info_list.tolist())\n\n# Concatenate the original dataframe with the new dataframe containing SMILES information\ntest_df_extended = pd.concat([testdata, test_smiles_info_list_df], axis=1)\ntest_df_extended.head()\n","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:45:36.049236Z","iopub.execute_input":"2024-07-14T19:45:36.049639Z","iopub.status.idle":"2024-07-14T19:45:36.337929Z","shell.execute_reply.started":"2024-07-14T19:45:36.049609Z","shell.execute_reply":"2024-07-14T19:45:36.336665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalize the specified numerical columns in the test data\ntest_df_extended[['Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']] = scaler.transform(test_df_extended[['Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']])\ntest_df_extended.head()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:45:59.908619Z","iopub.execute_input":"2024-07-14T19:45:59.909589Z","iopub.status.idle":"2024-07-14T19:45:59.929232Z","shell.execute_reply.started":"2024-07-14T19:45:59.909543Z","shell.execute_reply":"2024-07-14T19:45:59.927949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_hot = test_df_extended[['cell_type', 'sm_name']]\nencoded_test_features = encoder.transform(test_hot[['cell_type', 'sm_name']])\nencoded_test_df = pd.DataFrame(encoded_test_features, columns=encoder.get_feature_names_out(['cell_type', 'sm_name']))","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:47:01.071022Z","iopub.execute_input":"2024-07-14T19:47:01.071728Z","iopub.status.idle":"2024-07-14T19:47:01.082255Z","shell.execute_reply.started":"2024-07-14T19:47:01.071694Z","shell.execute_reply":"2024-07-14T19:47:01.081028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_test_df = pd.concat([encoded_test_df, test_df_extended[['Molecular Weight', 'LogP', 'Number of Atoms', 'Number of Bonds']]], axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:47:28.990213Z","iopub.execute_input":"2024-07-14T19:47:28.990888Z","iopub.status.idle":"2024-07-14T19:47:28.998293Z","shell.execute_reply.started":"2024-07-14T19:47:28.990855Z","shell.execute_reply":"2024-07-14T19:47:28.997126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_test_df","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:47:35.588878Z","iopub.execute_input":"2024-07-14T19:47:35.589769Z","iopub.status.idle":"2024-07-14T19:47:35.646387Z","shell.execute_reply.started":"2024-07-14T19:47:35.589732Z","shell.execute_reply":"2024-07-14T19:47:35.645297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_array = final_test_df.to_numpy()\n","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:47:51.234873Z","iopub.execute_input":"2024-07-14T19:47:51.235635Z","iopub.status.idle":"2024-07-14T19:47:51.240953Z","shell.execute_reply.started":"2024-07-14T19:47:51.235597Z","shell.execute_reply":"2024-07-14T19:47:51.239800Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_array.shape","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:47:53.968706Z","iopub.execute_input":"2024-07-14T19:47:53.969132Z","iopub.status.idle":"2024-07-14T19:47:53.975811Z","shell.execute_reply.started":"2024-07-14T19:47:53.969098Z","shell.execute_reply":"2024-07-14T19:47:53.974737Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_features_t = torch.tensor(test_array).to(device)\n","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:47:57.439737Z","iopub.execute_input":"2024-07-14T19:47:57.440623Z","iopub.status.idle":"2024-07-14T19:47:57.446029Z","shell.execute_reply.started":"2024-07-14T19:47:57.440586Z","shell.execute_reply":"2024-07-14T19:47:57.444885Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.eval()\n# target_pred = model(torch.Tensor(one_hot_test.toarray()).to(device))","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:48:02.003910Z","iopub.execute_input":"2024-07-14T19:48:02.004721Z","iopub.status.idle":"2024-07-14T19:48:02.012323Z","shell.execute_reply.started":"2024-07-14T19:48:02.004685Z","shell.execute_reply":"2024-07-14T19:48:02.011266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target_pred = model(torch.Tensor(test_array).to(device))","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:48:20.237313Z","iopub.execute_input":"2024-07-14T19:48:20.237730Z","iopub.status.idle":"2024-07-14T19:48:20.250739Z","shell.execute_reply.started":"2024-07-14T19:48:20.237697Z","shell.execute_reply":"2024-07-14T19:48:20.249597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target_pred","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:48:23.604867Z","iopub.execute_input":"2024-07-14T19:48:23.605844Z","iopub.status.idle":"2024-07-14T19:48:23.616927Z","shell.execute_reply.started":"2024-07-14T19:48:23.605795Z","shell.execute_reply":"2024-07-14T19:48:23.615750Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"res = target_pred.cpu()","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:52:16.055203Z","iopub.execute_input":"2024-07-14T19:52:16.055681Z","iopub.status.idle":"2024-07-14T19:52:16.068841Z","shell.execute_reply.started":"2024-07-14T19:52:16.055641Z","shell.execute_reply":"2024-07-14T19:52:16.067789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# one_hot_test = one_hot.transform(testdata)\n# print(features_one_hot.toarray().shape,one_hot_test.toarray().shape)","metadata":{"execution":{"iopub.status.busy":"2024-07-13T04:06:58.765215Z","iopub.status.idle":"2024-07-13T04:06:58.765562Z","shell.execute_reply":"2024-07-13T04:06:58.765404Z","shell.execute_reply.started":"2024-07-13T04:06:58.765389Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission = pd.read_csv(\"/kaggle/input/open-problems-single-cell-perturbations/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:49:40.455531Z","iopub.execute_input":"2024-07-14T19:49:40.456396Z","iopub.status.idle":"2024-07-14T19:49:43.517973Z","shell.execute_reply.started":"2024-07-14T19:49:40.456335Z","shell.execute_reply":"2024-07-14T19:49:43.517089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_columns = sample_submission.columns\nsample_columns= sample_columns[1:]\nsubmission_df = pd.DataFrame(res.detach().numpy(), columns=sample_columns)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:52:38.648394Z","iopub.execute_input":"2024-07-14T19:52:38.648798Z","iopub.status.idle":"2024-07-14T19:52:38.654642Z","shell.execute_reply.started":"2024-07-14T19:52:38.648767Z","shell.execute_reply":"2024-07-14T19:52:38.653392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.insert(0, 'id', range(255))","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:52:43.532058Z","iopub.execute_input":"2024-07-14T19:52:43.532810Z","iopub.status.idle":"2024-07-14T19:52:43.541178Z","shell.execute_reply.started":"2024-07-14T19:52:43.532773Z","shell.execute_reply":"2024-07-14T19:52:43.540084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_submission","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:52:45.630500Z","iopub.execute_input":"2024-07-14T19:52:45.631456Z","iopub.status.idle":"2024-07-14T19:52:45.684516Z","shell.execute_reply.started":"2024-07-14T19:52:45.631421Z","shell.execute_reply":"2024-07-14T19:52:45.683531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:53:01.047193Z","iopub.execute_input":"2024-07-14T19:53:01.047997Z","iopub.status.idle":"2024-07-14T19:53:01.083891Z","shell.execute_reply.started":"2024-07-14T19:53:01.047948Z","shell.execute_reply":"2024-07-14T19:53:01.082735Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2024-07-14T19:53:07.154703Z","iopub.execute_input":"2024-07-14T19:53:07.155433Z","iopub.status.idle":"2024-07-14T19:53:16.921123Z","shell.execute_reply.started":"2024-07-14T19:53:07.155400Z","shell.execute_reply":"2024-07-14T19:53:16.920222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}