{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Facial keypoint detection\n\n### If you find this notebook useful please upvote it.","metadata":{}},{"cell_type":"code","source":"import sys\nimport os\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom sklearn.impute import SimpleImputer\n\nimport torch","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-22T05:29:49.830897Z","iopub.execute_input":"2022-07-22T05:29:49.831641Z","iopub.status.idle":"2022-07-22T05:29:53.543666Z","shell.execute_reply.started":"2022-07-22T05:29:49.831502Z","shell.execute_reply":"2022-07-22T05:29:53.542298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 1. Data Extraction\n\nFacial keypoint detection data is present in zip format. So firstly we extract the data to get required csv files.","metadata":{"execution":{"iopub.status.busy":"2022-07-01T12:06:52.926037Z","iopub.execute_input":"2022-07-01T12:06:52.926843Z","iopub.status.idle":"2022-07-01T12:06:53.692968Z","shell.execute_reply.started":"2022-07-01T12:06:52.926792Z","shell.execute_reply":"2022-07-01T12:06:53.691407Z"}}},{"cell_type":"code","source":"\n!unzip -o /kaggle/input/facial-keypoints-detection/training.zip -d /kaggle/working/facial-keypoints-detection/\n!unzip -o /kaggle/input/facial-keypoints-detection/test.zip -d /kaggle/working/facial-keypoints-detection/","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:29:53.545986Z","iopub.execute_input":"2022-07-22T05:29:53.546548Z","iopub.status.idle":"2022-07-22T05:29:57.752528Z","shell.execute_reply.started":"2022-07-22T05:29:53.546518Z","shell.execute_reply":"2022-07-22T05:29:57.751166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Read CSV files as dataframes\n\nIn below we read all required files as dataframes.","metadata":{}},{"cell_type":"code","source":"input_dir = \"/kaggle/input/facial-keypoints-detection\"\noutput_dir = \"/kaggle/working\"\n\ndf_lookup = pd.read_csv(f\"{input_dir}/IdLookupTable.csv\")\ndf_submission = pd.read_csv(f\"{input_dir}/SampleSubmission.csv\")\n\ndf_training_csv = pd.read_csv(f\"{output_dir}/facial-keypoints-detection/training.csv\")\ndf_test_csv = pd.read_csv(f\"{output_dir}/facial-keypoints-detection/test.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:29:57.755808Z","iopub.execute_input":"2022-07-22T05:29:57.756425Z","iopub.status.idle":"2022-07-22T05:30:00.610577Z","shell.execute_reply.started":"2022-07-22T05:29:57.756314Z","shell.execute_reply":"2022-07-22T05:30:00.609249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.1 Explanation of CSV files\n\nNow as we could see that a number of CSV files are read in above cell. Lets understand what each file contains.\n\n#### Training Files:\n\n1. **training.csv:** This file contains training data. Last column of this file named `Image` contains image data. Rest of the columns denote `x` and `y` coordinate values values for different keypoints. E.g.: First two columns `left_eye_center_x`, `left_eye_center_y` denote `x` and `y` coordinates of `center point of left eye`.\n\n    Also, column `Image` contains image as a string of numbers, separated by spaces. Later in this notebook we will convert this image to 2D numpy array so that we could process it as actual image.\n\n#### Testing Files:\n\nRest of the files will be used to make submissions. Lets understand each one in detail:\n\n1. **test.csv:** This file contains only two columns `ImageId`, `Image`. Again, here also image is represented as string of numbers that will be later converted to 2D numpy arrays. We need to make predictions of keypoints for each image present in this file.\n\n2. **IdLookupTable.csv:** For each image in test we do not need to predict all keypoints. What keypoints are required for each image are mentioned in this file. This file has got 4 columns: `RowId`, `ImageId`, `FeatureName`, `Location`. So we need to extract `Location` for specific `FeatureName` for each `ImageId`.\n\n3. **SampleSubmission.csv:** This is the final submission file. This file is basically a subset of `IdLookupTable.csv`. Just take columns `RowId`, `Location` from `IdLookupTable.csv` and you could create your final `SampleSubmission.csv`.","metadata":{}},{"cell_type":"code","source":"df_training_csv.shape, df_test_csv.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:30:00.614030Z","iopub.execute_input":"2022-07-22T05:30:00.614698Z","iopub.status.idle":"2022-07-22T05:30:00.625172Z","shell.execute_reply.started":"2022-07-22T05:30:00.614654Z","shell.execute_reply":"2022-07-22T05:30:00.623862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we discussed earlier that `Image` column is a string so we will create a new column `ImageNumpy` and store numpy format of image in this column.","metadata":{}},{"cell_type":"code","source":"df_training_csv[\"Image_Numpy\"] = [ np.array(x.split(\" \")).reshape(96,96).astype(\"float\") for x in df_training_csv[\"Image\"]]\ndf_test_csv[\"Image_Numpy\"] = [ np.array(x.split(\" \")).reshape(96,96).astype(\"float\") for x in df_test_csv[\"Image\"]]","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:30:00.627405Z","iopub.execute_input":"2022-07-22T05:30:00.628201Z","iopub.status.idle":"2022-07-22T05:31:16.137486Z","shell.execute_reply.started":"2022-07-22T05:30:00.628141Z","shell.execute_reply":"2022-07-22T05:31:16.136126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Lets visualize some random images.","metadata":{}},{"cell_type":"code","source":"num_samples = 4\nids = np.random.randint(0, len(df_training_csv), num_samples)\nfor i, img in enumerate(df_training_csv.loc[ids, \"Image_Numpy\"].values):\n    plt.subplot(1, num_samples, i+1)\n    plt.imshow(img, cmap=\"gray\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:16.139466Z","iopub.execute_input":"2022-07-22T05:31:16.139976Z","iopub.status.idle":"2022-07-22T05:31:16.775665Z","shell.execute_reply.started":"2022-07-22T05:31:16.139933Z","shell.execute_reply":"2022-07-22T05:31:16.774433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.2 Handling missing values in data\n\nIn below cell we could see that there are total `7049` training points. However, many columns are having significant number of null (or missing) values. So, in order to handle null values if we remove rows or columns will null values then we will be loosing most of training data. \n\nHowever, if we closely check images then we find that generally most of the keypoint features are in same positions. E.g.: eyes are mostly located in upper half of image. So, instead of removing rows and columns will null values, we will replace null values with `mean()` value of that column. This will help us in retaining data.","metadata":{}},{"cell_type":"code","source":"print(\"Total training examples:\", len(df_training_csv))\n\nprint(\"Column wise null values:\")\ndf_training_csv.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:16.780602Z","iopub.execute_input":"2022-07-22T05:31:16.781338Z","iopub.status.idle":"2022-07-22T05:31:16.811768Z","shell.execute_reply.started":"2022-07-22T05:31:16.781298Z","shell.execute_reply":"2022-07-22T05:31:16.809866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Using `SimpleImputer` to fill missing values.","metadata":{}},{"cell_type":"code","source":"imputer = SimpleImputer(missing_values=np.nan, strategy='mean')\n\n# Impute train.\ndf_temp = df_training_csv.drop([\"Image\",\"Image_Numpy\"], axis=1)\ndf_temp = pd.DataFrame(imputer.fit_transform(df_temp), columns=list(df_temp.columns))\ndf_temp[\"Image\"] = df_training_csv[\"Image\"].values\ndf_temp[\"Image_Numpy\"] = df_training_csv[\"Image_Numpy\"].values\ndf_training_csv = df_temp.reset_index(drop=True)","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:16.813585Z","iopub.execute_input":"2022-07-22T05:31:16.814418Z","iopub.status.idle":"2022-07-22T05:31:16.848113Z","shell.execute_reply.started":"2022-07-22T05:31:16.814368Z","shell.execute_reply":"2022-07-22T05:31:16.846829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Defining input and output columns and prefix of columns with keypoints.","metadata":{"execution":{"iopub.status.busy":"2022-07-21T11:23:23.459165Z","iopub.execute_input":"2022-07-21T11:23:23.459580Z","iopub.status.idle":"2022-07-21T11:24:40.386739Z","shell.execute_reply.started":"2022-07-21T11:23:23.459543Z","shell.execute_reply":"2022-07-21T11:24:40.385589Z"}}},{"cell_type":"code","source":"out_col_names = list(df_training_csv.drop([\"Image\", \"Image_Numpy\"], axis=1).columns)\nin_col_names = \"Image_Numpy\"\n\nkeypoint_feature_prefixes = [\n 'left_eye_center',\n 'left_eye_inner_corner',\n 'left_eye_outer_corner',\n 'left_eyebrow_inner_end',\n 'left_eyebrow_outer_end',\n 'mouth_center_bottom_lip',\n 'mouth_center_top_lip',\n 'mouth_left_corner',\n 'mouth_right_corner',\n 'nose_tip',\n 'right_eye_center',\n 'right_eye_inner_corner',\n 'right_eye_outer_corner',\n 'right_eyebrow_inner_end',\n 'right_eyebrow_outer_end']","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:16.849786Z","iopub.execute_input":"2022-07-22T05:31:16.852881Z","iopub.status.idle":"2022-07-22T05:31:16.866686Z","shell.execute_reply.started":"2022-07-22T05:31:16.852845Z","shell.execute_reply":"2022-07-22T05:31:16.865434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1.3 Ploting images with keypoints\n\nNow that we have handled missing values, lets take a sneak peak on how an image will look with keypoints marked on it. In cell below a random image is selected and keypoints are marked as red dots.","metadata":{}},{"cell_type":"code","source":"def plot_image_with_points(img, keypoints):\n#     ncols = 4\n#     keypoints = np.squeeze(keypoints)\n#     if len(keypoints.shape)==1:\n#         keypoints = np.expand_dims(keypoints, axis=0)\n    keypoints = keypoints.reshape(len(keypoints)//2,2)\n    plt.plot(np.squeeze(keypoints[:,0]), np.squeeze(keypoints[:,1]), \"or\")\n    plt.imshow(img, cmap=\"gray\")\n    plt.show()\n\nidx = int(np.random.randint(0, len(df_training_csv), 1))\n\nimg = df_training_csv.loc[idx, in_col_names]\nprint(\"img\", img.shape)\nkeypoints = df_training_csv.loc[idx, out_col_names].values\nplot_image_with_points(img, keypoints)","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:16.873605Z","iopub.execute_input":"2022-07-22T05:31:16.874598Z","iopub.status.idle":"2022-07-22T05:31:17.147146Z","shell.execute_reply.started":"2022-07-22T05:31:16.874560Z","shell.execute_reply":"2022-07-22T05:31:17.145982Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Dataset\n\nIn cell below we divide available data into train and validation sets.","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler\n\ndf_train, df_val = train_test_split(df_training_csv.sample(frac=1).reset_index(drop=True), train_size=0.8)\n\nscaler = StandardScaler()\nscaler.fit(df_train[out_col_names])\n\ndf_train[out_col_names] = scaler.transform(df_train[out_col_names])\ndf_val[out_col_names] = scaler.transform(df_val[out_col_names])\n\ndf_train.shape, df_val.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:17.152605Z","iopub.execute_input":"2022-07-22T05:31:17.154245Z","iopub.status.idle":"2022-07-22T05:31:17.202894Z","shell.execute_reply.started":"2022-07-22T05:31:17.154189Z","shell.execute_reply":"2022-07-22T05:31:17.201631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now create input and output tensors for train and validation sets.","metadata":{}},{"cell_type":"code","source":"x_train = torch.Tensor(np.array(list(df_train[in_col_names].values)).astype(\"float\")/255.0)\nx_train = torch.unsqueeze(x_train, axis=1)\ny_train = torch.Tensor(df_train[out_col_names].values.astype(\"float\"))\n\n\nx_val = torch.Tensor(np.array(list(df_val[in_col_names].values)).astype(\"float\")/255.0)\nx_val = torch.unsqueeze(x_val, axis=1)\ny_val = torch.Tensor(df_val[out_col_names].values.astype(\"float\"))\n","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:17.204428Z","iopub.execute_input":"2022-07-22T05:31:17.204917Z","iopub.status.idle":"2022-07-22T05:31:18.013985Z","shell.execute_reply.started":"2022-07-22T05:31:17.204855Z","shell.execute_reply":"2022-07-22T05:31:18.012527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally create datasets and data loaders for training and validation data.","metadata":{}},{"cell_type":"code","source":"ds_train = torch.utils.data.TensorDataset(x_train[:], y_train[:])\nds_val = torch.utils.data.TensorDataset(x_val[:], y_val[:])\n\ndl_train = torch.utils.data.DataLoader(ds_train, batch_size=16)\ndl_val = torch.utils.data.DataLoader(ds_val, batch_size=16)","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:18.015950Z","iopub.execute_input":"2022-07-22T05:31:18.016776Z","iopub.status.idle":"2022-07-22T05:31:18.025336Z","shell.execute_reply.started":"2022-07-22T05:31:18.016735Z","shell.execute_reply":"2022-07-22T05:31:18.023532Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Model\n\nWe will be using a CNN based model. Input of the model would be image and output will be 30 values. Since there are total 15 keypoints present on each image and for each keypoint we will be predicting its `x` and `y` coordinates. So, a total of `15*2=30` values are predicted as output.","metadata":{}},{"cell_type":"code","source":"import pytorch_lightning as pl\nfrom tqdm.notebook import tqdm as tqdm_notebook","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:18.027105Z","iopub.execute_input":"2022-07-22T05:31:18.027717Z","iopub.status.idle":"2022-07-22T05:31:25.563686Z","shell.execute_reply.started":"2022-07-22T05:31:18.027677Z","shell.execute_reply":"2022-07-22T05:31:25.562293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ModelCNN(torch.nn.Module):\n    def __init__(self, in_channels=1, out_channels=30):\n        super(ModelCNN,self).__init__()\n        self.model = torch.nn.Sequential(\n            torch.nn.Conv2d(1,64, kernel_size=3, stride=1),\n            torch.nn.BatchNorm2d(64),\n            torch.nn.ReLU(),\n            torch.nn.MaxPool2d(2),\n            \n            torch.nn.Conv2d(64,128, kernel_size=3, stride=1),\n            torch.nn.Dropout(0.3),\n            torch.nn.BatchNorm2d(128),\n            torch.nn.ReLU(),\n            torch.nn.MaxPool2d(2),\n            \n            torch.nn.Conv2d(128,256, kernel_size=3, stride=1),\n            torch.nn.Dropout(0.3),\n            torch.nn.BatchNorm2d(256),\n            torch.nn.ReLU(),\n            torch.nn.MaxPool2d(2),\n            \n            torch.nn.Conv2d(256,512, kernel_size=3, stride=1),\n            torch.nn.Dropout(0.3),\n            torch.nn.BatchNorm2d(512),\n            torch.nn.ReLU(),\n            torch.nn.MaxPool2d(2),\n            \n            torch.nn.Conv2d(512,1024, kernel_size=3, stride=1),\n            torch.nn.Dropout(0.3),\n            torch.nn.BatchNorm2d(1024),\n            torch.nn.ReLU(),\n            torch.nn.MaxPool2d(2),\n            \n            torch.nn.Flatten(),\n            \n            torch.nn.Linear(1024, 512),\n            torch.nn.Dropout(0.3),\n            torch.nn.BatchNorm1d(512),\n            torch.nn.ReLU(),\n            \n            torch.nn.Linear(512, out_channels)\n            \n        )\n    \n    def forward(self, x):\n        x = self.model(x)\n        return x\n\n# _x = torch.Tensor(np.random.randn(4,1,96,96))\n# _m = ModelCNN()\n# _y = _m(_x)\n# _y.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:25.565591Z","iopub.execute_input":"2022-07-22T05:31:25.566781Z","iopub.status.idle":"2022-07-22T05:31:25.584362Z","shell.execute_reply.started":"2022-07-22T05:31:25.566734Z","shell.execute_reply":"2022-07-22T05:31:25.582959Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next we create a `Learner` class. This class is created simply to encapsulate all training related logic in one class.","metadata":{}},{"cell_type":"code","source":"class Learner:\n    def __init__(self, model, out_transforms):\n        self.model = model\n        self.out_transforms = out_transforms\n        \n    def configure_optimizers(self):\n        optimizer = torch.optim.Adam(self.model.parameters(), lr=0.0001)\n        return optimizer\n    \n#     def training_step(self, train_batch, batch_id):\n#         x,y = train_batch\n    \n    def loss_fn(self):\n        return torch.nn.MSELoss()\n        \n    \n    def train_one_epoch(self, loss_fn, optimizer, dataloader, device=\"cpu\"):        \n        self.model = self.model.to(device)\n        self.model.train()\n        N = len(dataloader)\n        losses = []\n        for x,y in tqdm_notebook(dataloader):\n            x = x.to(device)\n            y = y.to(device)\n            \n            y_pred = self.model(x)\n            loss = loss_fn(y_pred, y)\n            \n            optimizer.zero_grad()\n            loss.backward()\n            optimizer.step()\n            \n            _loss_val = float(loss.detach().cpu().numpy())\n            losses.append(_loss_val)\n            \n            print(f\"\\rTrainLoss:{_loss_val}\", end=\"\")\n        return losses, np.mean(losses)\n    \n    def eval_one_epoch(self, loss_fn, optimizer, dataloader, device=\"cpu\"):        \n        self.model = self.model.to(device)\n        self.model.eval()\n        N = len(dataloader)\n        losses = []\n        sample_idx = int(np.random.randint(0,N,1))\n        print(\"sample_idx\", sample_idx)\n        sample_x = None\n        sample_y = None\n        idx = 0\n        for x,y in tqdm_notebook(dataloader):\n            x = x.to(device)\n            y = y.to(device)\n            \n            y_pred = self.model(x)\n            loss = loss_fn(y_pred, y)\n            \n            _loss_val = float(loss.detach().cpu().numpy())\n            losses.append(_loss_val)\n            \n            if idx == sample_idx:\n                sample_x = x.detach().cpu().numpy()\n                sample_y = y.detach().cpu().numpy()\n            idx += 1\n            print(f\"\\rValLoss:{_loss_val}\", end=\"\")\n        return losses, np.mean(losses), sample_x, sample_y\n    \n    def fit(self, dl_train, dl_val, epochs, device, plot_freq=10):\n        loss_fn = self.loss_fn()\n        optimizer = self.configure_optimizers()\n        train_losses = []\n        val_losses = []\n        for epoch in range(epochs):\n            _, loss_train = self.train_one_epoch(loss_fn, optimizer, dl_train, device)\n            _, loss_val, sample_x, sample_y = self.eval_one_epoch(loss_fn, optimizer, dl_val, device)\n            \n            train_losses.append(loss_train)\n            val_losses.append(loss_val)\n            \n            print(f\"\\rTrainLoss:{loss_train} ValLoss:{loss_val}\")\n            \n            # Do some random predictions.\n            if (epoch+1)%plot_freq == 0:\n                self.plot_random_samples(sample_x, sample_y)\n                self.plot_losses(train_losses, val_losses)\n            print(f\"*********** Finished Epoch:{epoch+1}/{epochs} *************\")\n    \n    def plot_random_samples(self, sample_x, sample_y, ncols=None):\n        ncols = ncols if ncols is not None else len(sample_x)\n        nrows = int(np.ceil(len(sample_x)/ncols))\n        for trans in self.out_transforms:\n            sample_y = trans.inverse_transform(sample_y)\n        sample_y = sample_y.reshape(-1, sample_y.shape[1]//2, 2)\n        sample_x = sample_x*255\n        fig = plt.figure(figsize=(25,6))\n        for i in range(len(sample_x)):\n            img = sample_x[i]\n            keypoints = sample_y[i]\n            plt.subplot(nrows, ncols, i+1)\n            plt.scatter(np.squeeze(keypoints[:,0]), np.squeeze(keypoints[:,1]), c=\"red\", s=2)\n            plt.imshow(np.squeeze(img), cmap=\"gray\")\n        plt.show()\n    \n    def plot_losses(self, losses_train, losses_val):\n        # fig = plt.figure(figsize=())\n        plt.plot(losses_train, label=\"train\")\n        plt.plot(losses_val, label=\"val\")\n        plt.legend(loc=\"upper right\")\n        plt.show()\n        ","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:25.585865Z","iopub.execute_input":"2022-07-22T05:31:25.586191Z","iopub.status.idle":"2022-07-22T05:31:25.617282Z","shell.execute_reply.started":"2022-07-22T05:31:25.586165Z","shell.execute_reply":"2022-07-22T05:31:25.615702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally train the model. While training during each epoch few random samples are selected from test set and plotted along with predicted keypoints. This will help in visualizing how training is going on.","metadata":{}},{"cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n_learner = Learner(ModelCNN(), [scaler]) \n_learner.fit(dl_train, dl_val, epochs=20, device=device, plot_freq=5)\n        ","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:31:25.619368Z","iopub.execute_input":"2022-07-22T05:31:25.619787Z","iopub.status.idle":"2022-07-22T05:33:34.240204Z","shell.execute_reply.started":"2022-07-22T05:31:25.619748Z","shell.execute_reply":"2022-07-22T05:33:34.238669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Creating final `submission.csv`","metadata":{}},{"cell_type":"code","source":"col2index = dict([(col,i) for i, col in enumerate(df_training_csv.drop([\"Image\", \"Image_Numpy\"], axis=1).columns)])\nindex2col = dict([(i,col) for i, col in enumerate(df_training_csv.drop([\"Image\", \"Image_Numpy\"], axis=1).columns)])","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:33:34.241889Z","iopub.execute_input":"2022-07-22T05:33:34.242718Z","iopub.status.idle":"2022-07-22T05:33:34.252524Z","shell.execute_reply.started":"2022-07-22T05:33:34.242658Z","shell.execute_reply":"2022-07-22T05:33:34.251169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = _learner.model\nmodel = model.to(device)\nmodel.eval()\n\nrows = []\nfor i in tqdm_notebook(range(len(df_test_csv))):\n    img_id = df_test_csv.loc[i, \"ImageId\"]\n    img = df_test_csv.loc[i, \"Image_Numpy\"]\n    img = np.expand_dims(img, axis=0) # Add channel dimension\n    img = np.expand_dims(img, axis=0) # Add batch dimension\n    img = img / 255.0\n    img = torch.Tensor(img)\n    img = img.to(device)\n    keypoint_pred = model(img)\n    keypoint_pred = keypoint_pred.cpu().detach().numpy()\n    keypoint_pred = scaler.inverse_transform(keypoint_pred)\n    keypoint_pred = np.squeeze(keypoint_pred)\n    \n    for index, col in index2col.items():\n        location = keypoint_pred[index]\n        rows.append([img_id, col, location])\n    \n    \ndf_rows = pd.DataFrame(rows, columns=[\"ImageId\", \"FeatureName\", \"Location\"])","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:33:34.254150Z","iopub.execute_input":"2022-07-22T05:33:34.255482Z","iopub.status.idle":"2022-07-22T05:33:39.106251Z","shell.execute_reply.started":"2022-07-22T05:33:34.255428Z","shell.execute_reply":"2022-07-22T05:33:39.104723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_submission = pd.merge(df_lookup.drop(\"Location\", axis=1), df_rows, how=\"left\", on=[\"ImageId\", \"FeatureName\"])","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:33:39.110247Z","iopub.execute_input":"2022-07-22T05:33:39.110567Z","iopub.status.idle":"2022-07-22T05:33:39.143726Z","shell.execute_reply.started":"2022-07-22T05:33:39.110541Z","shell.execute_reply":"2022-07-22T05:33:39.142606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_submission_final = df_submission[[\"RowId\", \"Location\"]].reset_index(drop=True)\ndf_submission_final.to_csv(f\"/kaggle/working/submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:33:39.145228Z","iopub.execute_input":"2022-07-22T05:33:39.146206Z","iopub.status.idle":"2022-07-22T05:33:39.254343Z","shell.execute_reply.started":"2022-07-22T05:33:39.146163Z","shell.execute_reply":"2022-07-22T05:33:39.253152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Create submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-07-22T05:33:39.256247Z","iopub.execute_input":"2022-07-22T05:33:39.256716Z","iopub.status.idle":"2022-07-22T05:33:39.264398Z","shell.execute_reply.started":"2022-07-22T05:33:39.256676Z","shell.execute_reply":"2022-07-22T05:33:39.262778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Created `submission.csv` file could be used for submitting to the competition. Try submitting created file and see how you perform.","metadata":{}},{"cell_type":"markdown","source":"# Conclusion\n\nThis notebook presents getting started approach. To further enhance the solution you may try following approaches:\n\n1. Fine tune the model by adding more layers.\n2. Try tuning learning rate.\n3. Try tuning dropout rate.\n4. Use transfer learning.\n\n**If you find this notebook helpful then kindly support my work by upvoting and sharing this notebook.**","metadata":{}}]}