{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"GUIE: Pytorch Ensembling (Vit+Convnext_Xlarge)πŸ˜‰ \n\nCreated By Burhanuddin Latsaheb \n\n\n
GUIE: Pytorch Ensembling (Vit+Convnext_Xlarge)πŸ˜‰
\n\n##
If you find this notebook useful,support with an upvoteπŸ‘πŸ‘
\n","metadata":{}},{"cell_type":"markdown","source":"\nTable of Contents : \n\n- [1. Imports](#imports)\n- [2. Hyperparameters](#hyperparameters)\n-[3. List of pretrained models in Pytorch Image Library](#list)\n * [3.1 Examples](#Examples)\n- [4. Model Building](#model)\n * [4.1 Model Class](#model)\n * [4.2 Convnext Xlarge](#convnext_xlarge)\n * [4.3 Vision Transformer](#Vit)\n- [5. Model Ensembling](#ensemble)\n * [5.1. Ensemble Class](#em_class)\n * [5.2. Ensembling](#em_model)\n * [5.3. Saving the model](#saving)\n \n- [6. Submission](#submission)","metadata":{}},{"cell_type":"code","source":"!pip install timm","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-29T11:46:18.156324Z","iopub.execute_input":"2022-07-29T11:46:18.157162Z","iopub.status.idle":"2022-07-29T11:46:31.016726Z","shell.execute_reply.started":"2022-07-29T11:46:18.157038Z","shell.execute_reply":"2022-07-29T11:46:31.015678Z"},"_kg_hide-input":true,"_kg_hide-output":true,"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
Introduction
\nNotebook Overview : \n\n* This notebook contains ensembling of Vision transformer(Vit) and Convnext_xlarge model . \n* The models are loaded using the pytorch image library [(timm)](https://timm.fast.ai/) \n* The models are created using the `Pytorch` Library\n","metadata":{}},{"cell_type":"markdown","source":"\n **
1. Imports
**","metadata":{}},{"cell_type":"code","source":"import os\nimport cv2\nimport timm\nimport torch\nimport torchvision\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torchvision import models , transforms\nfrom zipfile import ZipFile\n\n","metadata":{"execution":{"iopub.status.busy":"2022-07-29T11:56:18.414776Z","iopub.execute_input":"2022-07-29T11:56:18.415518Z","iopub.status.idle":"2022-07-29T11:56:19.938626Z","shell.execute_reply.started":"2022-07-29T11:56:18.415481Z","shell.execute_reply":"2022-07-29T11:56:19.937276Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n **
2. Hyperparameters
**","metadata":{}},{"cell_type":"code","source":"class config : \n IMG_WIDTH = 224\n IMG_HEIGHT = 224\n SHUFFLE = True\n NUM_WORKERS = 2\n DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n NORMALIZE = True\n BATCH_BLENDING = True\n BACKBONE = [\"vit_small_r26_s32_224_in21k\" ,\"convnext_xlarge_in22k\"]","metadata":{"execution":{"iopub.status.busy":"2022-07-29T12:54:07.399359Z","iopub.execute_input":"2022-07-29T12:54:07.400525Z","iopub.status.idle":"2022-07-29T12:54:07.405811Z","shell.execute_reply.started":"2022-07-29T12:54:07.400481Z","shell.execute_reply":"2022-07-29T12:54:07.404906Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n **
3. List of pretrained models in Pytorch Image Library
**","metadata":{}},{"cell_type":"code","source":"avail_pretrained_models = timm.list_models(pretrained=True)\nprint(f\"The total number of models available in pytorch image library is {len(avail_pretrained_models)}\")","metadata":{"execution":{"iopub.status.busy":"2022-07-29T13:34:26.565019Z","iopub.execute_input":"2022-07-29T13:34:26.565460Z","iopub.status.idle":"2022-07-29T13:34:26.574524Z","shell.execute_reply.started":"2022-07-29T13:34:26.565428Z","shell.execute_reply":"2022-07-29T13:34:26.573352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n3.1. Examples :\n****\nFirst 10 available models :\n","metadata":{}},{"cell_type":"code","source":"avail_pretrained_models[:10]","metadata":{"execution":{"iopub.status.busy":"2022-07-29T13:37:21.018282Z","iopub.execute_input":"2022-07-29T13:37:21.018744Z","iopub.status.idle":"2022-07-29T13:37:21.026084Z","shell.execute_reply.started":"2022-07-29T13:37:21.018711Z","shell.execute_reply":"2022-07-29T13:37:21.025371Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n **
4. Model Building
**","metadata":{}},{"cell_type":"markdown","source":"\n4.1. Model Class :","metadata":{}},{"cell_type":"code","source":"class Model(nn.Module):\n def __init__(self , model_name , target_size = [config.IMG_HEIGHT , config.IMG_WIDTH] , normalize = True):\n super().__init__()\n self.target_size = target_size\n self.classifier = timm.create_model(model_name , pretrained = True , num_classes = 0)\n self.layer = nn.AdaptiveAvgPool1d(64)\n self.normalize = normalize\n \n def forward(self ,x):\n x = transforms.functional.resize(x , size = self.target_size)\n x = x/255.0\n x = transforms.functional.normalize(x , mean = [0.485 , 0.456 ,0.406] , std = [0.229 , 0.224 ,0.225])\n x = self.classifier(x) \n x = self.layer(x)\n if self.normalize:\n x = F.normalize(x)\n return x","metadata":{"execution":{"iopub.status.busy":"2022-07-29T12:21:50.636767Z","iopub.execute_input":"2022-07-29T12:21:50.637314Z","iopub.status.idle":"2022-07-29T12:21:50.644474Z","shell.execute_reply.started":"2022-07-29T12:21:50.637284Z","shell.execute_reply":"2022-07-29T12:21:50.643677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n4.2. Convnext Xlarge :","metadata":{}},{"cell_type":"code","source":"model = Model(\"convnext_xlarge_in22k\" ,normalize = True)\nmodel.eval()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T12:23:38.706689Z","iopub.execute_input":"2022-07-29T12:23:38.707632Z","iopub.status.idle":"2022-07-29T12:24:52.893809Z","shell.execute_reply.started":"2022-07-29T12:23:38.707596Z","shell.execute_reply":"2022-07-29T12:24:52.889931Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n4.3. Vision Transformer :","metadata":{}},{"cell_type":"code","source":"model_2 = Model(\"vit_small_r26_s32_224_in21k\" , normalize = True)\nmodel_2.eval()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T14:05:56.909868Z","iopub.execute_input":"2022-07-29T14:05:56.910383Z","iopub.status.idle":"2022-07-29T14:05:58.179775Z","shell.execute_reply.started":"2022-07-29T14:05:56.910350Z","shell.execute_reply":"2022-07-29T14:05:58.178553Z"},"_kg_hide-input":true,"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n **
5. Model Ensembling
**","metadata":{}},{"cell_type":"markdown","source":"\n5.1. Ensemble Class :","metadata":{}},{"cell_type":"code","source":"class Ensemble(nn.Module):\n def __init__(self , encoders , normalize = True):\n super().__init__()\n for index , encoder in enumerate(encoders):\n setattr(self,f\"encoder{index}\" , encoder)\n self.num_encoders = len(encoders)\n self.normalize = normalize\n \n def forward(self,x):\n output = []\n for name , encoder in self.named_children():\n output.append(encoder(x))\n output = torch.cat(output , dim =0)\n output = F.normalize(output)\n output = output.mean(dim = 0).unsqueeze(0)\n if self.normalize:\n output = F.normalize(output)\n return output\n \n ","metadata":{"execution":{"iopub.status.busy":"2022-07-29T14:08:26.522791Z","iopub.execute_input":"2022-07-29T14:08:26.523278Z","iopub.status.idle":"2022-07-29T14:08:26.531780Z","shell.execute_reply.started":"2022-07-29T14:08:26.523242Z","shell.execute_reply":"2022-07-29T14:08:26.530810Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n5.2. Ensembling :","metadata":{}},{"cell_type":"code","source":"encoders = []\nencoders.append(model)\nencoders.append(model_2)\nensemble_model = Ensemble(encoders , normalize = True)\nensemble_model.eval()","metadata":{"execution":{"iopub.status.busy":"2022-07-29T14:16:59.772077Z","iopub.execute_input":"2022-07-29T14:16:59.772495Z","iopub.status.idle":"2022-07-29T14:16:59.793637Z","shell.execute_reply.started":"2022-07-29T14:16:59.772462Z","shell.execute_reply":"2022-07-29T14:16:59.792375Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ensemble_model(torch.randn((1,3,224,224))).shape","metadata":{"execution":{"iopub.status.busy":"2022-07-29T14:17:19.485021Z","iopub.execute_input":"2022-07-29T14:17:19.485944Z","iopub.status.idle":"2022-07-29T14:17:20.995200Z","shell.execute_reply.started":"2022-07-29T14:17:19.485906Z","shell.execute_reply":"2022-07-29T14:17:20.993925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n5.3. Saving the model :","metadata":{}},{"cell_type":"code","source":"saved_model = torch.jit.script(ensemble_model)\nsaved_model.save('saved_model.pt')","metadata":{"execution":{"iopub.status.busy":"2022-07-29T13:04:10.522132Z","iopub.execute_input":"2022-07-29T13:04:10.522856Z","iopub.status.idle":"2022-07-29T13:04:17.532538Z","shell.execute_reply.started":"2022-07-29T13:04:10.522816Z","shell.execute_reply":"2022-07-29T13:04:17.531638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n **
6. Submission
**","metadata":{}},{"cell_type":"code","source":"from zipfile import ZipFile\n\nwith ZipFile('submission.zip','w') as zip: \n zip.write('saved_model.pt', arcname='saved_model.pt') ","metadata":{"execution":{"iopub.status.busy":"2022-07-29T13:04:17.534024Z","iopub.execute_input":"2022-07-29T13:04:17.534930Z","iopub.status.idle":"2022-07-29T13:04:26.167997Z","shell.execute_reply.started":"2022-07-29T13:04:17.534897Z","shell.execute_reply":"2022-07-29T13:04:26.166679Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
Thank You for reading 😊
\n
More versions of this notebook will be available during the course of this competition
\n
If you have any suggestions or feeback, please let me know
","metadata":{}}]}