\nHighlights \nโ\nโ\n \nIt's a good train pipeline for Image Classification problem that uses PyTorch Lightning and PyTorch Image Models. \nI used this pipeline for my job tasks and I think It will be useful for you too. \n \nIn this notebook I will show you: \n๐ฆ 1. What PyTorch Lightning is. What blocks PyTorch Lightning consists of. \n๐ 2. How to prepare you data before training. Spoiler - It's very easy. \n๐ช 3. How to use PyTorch Image Models Library to choose a network for training. \n๐ ๏ธ 4. How to configure your training parameters. \n๐ผ๏ธ 5. How to add preprocessing and augmentations. \n๐พ 6. How to save results. \n \n\n
References:
\n\n\n> [PyTorch Lighnting](https://www.pytorchlightning.ai/) \n> [PyTorch Image Models](https://github.com/rwightman/pytorch-image-models) \n\n\n \n\nโ\nOK, Let's go! ๐ ","metadata":{}},{"cell_type":"markdown","source":"Several words about Google Landmarks Recognition Competition\n\nIt's a pretty compicated task because we have more than 80k classes and about 100Gb of data. It can be very hard to train such a dataset if you don't have your own GPU capacities. \n\nI have Google Colab Pro and I can't upload this dataset because of lack of storage volume ๐ฅบ \n\nIt's obvious that in this situation we should train not all classes but the most frequent. ","metadata":{}},{"cell_type":"markdown","source":"Install and Import Libraries\n\nThere is nothing special. Install PyTorch Lightning and PyTorch Image Models. \n ","metadata":{}},{"cell_type":"code","source":"!pip install '/kaggle/input/pytorch-lightning142/pytorch_lightning-1.4.2-py3-none-any.whl' --no-deps\n!pip install '/kaggle/input/pytorchimagemodels/pytorch-image-models-master' --no-deps","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-08-25T04:59:48.921607Z","iopub.execute_input":"2021-08-25T04:59:48.921983Z","iopub.status.idle":"2021-08-25T04:59:59.302497Z","shell.execute_reply.started":"2021-08-25T04:59:48.921897Z","shell.execute_reply":"2021-08-25T04:59:59.30156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pytorch_lightning as pl\nimport timm\nimport albumentations as A\nimport numpy as np\nimport pickle\nimport os\nimport cv2\nimport torchmetrics\nimport torch\nimport datetime\nimport pandas as pd\n\n\nfrom tqdm import tqdm_notebook as tqdm\n\nfrom pathlib import Path\nfrom torch.utils import data\nfrom torch import nn\nfrom torch.utils.data import DataLoader, WeightedRandomSampler\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.model_selection import train_test_split\nfrom albumentations.pytorch import ToTensorV2\nfrom pytorch_lightning.callbacks.early_stopping import EarlyStopping\nfrom pytorch_lightning.loggers import TensorBoardLogger\nfrom pytorch_lightning.callbacks import ModelCheckpoint\nfrom pytorch_lightning.metrics.functional import accuracy","metadata":{"execution":{"iopub.status.busy":"2021-08-25T04:59:59.305997Z","iopub.execute_input":"2021-08-25T04:59:59.306272Z","iopub.status.idle":"2021-08-25T05:00:04.154641Z","shell.execute_reply.started":"2021-08-25T04:59:59.306238Z","shell.execute_reply":"2021-08-25T05:00:04.153737Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Models List\n\nNext cell is just a list of models that you can use for traning. \nYou can add other models from PyTorch Image Models. I will show you later how you can do it.","metadata":{}},{"cell_type":"code","source":"MODELS = [\n 'adv_inception_v3',\n 'cspdarknet53',\n 'cspresnet50',\n 'cspresnext50',\n 'densenet121',\n 'densenet161',\n 'densenet169',\n 'densenet201',\n 'densenetblur121d',\n 'dla102',\n 'dla102x',\n 'dla102x2',\n 'dla169',\n 'dla34',\n 'dla46_c',\n 'dla46x_c',\n 'dla60_res2net',\n 'dla60_res2next',\n 'dla60',\n 'dla60x_c',\n 'dla60x',\n 'dm_nfnet_f0',\n 'dm_nfnet_f1',\n 'dm_nfnet_f2',\n 'dm_nfnet_f3',\n 'dm_nfnet_f4',\n 'dm_nfnet_f5',\n 'dm_nfnet_f6',\n 'dpn107',\n 'dpn131',\n 'dpn68',\n 'dpn68b',\n 'dpn92',\n 'dpn98',\n 'ecaresnet101d_pruned',\n 'ecaresnet101d',\n 'ecaresnet269d',\n 'ecaresnet26t',\n 'ecaresnet50d_pruned',\n 'ecaresnet50d',\n 'ecaresnet50t',\n 'ecaresnetlight',\n 'efficientnet_b0',\n 'efficientnet_b1_pruned',\n 'efficientnet_b1',\n 'efficientnet_b2',\n 'efficientnet_b2a',\n 'efficientnet_b3_pruned',\n 'efficientnet_b3',\n 'efficientnet_b3a',\n 'efficientnet_em',\n 'efficientnet_es',\n 'efficientnet_lite0',\n 'ens_adv_inception_resnet_v2',\n 'ese_vovnet19b_dw',\n 'ese_vovnet39b',\n 'fbnetc_100',\n 'gernet_l',\n 'gernet_m',\n 'gernet_s',\n 'gluon_inception_v3',\n 'gluon_resnet101_v1b',\n 'gluon_resnet101_v1c',\n 'gluon_resnet101_v1d',\n 'gluon_resnet101_v1s',\n 'gluon_resnet152_v1b',\n 'gluon_resnet152_v1c',\n 'gluon_resnet152_v1d',\n 'gluon_resnet152_v1s',\n 'gluon_resnet18_v1b',\n 'gluon_resnet34_v1b',\n 'gluon_resnet50_v1b',\n 'gluon_resnet50_v1c',\n 'gluon_resnet50_v1d',\n 'gluon_resnet50_v1s',\n 'gluon_resnext101_32x4d',\n 'gluon_resnext101_64x4d',\n 'gluon_resnext50_32x4d',\n 'gluon_senet154',\n 'gluon_seresnext101_32x4d',\n 'gluon_seresnext101_64x4d',\n 'gluon_seresnext50_32x4d',\n 'gluon_xception65',\n 'hrnet_w18_small_v2',\n 'hrnet_w18_small',\n 'hrnet_w18',\n 'hrnet_w30',\n 'hrnet_w32',\n 'hrnet_w40',\n 'hrnet_w44',\n 'hrnet_w48',\n 'hrnet_w64',\n 'ig_resnext101_32x16d',\n 'ig_resnext101_32x32d',\n 'ig_resnext101_32x48d',\n 'ig_resnext101_32x8d',\n 'inception_resnet_v2',\n 'inception_v3',\n 'inception_v4',\n 'legacy_senet154',\n 'legacy_seresnet101',\n 'legacy_seresnet152',\n 'legacy_seresnet18',\n 'legacy_seresnet34',\n 'legacy_seresnet50',\n 'legacy_seresnext101_32x4d',\n 'legacy_seresnext26_32x4d',\n 'legacy_seresnext50_32x4d',\n 'mixnet_l',\n 'mixnet_m',\n 'mixnet_s',\n 'mixnet_xl',\n 'mnasnet_100',\n 'mobilenetv2_100',\n 'mobilenetv2_110d',\n 'mobilenetv2_120d',\n 'mobilenetv2_140',\n 'mobilenetv3_large_100',\n 'mobilenetv3_rw',\n 'nasnetalarge',\n 'nf_regnet_b1',\n 'nf_resnet50',\n 'nfnet_l0c',\n 'pnasnet5large',\n 'regnetx_002',\n 'regnetx_004',\n 'regnetx_006',\n 'regnetx_008',\n 'regnetx_016',\n 'regnetx_032',\n 'regnetx_040',\n 'regnetx_064',\n 'regnetx_080',\n 'regnetx_120',\n 'regnetx_160',\n 'regnetx_320',\n 'regnety_002',\n 'regnety_004',\n 'regnety_006',\n 'regnety_008',\n 'regnety_016',\n 'regnety_032',\n 'regnety_040',\n 'regnety_064',\n 'regnety_080',\n 'regnety_120',\n 'regnety_160',\n 'regnety_320',\n 'repvgg_a2',\n 'repvgg_b0',\n 'repvgg_b1',\n 'repvgg_b1g4',\n 'repvgg_b2',\n 'repvgg_b2g4',\n 'repvgg_b3',\n 'repvgg_b3g4',\n 'res2net101_26w_4s',\n 'res2net50_14w_8s',\n 'res2net50_26w_4s',\n 'res2net50_26w_6s',\n 'res2net50_26w_8s',\n 'res2net50_48w_2s',\n 'res2next50',\n 'resnest101e',\n 'resnest14d',\n 'resnest200e',\n 'resnest269e',\n 'resnest26d',\n 'resnest50d_1s4x24d',\n 'resnest50d_4s2x40d',\n 'resnest50d',\n 'resnet101d',\n 'resnet152d',\n 'resnet18',\n 'resnet18d',\n 'resnet200d',\n 'resnet26',\n 'resnet26d',\n 'resnet34',\n 'resnet34d',\n 'resnet50',\n 'resnet50d',\n 'resnetblur50',\n 'resnetv2_101x1_bitm_in21k',\n 'resnetv2_101x1_bitm',\n 'resnetv2_101x3_bitm_in21k',\n 'resnetv2_101x3_bitm',\n 'resnetv2_152x2_bitm_in21k',\n 'resnetv2_152x2_bitm',\n 'resnetv2_152x4_bitm_in21k',\n 'resnetv2_152x4_bitm',\n 'resnetv2_50x1_bitm_in21k',\n 'resnetv2_50x1_bitm',\n 'resnetv2_50x3_bitm_in21k',\n 'resnetv2_50x3_bitm',\n 'resnext101_32x8d',\n 'resnext50_32x4d',\n 'resnext50d_32x4d',\n 'rexnet_100',\n 'rexnet_130',\n 'rexnet_150',\n 'rexnet_200',\n 'selecsls42b',\n 'selecsls60',\n 'selecsls60b',\n 'semnasnet_100',\n 'seresnet152d',\n 'seresnet50',\n 'seresnext26d_32x4d',\n 'seresnext26t_32x4d',\n 'seresnext50_32x4d',\n 'skresnet18',\n 'skresnet34',\n 'skresnext50_32x4d',\n 'spnasnet_100',\n 'ssl_resnet18',\n 'ssl_resnet50',\n 'ssl_resnext101_32x16d',\n 'ssl_resnext101_32x4d',\n 'ssl_resnext101_32x8d',\n 'ssl_resnext50_32x4d',\n 'swsl_resnet18',\n 'swsl_resnet50',\n 'swsl_resnext101_32x16d',\n 'swsl_resnext101_32x4d',\n 'swsl_resnext101_32x8d',\n 'swsl_resnext50_32x4d',\n 'tf_efficientnet_b0_ap',\n 'tf_efficientnet_b0_ns',\n 'tf_efficientnet_b0',\n 'tf_efficientnet_b1_ap',\n 'tf_efficientnet_b1_ns',\n 'tf_efficientnet_b1',\n 'tf_efficientnet_b2_ap',\n 'tf_efficientnet_b2_ns',\n 'tf_efficientnet_b2',\n 'tf_efficientnet_b3_ap',\n 'tf_efficientnet_b3_ns',\n 'tf_efficientnet_b3',\n 'tf_efficientnet_b4_ap',\n 'tf_efficientnet_b4_ns',\n 'tf_efficientnet_b4',\n 'tf_efficientnet_b5_ap',\n 'tf_efficientnet_b5_ns',\n 'tf_efficientnet_b5',\n 'tf_efficientnet_b6_ap',\n 'tf_efficientnet_b6_ns',\n 'tf_efficientnet_b6',\n 'tf_efficientnet_b7_ap',\n 'tf_efficientnet_b7_ns',\n 'tf_efficientnet_b7',\n 'tf_efficientnet_b8_ap',\n 'tf_efficientnet_b8',\n 'tf_efficientnet_cc_b0_4e',\n 'tf_efficientnet_cc_b0_8e',\n 'tf_efficientnet_cc_b1_8e',\n 'tf_efficientnet_el',\n 'tf_efficientnet_em',\n 'tf_efficientnet_es',\n 'tf_efficientnet_l2_ns_475',\n 'tf_efficientnet_l2_ns',\n 'tf_efficientnet_lite0',\n 'tf_efficientnet_lite1',\n 'tf_efficientnet_lite2',\n 'tf_efficientnet_lite3',\n 'tf_efficientnet_lite4',\n 'tf_inception_v3',\n 'tf_mixnet_l',\n 'tf_mixnet_m',\n 'tf_mixnet_s',\n 'tf_mobilenetv3_large_075',\n 'tf_mobilenetv3_large_100',\n 'tf_mobilenetv3_large_minimal_100',\n 'tf_mobilenetv3_small_075',\n 'tf_mobilenetv3_small_100',\n 'tf_mobilenetv3_small_minimal_100',\n 'tresnet_l_448',\n 'tresnet_l',\n 'tresnet_m_448',\n 'tresnet_m',\n 'tresnet_xl_448',\n 'tresnet_xl',\n 'tv_densenet121',\n 'tv_resnet101',\n 'tv_resnet152',\n 'tv_resnet34',\n 'tv_resnet50',\n 'tv_resnext50_32x4d',\n 'vgg11_bn',\n 'vgg11',\n 'vgg13_bn',\n 'vgg13',\n 'vgg16_bn',\n 'vgg16',\n 'vgg19_bn',\n 'vgg19',\n 'vit_base_patch16_224_in21k',\n 'vit_base_patch16_224',\n 'vit_base_patch16_384',\n 'vit_base_patch32_224_in21k',\n 'vit_base_patch32_384',\n 'vit_base_resnet50_224_in21k',\n 'vit_base_resnet50_384',\n 'vit_deit_base_distilled_patch16_224',\n 'vit_deit_base_distilled_patch16_384',\n 'vit_deit_base_patch16_224',\n 'vit_deit_base_patch16_384',\n 'vit_deit_small_distilled_patch16_224',\n 'vit_deit_small_patch16_224',\n 'vit_deit_tiny_distilled_patch16_224',\n 'vit_deit_tiny_patch16_224',\n 'vit_large_patch16_224_in21k',\n 'vit_large_patch16_224',\n 'vit_large_patch16_384',\n 'vit_large_patch32_224_in21k',\n 'vit_large_patch32_384',\n 'vit_small_patch16_224',\n 'wide_resnet101_2',\n 'wide_resnet50_2',\n 'xception',\n 'xception41',\n 'xception65',\n 'xception71',\n 'tf_efficientnetv2_b0',\n 'tf_efficientnetv2_l',\n 'cait_m36_384',\n 'cait_m48_448',\n 'cait_s24_224',\n 'cait_s24_384',\n 'cait_s36_384',\n 'cait_xs24_384',\n 'cait_xxs24_224',\n 'cait_xxs24_384',\n 'cait_xxs36_224',\n 'cait_xxs36_384',\n 'coat_lite_mini',\n 'coat_lite_small',\n 'coat_lite_tiny',\n 'coat_mini',\n 'coat_tiny',\n 'convit_base',\n 'convit_small',\n 'convit_tiny',\n 'deit_base_distilled_patch16_224',\n 'eca_efficientnet_b0',\n 'efficientnet_b2_pruned',\n 'efficientnet_b4',\n 'efficientnet_b5',\n 'efficientnet_b6',\n 'efficientnet_b7',\n 'efficientnet_b8',\n 'efficientnet_cc_b0_4e',\n 'efficientnet_cc_b0_8e',\n 'efficientnet_cc_b1_8e',\n 'efficientnet_el',\n 'efficientnet_el_pruned',\n 'efficientnet_es_pruned',\n 'efficientnet_l2',\n 'efficientnet_lite1',\n 'efficientnet_lite2',\n 'efficientnet_lite3',\n 'efficientnet_lite4',\n 'efficientnetv2_l',\n 'efficientnetv2_m',\n 'efficientnetv2_rw_m',\n 'efficientnetv2_rw_s',\n 'efficientnetv2_s',\n 'gc_efficientnet_b0',\n]","metadata":{"execution":{"iopub.status.busy":"2021-08-25T05:00:04.156603Z","iopub.execute_input":"2021-08-25T05:00:04.156954Z","iopub.status.idle":"2021-08-25T05:00:04.180652Z","shell.execute_reply.started":"2021-08-25T05:00:04.156919Z","shell.execute_reply":"2021-08-25T05:00:04.178383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Pytorch Lightning\n\nBefore we continue I should say several words about PyTorch Lightning. \n\nPyTorch Lightning is a library that wraps PyTorch and makes training process easier. In this case, you are sacrificing flexibility for convenience. \n\nYou can read more about PyTorch Lightning [here](https://www.pytorchlightning.ai/). \n\nPyTorch Lightning consists of folowing blocks: \n1. Dataset - how you process every image of your dataset. \n2. DataModule - in this block you prepare your data before training. Set dataloaders. \n3. Model - in this block you configure a model, Set schedulers and optimizers. \n4. Train - it's my own block. It's a main training pipeline. \n \n\nThis isn't a strict gradation, you can add your own blocks and modify existing ones. ","metadata":{}},{"cell_type":"markdown","source":"Dataset \nThis is how you process a pair \"image - label\" of your dataset. ","metadata":{}},{"cell_type":"code","source":"class ICPDataset(data.Dataset):\n def __init__(self,\n data,\n input_resize,\n augments=None,\n preprocessing=None):\n super().__init__()\n self.imgs, self.labels = data\n self.input_resize = (input_resize, input_resize) \n self.augments = augments\n self.preprocessing = preprocessing\n\n def __len__(self):\n return len(self.imgs)\n\n def __getitem__(self, idx):\n img_path = str(self.imgs[idx])\n label = self.labels[idx]\n\n img = cv2.imread(img_path, cv2.IMREAD_COLOR)\n img = cv2.resize(img, self.input_resize, interpolation=cv2.INTER_NEAREST)\n\n if self.augments: # add augmentation\n augmented = self.augments(image=img)\n img = augmented['image']\n\n if self.preprocessing: # add preprocessing\n preprocessed = self.preprocessing(image=img)\n img = preprocessed['image']\n\n return img, label","metadata":{"execution":{"iopub.status.busy":"2021-08-25T05:00:04.183373Z","iopub.execute_input":"2021-08-25T05:00:04.183643Z","iopub.status.idle":"2021-08-25T05:00:04.198122Z","shell.execute_reply.started":"2021-08-25T05:00:04.183616Z","shell.execute_reply":"2021-08-25T05:00:04.197358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"DataModule \n\nThere is a lot of code. I added comments. ","metadata":{}},{"cell_type":"code","source":"class ICPDataModule(pl.LightningDataModule):\n def __init__(self, model_type,\n batch_size,\n data_dir,\n input_resize,\n input_resize_test,\n mean,\n std,\n augment_p=0.7,\n images_ext='jpg'):\n super().__init__()\n self.data_dir = data_dir\n self.batch_size = batch_size\n self.input_resize = input_resize\n self.input_resize_test = input_resize_test\n self.mean = mean,\n self.std = std,\n self.augment_p = augment_p\n self.images_ext = images_ext\n\n # get preprocessing and augmentation\n transforms_composed = self._get_transforms()\n self.augments, self.preprocessing = transforms_composed\n\n self.dataset_train, self.dataset_val, self.dataset_test = None, None, None\n\n # dataset transforms: normalize and toTensor\n def _get_transforms(self):\n transforms = []\n\n if self.mean is not None:\n transforms += [A.Normalize(mean=self.mean, std=self.std)]\n\n transforms += [ToTensorV2(transpose_mask=True)]\n preprocessing = A.Compose(transforms)\n\n return self._get_train_transforms(self.augment_p), preprocessing\n\n # dataset augmentation. I used albumentation library. You can add your own augmentations. \n def _get_train_transforms(self, p):\n return A.Compose([\n A.Rotate(limit=35, p=1.0),\n A.HorizontalFlip(p=0.5),\n A.VerticalFlip(p=0.5),\n A.Transpose(p=0.3),\n A.GaussNoise(p=0.4),\n A.OneOf([A.MotionBlur(p=0.5),\n A.MedianBlur(blur_limit=3, p=0.5),\n A.Blur(blur_limit=3, p=0.1)], p=0.5),\n A.OneOf([A.CLAHE(clip_limit=2),\n A.Sharpen(),\n A.Emboss(),\n A.RandomBrightnessContrast()], p=0.5),\n ], p=p)\n \n \n # It's a main procedure for preparing your data. I have writen it especially for landmark-recognition-2021 dataset. \n # Below this procedure there is another one. You should check it too. \n def setup(self, stage=None):\n\n path = Path(self.data_dir)\n\n train_df = pd.read_csv('/kaggle/input/landmark-recognition-2021/train.csv')\n landmark = train_df.landmark_id.value_counts()\n # we take only 5 most frequent classes. Your can change count of classes - for example to 1000. \n l = landmark[:5].index.values\n\n freq_landmarks_df = train_df[train_df['landmark_id'].isin(l)]\n image_ids = freq_landmarks_df['id'].tolist()\n landmark_ids = freq_landmarks_df['landmark_id'].tolist()\n \n # convert from classes to codes 0, 1, 2, 3, ... etc.\n label_encoder = LabelEncoder()\n encoded = label_encoder.fit_transform(l)\n \n self.num_classes = len(np.unique(encoded))\n\n # save labels dict to file. We will use this file during inference. \n with open('/kaggle/working/label_encoder.pkl', 'wb') as le_dump_file:\n pickle.dump(label_encoder, le_dump_file)\n\n # mapping classes and codes\n mapping = dict(zip(label_encoder.classes_, range(len(label_encoder.classes_))))\n \n # image_ids landmark_ids dict\n im_land_dict = dict((k,i) for k,i in zip(image_ids, landmark_ids))\n \n # get paths of all images in dataset\n print('Unpacking images...')\n path_list = [os.path.join(dirpath,filename) for dirpath, _, filenames in os.walk(str(path)) for filename in filenames if filename.endswith('.jpg')]\n \n # get filenames from paths\n filenames = []\n for path in tqdm(path_list):\n filename, _ = os.path.splitext(path.split('/')[-1])\n filenames.append(filename)\n \n # find intersection of images filenames of our frequent classes and all filenames\n ind_dict = dict((k,i) for i,k in enumerate(filenames))\n inter = set(ind_dict).intersection(image_ids)\n indices = [ind_dict[x] for x in inter]\n \n # find paths of images of our frequent classes\n image_ids_paths = []\n for ind in indices:\n image_ids_paths.append(path_list[ind])\n\n # find landmarks ids for our images\n labels_ids = []\n for img in tqdm(image_ids_paths):\n filename, _ = os.path.splitext(img.split('/')[-1])\n land_id = im_land_dict[filename]\n labels_ids.append(mapping[int(land_id)])\n \n # you can set classes_weights but I skipped this step\n self.classes_weights = None\n\n image_ids_paths = [Path(p) for p in image_ids_paths]\n\n # set train images and labels\n train_files, val_test_files, train_labels, val_test_labels = train_test_split(image_ids_paths, labels_ids,\n test_size=0.3, random_state=42,\n stratify=landmark_ids)\n print(f'train_files: {len(train_files)}, train_labels: {len(train_labels)}')\n \n train_data = train_files, train_labels\n\n # set val and test images and labels\n val_files, test_files, val_labels, test_labels = train_test_split(val_test_files, val_test_labels,\n test_size=0.5, random_state=42,\n stratify=val_test_labels)\n\n print(f'val_files: {len(val_files)}, val_labels: {len(val_labels)}')\n val_data = val_files, val_labels\n \n print(f'test_files: {len(test_files)}, test_labels: {len(test_labels)}')\n test_data = test_files, test_labels\n \n\n self.sampler = None\n # self.sampler = WeightedRandomSampler(sample_weights, num_samples=len(sample_weights), replacement=True)\n\n if stage == 'fit' or stage is None:\n self.dataset_train = ICPDataset(\n data=train_data,\n input_resize=self.input_resize,\n augments=self.augments,\n preprocessing=self.preprocessing)\n\n # notice that we don't add augments for val dataset but only for training\n self.dataset_val = ICPDataset(\n data=val_data,\n input_resize=self.input_resize,\n preprocessing=self.preprocessing)\n\n self.dims = tuple(self.dataset_train[0][0].shape)\n\n # Assign test dataset for use in dataloader(s)\n if stage == 'test' or stage is None:\n self.dataset_test = ICPDataset(\n data=test_data,\n input_resize=self.input_resize_test,\n preprocessing=self.preprocessing)\n\n self.dims = tuple(self.dataset_test[0][0].shape)\n\n\n# !!!ATENTION!!! Below there is another setup procedure. If you plan to use this pipeline for your Image Classification problems \n# just put images of every class to separate folders with names of classes. \n# Uncomment this procedure and everything will work.\n\n# Example of folder structure:\n# -- animals\n# -- cat\n# -- cat1.png\n# -- cat2.png\n# -- cat3.png\n# -- dog\n# -- dog1.png\n# -- dog2.png\n# -- dog3.png\n\n# You can see such folder structure in \"input\" section on the right side: folder - simpsons. \n\n\n# def setup(self, stage=None):\n# # Assign train/val datasets for use in dataloaders\n\n# path = Path(self.data_dir)\n\n# train_val_files = list(path.rglob('*.' + self.images_ext))\n# train_val_labels = [path.parent.name for path in train_val_files]\n\n# label_encoder = LabelEncoder()\n# encoded = label_encoder.fit_transform(train_val_labels)\n# self.num_classes = len(np.unique(encoded))\n\n# # save labels dict to file\n# with open('label_encoder.pkl', 'wb') as le_dump_file:\n# pickle.dump(label_encoder, le_dump_file)\n\n# train_files, val_test_files = train_test_split(train_val_files, test_size=0.3, stratify=train_val_labels)\n\n# train_labels = [path.parent.name for path in train_files]\n# train_labels = label_encoder.transform(train_labels)\n# train_data = train_files, train_labels\n\n# class_weights = []\n# count_all_files = 0\n# for root, subdir, files in os.walk(self.data_dir):\n# if len(files) > 0:\n# class_weights.append(len(files))\n# count_all_files += len(files)\n\n# self.classes_weights = [x / count_all_files for x in class_weights]\n# print('classes_weights', self.classes_weights)\n\n# sample_weights = [0] * len(train_files)\n\n# for idx, (data, label) in enumerate(zip(train_files, train_labels)):\n# class_weight = self.classes_weights[label]\n# sample_weights[idx] = class_weight\n\n# self.sampler = WeightedRandomSampler(sample_weights, num_samples=len(sample_weights), replacement=True)\n\n# # self.classes_weights = [round(x / sum(list(Counter(sorted(train_labels)).values())), 2) for x in\n# # list(Counter(sorted(train_labels)).values())]\n\n# # without test step\n# # val_labels = [path.parent.name for path in val_test_files]\n# # val_labels = label_encoder.transform(val_labels)\n# # val_data = val_test_files, val_labels\n\n# # with test step\n# val_test_labels = [path.parent.name for path in val_test_files]\n# val_files, test_files = train_test_split(val_test_files, test_size=0.5, stratify=val_test_labels)\n\n# val_labels = [path.parent.name for path in val_files]\n# val_labels = label_encoder.transform(val_labels)\n\n# test_labels = [path.parent.name for path in test_files]\n# test_labels = label_encoder.transform(test_labels)\n\n# val_data = val_files, val_labels\n# test_data = test_files, test_labels\n\n# if stage == 'fit' or stage is None:\n# self.dataset_train = ICPDataset(\n# data=train_data,\n# input_resize=self.input_resize,\n# augments=self.augments,\n# preprocessing=self.preprocessing)\n\n# self.dataset_val = ICPDataset(\n# data=val_data,\n# input_resize=self.input_resize,\n# preprocessing=self.preprocessing)\n\n# self.dims = tuple(self.dataset_train[0][0].shape)\n\n# # Assign test dataset for use in dataloader(s)\n# if stage == 'test' or stage is None:\n# self.dataset_test = ICPDataset(\n# data=test_data,\n# input_resize=self.input_resize_test,\n# preprocessing=self.preprocessing)\n\n# self.dims = tuple(self.dataset_test[0][0].shape)\n\n def train_dataloader(self):\n if self.sampler:\n loader = DataLoader(self.dataset_train, batch_size=self.batch_size, sampler=self.sampler, num_workers=4)\n else:\n loader = DataLoader(self.dataset_train, batch_size=self.batch_size, shuffle=True, num_workers=4)\n\n return loader\n\n def val_dataloader(self):\n return DataLoader(self.dataset_val, batch_size=self.batch_size, num_workers=4)\n\n def test_dataloader(self):\n return DataLoader(self.dataset_test, batch_size=self.batch_size, num_workers=4)\n","metadata":{"execution":{"iopub.status.busy":"2021-08-25T10:02:08.57964Z","iopub.execute_input":"2021-08-25T10:02:08.579976Z","iopub.status.idle":"2021-08-25T10:02:08.613179Z","shell.execute_reply.started":"2021-08-25T10:02:08.579943Z","shell.execute_reply":"2021-08-25T10:02:08.612274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Model \n\nThis is a module where we define our model, loss function, optimizer, scheduler, train/val/test steps.
\nBelow you will see a big \"if-elif-else\" construction with models names. It's because the models have different name for their last fully-connected layer where we should change default count of classes to our. \nIn some models it is called \"fc\" or \"head\" or \"classifier\". When we create model for training we will only pass it's name and count of classes and this \"if-elif-else\" construction creates valid model for us. ","metadata":{}},{"cell_type":"code","source":"class ICPModel(pl.LightningModule):\n def __init__(self,\n model_type,\n num_classes,\n classes_weights,\n learning_rate=0.0001):\n super().__init__()\n\n # log hyperparameters\n self.save_hyperparameters()\n\n self.learning_rate = learning_rate\n self.num_classes = num_classes\n self.model_type = model_type\n\n # load network\n if self.model_type in ['densenet121', # classifier\n 'densenet161',\n 'densenet169',\n 'densenet201',\n 'densenetblur121d',\n 'efficientnet_b0',\n 'efficientnet_b1',\n 'efficientnet_b1_pruned',\n 'efficientnet_b2',\n 'efficientnet_b2a',\n 'efficientnet_b3',\n 'efficientnet_b3_pruned',\n 'efficientnet_b3a',\n 'efficientnet_em',\n 'efficientnet_es',\n 'efficientnet_lite0',\n 'fbnetc_100',\n 'hrnet_w18',\n 'hrnet_w18_small',\n 'hrnet_w18_small_v2',\n 'hrnet_w30',\n 'hrnet_w32',\n 'hrnet_w40',\n 'hrnet_w44',\n 'hrnet_w48',\n 'hrnet_w64',\n 'mixnet_l',\n 'mixnet_m',\n 'mixnet_s',\n 'mixnet_xl',\n 'mnasnet_100',\n 'mobilenetv2_100',\n 'mobilenetv2_110d',\n 'mobilenetv2_120d',\n 'mobilenetv2_140',\n 'mobilenetv3_large_100',\n 'mobilenetv3_rw',\n 'semnasnet_100',\n 'spnasnet_100',\n 'tf_efficientnet_b0',\n 'tf_efficientnet_b0_ap',\n 'tf_efficientnet_b0_ns',\n 'tf_efficientnet_b1',\n 'tf_efficientnet_b1_ap',\n 'tf_efficientnet_b1_ns',\n 'tf_efficientnet_b2',\n 'tf_efficientnet_b2_ap',\n 'tf_efficientnet_b2_ns',\n 'tf_efficientnet_b3',\n 'tf_efficientnet_b3_ap',\n 'tf_efficientnet_b3_ns',\n 'tf_efficientnet_b4',\n 'tf_efficientnet_b4_ap',\n 'tf_efficientnet_b4_ns',\n 'tf_efficientnet_b5',\n 'tf_efficientnet_b5_ap',\n 'tf_efficientnet_b5_ns',\n 'tf_efficientnet_b6',\n 'tf_efficientnet_b6_ap',\n 'tf_efficientnet_b6_ns',\n 'tf_efficientnet_b7',\n 'tf_efficientnet_b7_ap',\n 'tf_efficientnet_b7_ns',\n 'tf_efficientnet_b8',\n 'tf_efficientnet_b8_ap',\n 'tf_efficientnet_cc_b0_4e',\n 'tf_efficientnet_cc_b0_8e',\n 'tf_efficientnet_cc_b1_8e',\n 'tf_efficientnet_el',\n 'tf_efficientnet_em',\n 'tf_efficientnet_es',\n 'tf_efficientnet_l2_ns',\n 'tf_efficientnet_l2_ns_475',\n 'tf_efficientnet_lite0',\n 'tf_efficientnet_lite1',\n 'tf_efficientnet_lite2',\n 'tf_efficientnet_lite3',\n 'tf_efficientnet_lite4',\n 'tf_mixnet_l',\n 'tf_mixnet_m',\n 'tf_mixnet_s',\n 'tf_mobilenetv3_large_075',\n 'tf_mobilenetv3_large_100',\n 'tf_mobilenetv3_large_minimal_100',\n 'tf_mobilenetv3_small_075',\n 'tf_mobilenetv3_small_100',\n 'tf_mobilenetv3_small_minimal_100',\n 'tv_densenet121',\n 'tf_efficientnetv2_b0',\n 'tf_efficientnetv2_l',\n 'eca_efficientnet_b0',\n 'efficientnet_b2_pruned',\n 'efficientnet_b4',\n 'efficientnet_b5',\n 'efficientnet_b6',\n 'efficientnet_b7',\n 'efficientnet_b8',\n 'efficientnet_cc_b0_4e',\n 'efficientnet_cc_b0_8e',\n 'efficientnet_cc_b1_8e',\n 'efficientnet_el',\n 'efficientnet_el_pruned',\n 'efficientnet_es_pruned',\n 'efficientnet_l2',\n 'efficientnet_lite1',\n 'efficientnet_lite2',\n 'efficientnet_lite3',\n 'efficientnet_lite4',\n 'efficientnetv2_l',\n 'efficientnetv2_m',\n 'efficientnetv2_rw_m',\n 'efficientnetv2_rw_s',\n 'efficientnetv2_s',\n 'gc_efficientnet_b0',]:\n model = timm.create_model(model_type, pretrained=True)\n in_features = model.classifier.in_features\n model.classifier = nn.Linear(in_features, self.num_classes)\n self.model = model\n elif self.model_type in ['adv_inception_v3', # fc\n 'ecaresnet26t',\n 'ecaresnet50d',\n 'ecaresnet50d_pruned',\n 'ecaresnet50t',\n 'ecaresnet101d',\n 'ecaresnet101d_pruned',\n 'ecaresnet269d',\n 'ecaresnetlight',\n 'gluon_inception_v3',\n 'gluon_resnet18_v1b',\n 'gluon_resnet34_v1b',\n 'gluon_resnet50_v1b',\n 'gluon_resnet50_v1c',\n 'gluon_resnet50_v1d',\n 'gluon_resnet50_v1s',\n 'gluon_resnet101_v1b',\n 'gluon_resnet101_v1c',\n 'gluon_resnet101_v1d',\n 'gluon_resnet101_v1s',\n 'gluon_resnet152_v1b',\n 'gluon_resnet152_v1c',\n 'gluon_resnet152_v1d',\n 'gluon_resnet152_v1s',\n 'gluon_resnext50_32x4d',\n 'gluon_resnext101_32x4d',\n 'gluon_resnext101_64x4d',\n 'gluon_senet154',\n 'gluon_seresnext50_32x4d',\n 'gluon_seresnext101_32x4d',\n 'gluon_seresnext101_64x4d',\n 'gluon_xception65',\n 'ig_resnext101_32x8d',\n 'ig_resnext101_32x16d',\n 'ig_resnext101_32x32d',\n 'ig_resnext101_32x48d',\n 'inception_v3',\n 'res2net50_14w_8s',\n 'res2net50_26w_4s',\n 'res2net50_26w_6s',\n 'res2net50_26w_8s',\n 'res2net50_48w_2s',\n 'res2net101_26w_4s',\n 'res2next50',\n 'resnest14d',\n 'resnest26d',\n 'resnest50d',\n 'resnest50d_1s4x24d',\n 'resnest50d_4s2x40d',\n 'resnest101e',\n 'resnest200e',\n 'resnest269e',\n 'resnet18',\n 'resnet18d',\n 'resnet26',\n 'resnet26d',\n 'resnet34',\n 'resnet34d',\n 'resnet50',\n 'resnet50d',\n 'resnet101d',\n 'resnet152d',\n 'resnet200d',\n 'resnetblur50',\n 'resnext50_32x4d',\n 'resnext50d_32x4d',\n 'resnext101_32x8d',\n 'selecsls42b',\n 'selecsls60',\n 'selecsls60b',\n 'seresnet50',\n 'seresnet152d',\n 'seresnext26d_32x4d',\n 'seresnext26t_32x4d',\n 'seresnext50_32x4d',\n 'skresnet18',\n 'skresnet34',\n 'skresnext50_32x4d',\n 'ssl_resnet18',\n 'ssl_resnet50',\n 'ssl_resnext50_32x4d',\n 'ssl_resnext101_32x4d',\n 'ssl_resnext101_32x8d',\n 'ssl_resnext101_32x16d',\n 'swsl_resnet18',\n 'swsl_resnet50',\n 'swsl_resnext50_32x4d',\n 'swsl_resnext101_32x4d',\n 'swsl_resnext101_32x8d',\n 'swsl_resnext101_32x16d',\n 'tf_inception_v3',\n 'tv_resnet34',\n 'tv_resnet50',\n 'tv_resnet101',\n 'tv_resnet152',\n 'tv_resnext50_32x4d',\n 'wide_resnet50_2',\n 'wide_resnet101_2',\n 'xception', ]:\n model = timm.create_model(model_type, pretrained=True)\n in_features = model.fc.in_features\n model.classifier = nn.Linear(in_features, self.num_classes)\n self.model = model\n elif self.model_type in ['dla34',\n 'dla46_c',\n 'dla46x_c',\n 'dla60',\n 'dla60_res2net',\n 'dla60_res2next',\n 'dla60x',\n 'dla60x_c',\n 'dla102',\n 'dla102x',\n 'dla102x2',\n 'dla169',\n 'dpn68',\n 'dpn68b',\n 'dpn92',\n 'dpn98',\n 'dpn107',\n 'dpn131',\n ]:\n model = timm.create_model(model_type, pretrained=True)\n if self.model_type == 'dla34':\n model.fc = nn.Conv2d(512, self.num_classes, kernel_size=(1, 1), stride=(1, 1))\n elif self.model_type in ['dla46_c',\n 'dla46x_c',\n 'dla60x_c', ]:\n model.fc = nn.Conv2d(256, self.num_classes, kernel_size=(1, 1), stride=(1, 1))\n elif self.model_type in ['dla60',\n 'dla60_res2net',\n 'dla60_res2next',\n 'dla60x',\n 'dla102',\n 'dla102x',\n 'dla102x2',\n 'dla169']:\n model.fc = nn.Conv2d(1024, self.num_classes, kernel_size=(1, 1), stride=(1, 1))\n elif self.model_type in ['dpn68', 'dpn68b', ]:\n model.fc = nn.Conv2d(832, self.num_classes, kernel_size=(1, 1), stride=(1, 1))\n elif self.model_type in ['dpn92', 'dpn98', 'dpn107', 'dpn131', ]:\n model.fc = nn.Conv2d(2688, self.num_classes, kernel_size=(1, 1), stride=(1, 1))\n self.model = model\n elif self.model_type in ['cspdarknet53', # head.fc\n 'cspresnet50',\n 'cspresnext50',\n 'dm_nfnet_f0',\n 'dm_nfnet_f1',\n 'dm_nfnet_f2',\n 'dm_nfnet_f3',\n 'dm_nfnet_f4',\n 'dm_nfnet_f5',\n 'dm_nfnet_f6',\n 'ese_vovnet19b_dw',\n 'ese_vovnet39b',\n 'gernet_l',\n 'gernet_m',\n 'gernet_s',\n 'nf_regnet_b1',\n 'nf_resnet50',\n 'nfnet_l0c',\n 'regnetx_002',\n 'regnetx_004',\n 'regnetx_006',\n 'regnetx_008',\n 'regnetx_016',\n 'regnetx_032',\n 'regnetx_040',\n 'regnetx_064',\n 'regnetx_080',\n 'regnetx_120',\n 'regnetx_160',\n 'regnetx_320',\n 'regnety_002',\n 'regnety_004',\n 'regnety_006',\n 'regnety_008',\n 'regnety_016',\n 'regnety_032',\n 'regnety_040',\n 'regnety_064',\n 'regnety_080',\n 'regnety_120',\n 'regnety_160',\n 'regnety_320',\n 'repvgg_a2',\n 'repvgg_b0',\n 'repvgg_b1',\n 'repvgg_b1g4',\n 'repvgg_b2',\n 'repvgg_b2g4',\n 'repvgg_b3',\n 'repvgg_b3g4',\n 'resnetv2_50x1_bitm',\n 'resnetv2_50x1_bitm_in21k',\n 'resnetv2_50x3_bitm',\n 'resnetv2_50x3_bitm_in21k',\n 'resnetv2_101x1_bitm',\n 'resnetv2_101x1_bitm_in21k',\n 'resnetv2_101x3_bitm',\n 'resnetv2_101x3_bitm_in21k',\n 'resnetv2_152x2_bitm',\n 'resnetv2_152x2_bitm_in21k',\n 'resnetv2_152x4_bitm',\n 'resnetv2_152x4_bitm_in21k',\n 'rexnet_100',\n 'rexnet_130',\n 'rexnet_150',\n 'rexnet_200',\n 'tresnet_l',\n 'tresnet_l_448',\n 'tresnet_m',\n 'tresnet_m_448',\n 'tresnet_xl',\n 'tresnet_xl_448',\n 'vgg11',\n 'vgg11_bn',\n 'vgg13',\n 'vgg13_bn',\n 'vgg16',\n 'vgg16_bn',\n 'vgg19',\n 'vgg19_bn',\n 'xception41',\n 'xception65',\n 'xception71', ]:\n model = timm.create_model(model_type, pretrained=True)\n in_features = model.head.fc.in_features\n model.classifier = nn.Linear(in_features, self.num_classes)\n self.model = model\n elif self.model_type in ['deit_base_distilled_patch16_224']:\n model = timm.create_model(model_type, pretrained=True)\n in_features_head = model.head.in_features\n in_features_head_dist = model.head_dist.in_features\n model.head = nn.Linear(in_features_head, self.num_classes)\n model.head_dist = nn.Linear(in_features_head_dist, self.num_classes)\n print(model)\n self.model = model\n elif self.model_type in ['ens_adv_inception_resnet_v2', # classif\n 'inception_resnet_v2', ]:\n model = timm.create_model(model_type, pretrained=True)\n in_features = model.classif.in_features\n model.classifier = nn.Linear(in_features, self.num_classes)\n self.model = model\n elif self.model_type in ['inception_v4', # last_linear\n 'legacy_senet154',\n 'legacy_seresnet18',\n 'legacy_seresnet34',\n 'legacy_seresnet50',\n 'legacy_seresnet101',\n 'legacy_seresnet152',\n 'legacy_seresnext26_32x4d',\n 'legacy_seresnext50_32x4d',\n 'legacy_seresnext101_32x4d',\n 'nasnetalarge',\n 'pnasnet5large', ]:\n model = timm.create_model(model_type, pretrained=True)\n in_features = model.last_linear.in_features\n model.last_linear = nn.Linear(in_features, self.num_classes)\n self.model = model\n elif self.model_type in ['vit_base_patch16_224', # head\n 'vit_base_patch16_224_in21k',\n 'vit_base_patch16_384',\n 'vit_base_patch32_224_in21k',\n 'vit_base_patch32_384',\n 'vit_base_resnet50_224_in21k',\n 'vit_base_resnet50_384',\n 'vit_deit_base_distilled_patch16_224',\n 'vit_deit_base_distilled_patch16_384',\n 'vit_deit_base_patch16_224',\n 'vit_deit_base_patch16_384',\n 'vit_deit_small_distilled_patch16_224',\n 'vit_deit_small_patch16_224',\n 'vit_deit_tiny_distilled_patch16_224',\n 'vit_deit_tiny_patch16_224',\n 'vit_large_patch16_224',\n 'vit_large_patch16_224_in21k',\n 'vit_large_patch16_384',\n 'vit_large_patch32_224_in21k',\n 'vit_large_patch32_384',\n 'vit_small_patch16_224',\n 'cait_m36_384',\n 'cait_m48_448',\n 'cait_s24_224',\n 'cait_s24_384',\n 'cait_s36_384',\n 'cait_xs24_384',\n 'cait_xxs24_224',\n 'cait_xxs24_384',\n 'cait_xxs36_224',\n 'cait_xxs36_384',\n 'coat_lite_mini',\n 'coat_lite_small',\n 'coat_lite_tiny',\n 'coat_mini',\n 'coat_tiny',\n 'convit_base',\n 'convit_small',\n 'convit_tiny', ]:\n model = timm.create_model(model_type, pretrained=True)\n in_features = model.head.in_features\n model.classifier = nn.Linear(in_features, self.num_classes)\n self.model = model\n elif self.model_type in ['senet154']:\n model = pretrainedmodels.__dict__[model_type](num_classes=1000, pretrained='imagenet')\n model.eval()\n num_features = model.last_linear.in_features\n model.last_linear = nn.Linear(num_features, self.num_classes)\n self.model = model\n else:\n assert (\n False\n ), f\"model_type '{self.model_type}' not implemented. Please, choose from {MODELS}\"\n\n if classes_weights:\n self.classes_weights = torch.FloatTensor(classes_weights).cuda()\n self.loss_func = nn.CrossEntropyLoss(weight=self.classes_weights)\n else:\n self.loss_func = nn.CrossEntropyLoss()\n\n# self.f1 = torchmetrics.F1(num_classes=self.num_classes)\n\n def loss(self, logits, labels):\n return self.loss_func(input=logits, target=labels)\n\n # will be used during inference\n def forward(self, x):\n return self.model(x)\n\n # Using custom or multiple metrics (default_hp_metric=False)\n def on_train_start(self):\n self.logger.log_hyperparams(self.hparams)\n\n # logic for a single training step\n def training_step(self, batch, batch_idx):\n x, y = batch\n output = self.forward(x)\n train_loss = self.loss(output, y)\n\n # training metrics\n output = torch.argmax(output, dim=1)\n acc = accuracy(output, y)\n\n self.log('train_loss', train_loss, prog_bar=True)\n self.log('train_acc', acc, prog_bar=True)\n\n return train_loss\n\n # logic for a single validation step\n def validation_step(self, batch, batch_idx):\n x, y = batch\n output = self.forward(x)\n val_loss = self.loss(output, y)\n\n # validation metrics\n output = torch.argmax(output, dim=1)\n acc = accuracy(output, y)\n\n self.log('val_loss', val_loss, prog_bar=True)\n self.log('val_acc', acc, prog_bar=True)\n\n return val_loss\n\n # logic for a single testing step\n def test_step(self, batch, batch_idx):\n x, y = batch\n output = self(x)\n test_loss = self.loss(output, y)\n\n # validation metrics\n output = torch.argmax(output, dim=1)\n acc = accuracy(output, y)\n\n self.log('test_loss', test_loss, prog_bar=True)\n self.log('test_acc', acc, prog_bar=True)\n\n return test_loss\n\n # def training_epoch_end(self, outputs):\n # self.log('train_f1_epoch', self.f1.compute())\n # self.f1.reset()\n #\n # def validation_epoch_end(self, outputs):\n # self.log('val_f1_epoch', self.f1.compute(), prog_bar=True)\n # self.f1.reset()\n #\n # def test_epoch_end(self, outputs):\n # self.log('test_f1_epoch', self.f1.compute())\n # self.f1.reset()\n\n def configure_optimizers(self):\n gen_opt = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n\n gen_sched = {'scheduler': torch.optim.lr_scheduler.ExponentialLR(gen_opt, gamma=0.999, verbose=False),\n 'interval': 'epoch'}\n\n return [gen_opt], [gen_sched]","metadata":{"execution":{"iopub.status.busy":"2021-08-25T10:02:09.195427Z","iopub.execute_input":"2021-08-25T10:02:09.195788Z","iopub.status.idle":"2021-08-25T10:02:09.247254Z","shell.execute_reply.started":"2021-08-25T10:02:09.195754Z","shell.execute_reply":"2021-08-25T10:02:09.246237Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Train \n\nWe have prepared all the blocks and are ready for training.","metadata":{}},{"cell_type":"code","source":"project_name = 'landmarks' # folder name for logs\n# data_dir = '/kaggle/input/simpsons/data'\ndata_dir = '/kaggle/input/landmark-recognition-2021/train' # folder with images\nimages_ext = 'jpg'\naugment_p = 0.8 # this is probability of applying augmentation\ninit_lr = 0.0003 # learning rate\nearly_stop_patience = 5 # count of epochs when we should stop if our metrics don't improve\nmax_epochs = 5 \nprogress_bar_refresh_rate = 10 ","metadata":{"execution":{"iopub.status.busy":"2021-08-25T10:02:09.939349Z","iopub.execute_input":"2021-08-25T10:02:09.939724Z","iopub.status.idle":"2021-08-25T10:02:09.945169Z","shell.execute_reply.started":"2021-08-25T10:02:09.93969Z","shell.execute_reply":"2021-08-25T10:02:09.944116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_names = ['inception_v4'] # you can choose several models. In this case, the models will train sequentially.\n\nmodels = []\n# append parameters for models\nfor m in model_names:\n\n # this models not implemented in this tutorial\n if m in ['deit_base_distilled_patch16_224',\n 'deit_base_distilled_patch16_384',\n 'deit_base_patch16_224',\n 'deit_base_patch16_384',\n 'deit_small_distilled_patch16_224',\n 'deit_small_patch16_224',\n 'deit_tiny_distilled_patch16_224',\n 'deit_tiny_patch16_224', ]:\n continue\n\n \n model_dict = {}\n model_dict['batch_size'] = 8\n model_dict['model_type'] = m\n model_dict['im_size'] = None\n model_dict['im_size_test'] = None\n\n models.append(model_dict)\n\n\nmodels_for_training = []\n\n\nfor m in tqdm(models):\n model_data = {'model': m}\n\n # get mean, std, image_size of every model\n mod = timm.create_model(model_data['model']['model_type'], pretrained=False)\n model_mean = list(mod.default_cfg['mean'])\n model_std = list(mod.default_cfg['std'])\n\n # get input size\n im_size = 0\n im_size_test = 0\n\n print(model_data['model']['model_type'] + ' input size is ' + str(mod.default_cfg['input_size']))\n\n if model_data['model']['im_size']:\n im_size = model_data['model']['im_size']\n else:\n im_size = mod.default_cfg['input_size'][1]\n\n if model_data['model']['im_size_test']:\n im_size_test = model_data['model']['im_size']\n else:\n im_size_test = mod.default_cfg['input_size'][1]\n\n # create datamodule object\n dm = ICPDataModule(data_dir=data_dir,\n augment_p=augment_p,\n images_ext=images_ext,\n model_type=model_data['model']['model_type'],\n batch_size=model_data['model']['batch_size'],\n input_resize=im_size,\n input_resize_test=im_size_test,\n mean=model_mean,\n std=model_std)\n\n # To access the x_dataloader we need to call prepare_data and setup.\n dm.setup()\n\n # Init our model\n model = ICPModel(model_type=model_data['model']['model_type'],\n num_classes=dm.num_classes,\n classes_weights=None,\n learning_rate=init_lr)\n\n # Initialize a trainer\n early_stop_callback = EarlyStopping(\n monitor='val_loss',\n patience=early_stop_patience,\n verbose=True,\n mode='min'\n )\n\n # logs for tensorboard\n experiment_name = model_data['model']['model_type']\n logger = TensorBoardLogger('/kaggle/working/tb_logs/' + project_name + '/', name=experiment_name)\n\n checkpoint_name = experiment_name + '_' + '_{epoch}_{val_loss:.3f}_{val_acc:.3f}_{val_f1_epoch:.3f}'\n\n # mertics\n checkpoint_callback_loss = ModelCheckpoint(monitor='val_loss',\n mode='min',\n filename=checkpoint_name,\n verbose=True,\n save_top_k=3,\n save_last=False)\n\n checkpoint_callback_acc = ModelCheckpoint(monitor='val_acc',\n mode='max',\n filename=checkpoint_name,\n verbose=True,\n save_top_k=3,\n save_last=False)\n\n checkpoints = [checkpoint_callback_acc, checkpoint_callback_loss, early_stop_callback]\n callbacks = checkpoints\n\n # create Trainer\n trainer = pl.Trainer(max_epochs=max_epochs,\n progress_bar_refresh_rate=progress_bar_refresh_rate,\n gpus=1,\n logger=logger,\n callbacks=callbacks,\n # amp_level='02',\n # precision=16\n )\n\n model_data['icp_datamodule'] = dm\n model_data['icp_model'] = model\n model_data['icp_trainer'] = trainer\n\n models_for_training.append(model_data)\n\n# train process\nfor model in models_for_training:\n print('##################### START Training ' + model['model']['model_type'] + '... #####################')\n\n # Train the model โกg๐ โก\n model['icp_trainer'].fit(model['icp_model'], model['icp_datamodule'])\n\n # Evaluate the model on the held out test set โกโก\n results = model['icp_trainer'].test()[0]\n\n # save test results\n best_checkpoint = 'best_checkpoint: ' + model['icp_trainer'].checkpoint_callback.best_model_path\n results['best_checkpoint'] = best_checkpoint\n\n filename = datetime.datetime.now().strftime(\"%Y_%m_%d_%H_%M_%S\") + '__test_acc_' + str(\n round(results.get('test_acc'), 4)) + '.txt'\n\n path = '/kaggle/working/test_logs/' + project_name + '/' + model['model']['model_type']\n Path(path).mkdir(parents=True, exist_ok=True)\n\n with open(path + '/' + filename, 'w+') as f:\n print(results, file=f)\n\n print('##################### END Training ' + model['model']['model_type'] + '... #####################')","metadata":{"execution":{"iopub.status.busy":"2021-08-25T10:02:10.464897Z","iopub.execute_input":"2021-08-25T10:02:10.465224Z","iopub.status.idle":"2021-08-25T12:58:05.972056Z","shell.execute_reply.started":"2021-08-25T10:02:10.465192Z","shell.execute_reply":"2021-08-25T12:58:05.970525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Download weights \n\nAfter training you will have folders with best weights for every model.
\nDownload .txt file from kaggle/working/landmarks/%model_name%/. \nInside this file you will see text like this: \n{'test_loss': 0.32088467478752136, 'test_acc': 0.8716216087341309, 'best_checkpoint': 'best_checkpoint: /kaggle/working/tb_logs/landmarks/inception_v4/version_0/checkpoints/inception_v4__epoch=1_val_loss=0.414_val_acc=0.865_val_f1_epoch=0.000.ckpt'}\n
\nCopy best weights path.","metadata":{}},{"cell_type":"code","source":"!ls /kaggle/working/tb_logs/landmarks/inception_v4/version_1/checkpoints/inception_v4__epoch=1_val_loss=1.924_val_acc=0.556_val_f1_epoch=0.000-v1.ckpt","metadata":{"execution":{"iopub.status.busy":"2021-08-25T12:59:25.751658Z","iopub.execute_input":"2021-08-25T12:59:25.752084Z","iopub.status.idle":"2021-08-25T12:59:26.538308Z","shell.execute_reply.started":"2021-08-25T12:59:25.752045Z","shell.execute_reply":"2021-08-25T12:59:26.537351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nos.chdir(r'/kaggle/working')","metadata":{"execution":{"iopub.status.busy":"2021-08-25T12:59:58.601165Z","iopub.execute_input":"2021-08-25T12:59:58.601543Z","iopub.status.idle":"2021-08-25T12:59:58.605738Z","shell.execute_reply.started":"2021-08-25T12:59:58.601499Z","shell.execute_reply":"2021-08-25T12:59:58.604781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from IPython.display import FileLink\nFileLink(r'tb_logs/landmarks/inception_v4/version_1/checkpoints/inception_v4__epoch=1_val_loss=1.924_val_acc=0.556_val_f1_epoch=0.000-v1.ckpt')\n# click on link for downloading weights","metadata":{"execution":{"iopub.status.busy":"2021-08-25T13:00:00.123296Z","iopub.execute_input":"2021-08-25T13:00:00.123705Z","iopub.status.idle":"2021-08-25T13:00:00.131134Z","shell.execute_reply.started":"2021-08-25T13:00:00.123672Z","shell.execute_reply":"2021-08-25T13:00:00.130217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Add you own model from PyTorch Image Models ","metadata":{}},{"cell_type":"code","source":"# show all models\nimport timm\nfrom pprint import pprint\n\nmodel_names = timm.list_models(pretrained=True)\npprint(model_names)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# show parameters of current model\nmodel = 'tf_efficientnet_b3_ns'\nm = timm.create_model(model, pretrained=True)\nprint(model)\npprint(m.default_cfg)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Thank you for attention!\n \nI know there is a lot of code but if you understand it you will have a good tool for Image Classification Problem. \n\nIf you have some questions or suggestions you can write comments ๐ฌ \n\nDon't give up, keep trying! ๐ช","metadata":{}},{"cell_type":"markdown","source":"My specs ๐ป\n \n\n* 1 CPU, 16 cores & 16GB Memory ๐พ\n* NVIDIA **GeForce RTX 2080 Ti** ๐ฎ","metadata":{}}]}