{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":33679,"databundleVersionId":3212216,"sourceType":"competition"},{"sourceId":7961746,"sourceType":"datasetVersion","datasetId":4683667},{"sourceId":8004610,"sourceType":"datasetVersion","datasetId":4714148},{"sourceId":8038994,"sourceType":"datasetVersion","datasetId":4739432},{"sourceId":8050280,"sourceType":"datasetVersion","datasetId":4747361},{"sourceId":8066836,"sourceType":"datasetVersion","datasetId":4759325},{"sourceId":8070051,"sourceType":"datasetVersion","datasetId":4761586},{"sourceId":8125125,"sourceType":"datasetVersion","datasetId":4801667},{"sourceId":8137099,"sourceType":"datasetVersion","datasetId":4810404},{"sourceId":8305136,"sourceType":"datasetVersion","datasetId":4933648},{"sourceId":8333290,"sourceType":"datasetVersion","datasetId":4948579},{"sourceId":21917,"sourceType":"modelInstanceVersion","modelInstanceId":18150}],"dockerImageVersionId":30171,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n
\n\n

Table of Contents\n

\n
","metadata":{}},{"cell_type":"markdown","source":" | S.No | Heading |\n | :------------- | :-------------------: | \n | 01 | [**Libraries**](#library) | \n | 02 | [**JSON to Pandas CSV**](#json) |\n | 03 | [**Sample Images**](#sample-images) |\n | 04 | [**Config**](#config) |\n | 05 | [**Custom Dataset**](#dataset) |\n | 06 | [**Image Augmentation : Albumentations**](#Transform) |\n | 07 | [**Create Dataloader**](#dataloader) |\n | 08 | [**Model**](#model) |\n | 09 | [**Trainer⚡**](#Trainer) |\n | 10 | [**Plot Metrics**](#metrics) |","metadata":{}},{"cell_type":"markdown","source":"## **Identify plant species of the Americas from herbarium specimens**\n\nThe FGVC9 2022 Herbarium Challenge is to identify melastome species from herbarium specimens provided by the New York Botanical Garden (NYBG).\nBackground\n\nThere are more than 400,000 known plant species with an estimated 80,000 still to be discovered. In flowering plants, it takes approximately 35 years from plant collection to species description while less than 16% of new species are described in less than 5 years. It has also been suggested that ‘herbaria are a major frontier for species’ with more than 50% of unknown species already in herbarium collections.\n\nThe dataset encompasses over 90% of all plant taxa in North America, spanning 1.05M images for 15501 plant taxa, collected from 60 different institutions around the world.\n\nReferences : \n* https://www.kaggle.com/c/herbarium-2022-fgvc9/discussion/307792\n* https://www.kaggle.com/c/herbarium-2019-fgvc6","metadata":{}},{"cell_type":"markdown","source":"\n
\n\n

Libraries\n

\n
","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np \nimport matplotlib.pyplot as plt \nimport pandas as pd \nimport json\nimport cv2\nimport seaborn as sns\nfrom skimage import io\n\n!pip install albumentations\n!pip install pytorch-lightning\n!pip install timm\n!pip install torchmetrics\n# \n!pip install torchsummary\n!pip install efficientnet_pytorch\n# \n\nimport timm\nimport torch\nimport torch.nn as nn\nimport torch.functional as F\nfrom torchvision import datasets, transforms,models\nfrom torch.utils.data import Dataset,DataLoader\n\nfrom pytorch_lightning import Trainer, seed_everything\nfrom pytorch_lightning import Callback\nfrom pytorch_lightning.loggers import CSVLogger\nfrom pytorch_lightning.callbacks import ModelCheckpoint, EarlyStopping\nimport torchmetrics\n# \n# from pytorch_lightning.metrics.functional\n#\n\n# \nimport torchvision\nfrom torchsummary import summary\nfrom efficientnet_pytorch import EfficientNet\n# \n\nimport albumentations as A\nfrom albumentations.core.composition import Compose\nfrom albumentations.pytorch import ToTensorV2\n\nimport pytorch_lightning as pl\n%matplotlib inline","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-05-06T08:30:28.946251Z","iopub.execute_input":"2024-05-06T08:30:28.947034Z","iopub.status.idle":"2024-05-06T08:31:34.705973Z","shell.execute_reply.started":"2024-05-06T08:30:28.946933Z","shell.execute_reply":"2024-05-06T08:31:34.705284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

JSON to Pandas CSV\n

\n
","metadata":{}},{"cell_type":"markdown","source":"**Reference/Credits** : [JSON -> PANDAS📊\n@Sanskar Hasija](https://www.kaggle.com/odins0n/json-pandas-herbarium-2022/)","metadata":{}},{"cell_type":"code","source":"TRAIN_DIR = \"/kaggle/input/herbarium-2022-fgvc9-with-presizer-data/Herb_presizer_data/\"\nTEST_DIR = \"../input/herbarium-2022-fgvc9/test_images/\"\n\nwith open(\"../input/herbarium-2022-fgvc9/train_metadata.json\") as json_file:\n train_meta = json.load(json_file)\nwith open(\"../input/herbarium-2022-fgvc9/test_metadata.json\") as json_file:\n test_meta = json.load(json_file)\n \nimage_ids = [image[\"image_id\"] for image in train_meta[\"images\"]]\nimage_dirs = [TRAIN_DIR + image[\"file_name\"] for image in train_meta[\"images\"]]\ncategory_ids = [annot[\"category_id\"] for annot in train_meta[\"annotations\"]]\ngenus_ids = [annot[\"genus_id\"] for annot in train_meta[\"annotations\"] ]\ntest_ids = [image[\"image_id\"] for image in test_meta]\ntest_dirs = [TEST_DIR + image[\"file_name\"] for image in test_meta ]\n\ntrain = pd.DataFrame(data =np.array([image_ids , image_dirs, genus_ids, category_ids ]).T, \n columns = [\"image_id\", \"directory\",\"genus_id\", \"category\",])\ntest = pd.DataFrame(data =np.array([test_ids , test_dirs ]).T, \n columns = [\"image_id\", \"directory\",])\n\ntrain.to_csv(\"train.csv\", index = False)\ntest.to_csv(\"test.csv\", index = False)","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:31:47.157327Z","iopub.execute_input":"2024-05-06T08:31:47.157622Z","iopub.status.idle":"2024-05-06T08:32:14.057373Z","shell.execute_reply.started":"2024-05-06T08:31:47.157589Z","shell.execute_reply":"2024-05-06T08:32:14.0567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = pd.read_csv('train.csv')\ngenera = pd.DataFrame(train_meta['genera'])\ngenera = df_train.merge(genera,on='genus_id')\ngenera.head()","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:19.834223Z","iopub.execute_input":"2024-05-06T08:32:19.834933Z","iopub.status.idle":"2024-05-06T08:32:21.700943Z","shell.execute_reply.started":"2024-05-06T08:32:19.834894Z","shell.execute_reply":"2024-05-06T08:32:21.700137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.category.nunique()","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:23.645798Z","iopub.execute_input":"2024-05-06T08:32:23.646093Z","iopub.status.idle":"2024-05-06T08:32:23.657742Z","shell.execute_reply.started":"2024-05-06T08:32:23.646058Z","shell.execute_reply":"2024-05-06T08:32:23.656883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Total No. Of Unique Classes/Labels : **15501**","metadata":{}},{"cell_type":"markdown","source":"## **Top 10 Genera Distribution**","metadata":{}},{"cell_type":"code","source":"genera['genus'].value_counts().head(10).plot(kind='bar')","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:25.636676Z","iopub.execute_input":"2024-05-06T08:32:25.636986Z","iopub.status.idle":"2024-05-06T08:32:26.004194Z","shell.execute_reply.started":"2024-05-06T08:32:25.636952Z","shell.execute_reply":"2024-05-06T08:32:26.003469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Sample Images \n

\n
","metadata":{}},{"cell_type":"code","source":"from PIL import Image\nimport cv2\n\ndef display_images(genus):\n images = df_train.loc[genera['genus'] == genus]['directory'][:12]\n i = 1\n fig = plt.figure(figsize = (15, 15))\n plt.suptitle(genus, fontsize = '35')\n for image in images:\n ax = fig.add_subplot(3, 4, i)\n image = Image.open(image).convert('RGB')\n ax.imshow(image)\n width, height = image.size\n ax.text(0.5, -0.1, f\"Image Dimensions: {width}x{height}x3\", transform=ax.transAxes, ha='center', fontsize=10)\n i+=1\n fig.tight_layout()\n plt.show()\n\ndisplay_images('Carex')","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:27.829518Z","iopub.execute_input":"2024-05-06T08:32:27.829823Z","iopub.status.idle":"2024-05-06T08:32:30.552463Z","shell.execute_reply.started":"2024-05-06T08:32:27.829789Z","shell.execute_reply":"2024-05-06T08:32:30.551689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Encode Class Labels**","metadata":{}},{"cell_type":"code","source":"from sklearn import preprocessing\n\nle = preprocessing.LabelEncoder()\nle.fit(df_train['category'])\ndf_train['category'] = le.transform(df_train['category'])\ndf_train.category.nunique()","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:33.926542Z","iopub.execute_input":"2024-05-06T08:32:33.927162Z","iopub.status.idle":"2024-05-06T08:32:33.99117Z","shell.execute_reply.started":"2024-05-06T08:32:33.927121Z","shell.execute_reply":"2024-05-06T08:32:33.990304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since the Training Data is large approx 119 GB will take a lot time in training processs,We will work with a smaller sample of first 5000 images.","metadata":{}},{"cell_type":"markdown","source":"\n
\n\n

Config \n

\n
","metadata":{}},{"cell_type":"code","source":"\nclass CFG:\n seed = 42\n model_name = 'efficientnet-b2'\n PRETRAINED_PATH = \"/kaggle/input/efficientnetpytorch_v2_m/pytorch/v2_m_pth/1/efficientnet_v2_m-dc08266a.pth\"\n img_size = 224\n num_classes = df_train.category.nunique() \n lr = 1e-4\n min_lr = 1e-6\n t_max = 20\n num_epochs = 30\n batch_size = 32\n accum = 1\n precision = 16\n device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:36.956528Z","iopub.execute_input":"2024-05-06T08:32:36.957277Z","iopub.status.idle":"2024-05-06T08:32:37.019502Z","shell.execute_reply.started":"2024-05-06T08:32:36.95724Z","shell.execute_reply":"2024-05-06T08:32:37.018658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"seed_everything(CFG.seed)","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:39.117153Z","iopub.execute_input":"2024-05-06T08:32:39.117436Z","iopub.status.idle":"2024-05-06T08:32:39.126478Z","shell.execute_reply.started":"2024-05-06T08:32:39.117403Z","shell.execute_reply":"2024-05-06T08:32:39.125774Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df_train.shape)\ndf_train.head()","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:40.826353Z","iopub.execute_input":"2024-05-06T08:32:40.826993Z","iopub.status.idle":"2024-05-06T08:32:40.838313Z","shell.execute_reply.started":"2024-05-06T08:32:40.826958Z","shell.execute_reply":"2024-05-06T08:32:40.837526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Custom Dataset \n

\n
","metadata":{}},{"cell_type":"code","source":"class HerbariumDataset(Dataset):\n def __init__(self,data,transform=None):\n self.annotations = data\n self.transform = transform\n \n def __len__(self):\n return len(self.annotations)\n \n def __getitem__(self,index):\n \n img_path = self.annotations.iloc[index,1]\n image = io.imread(img_path)\n y_label = torch.tensor(int(self.annotations.iloc[index,3]))\n \n if self.transform:\n image = self.transform(image=np.array(image))\n \n return(image,y_label) ","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:42.532639Z","iopub.execute_input":"2024-05-06T08:32:42.53324Z","iopub.status.idle":"2024-05-06T08:32:42.54024Z","shell.execute_reply.started":"2024-05-06T08:32:42.533198Z","shell.execute_reply":"2024-05-06T08:32:42.53927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Image Augmentation : Albumentations \n

\n
","metadata":{}},{"cell_type":"markdown","source":"**Image augmentation** is a process of creating new training examples from the existing ones. To make a new sample, you slightly change the original image. For instance, you could make a new image a little brighter; you could cut a piece from the original image; you could make a new image by mirroring the original one, etc.\n![](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Falbumentations.ai%2Fdocs%2Fimages%2Fintroduction%2Fimage_augmentation%2Faugmentation.jpg&f=1&nofb=1)\n\n**Albumentations** is a Python library for fast and flexible image augmentations. Albumentations efficiently implements a rich variety of image transform operations that are optimized for performance, and does so while providing a concise, yet powerful image augmentation interface for different computer vision tasks, including object classification, segmentation, and detection.\n\nTo learn more [**Albumentations**](https://albumentations.ai/docs/)","metadata":{}},{"cell_type":"code","source":"def Transform(phase: str):\n if phase == 'train':\n return Compose([\n A.RandomResizedCrop(height=CFG.img_size, width=CFG.img_size),\n A.HorizontalFlip(p=0.5),\n A.ShiftScaleRotate(p=0.5),\n A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n ToTensorV2(),\n ])\n else:\n return A.Compose([\n A.Resize(height=CFG.img_size, width=CFG.img_size),\n A.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0),\n ToTensorV2(),\n ])","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:46.421067Z","iopub.execute_input":"2024-05-06T08:32:46.421618Z","iopub.status.idle":"2024-05-06T08:32:46.429146Z","shell.execute_reply.started":"2024-05-06T08:32:46.421583Z","shell.execute_reply":"2024-05-06T08:32:46.428433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Create Dataloader \n

\n
","metadata":{}},{"cell_type":"code","source":"train_dataset = HerbariumDataset(df_train,transform=Transform('train'))\nvalid_dataset = HerbariumDataset(df_train,transform=Transform('valid'))\n\ntrain_loader = DataLoader(train_dataset, batch_size=CFG.batch_size, shuffle=True, num_workers=2, pin_memory=True, drop_last=True)\nvalid_loader = DataLoader(valid_dataset, batch_size=CFG.batch_size, shuffle=False, num_workers=2)","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:49.823093Z","iopub.execute_input":"2024-05-06T08:32:49.823354Z","iopub.status.idle":"2024-05-06T08:32:49.829664Z","shell.execute_reply.started":"2024-05-06T08:32:49.823326Z","shell.execute_reply":"2024-05-06T08:32:49.828812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Transformed Image**","metadata":{}},{"cell_type":"code","source":"dataiter = iter(train_loader)\nimages, labels = dataiter.next()\nprint(images['image'][1].shape)\nprint(labels[1].shape)\nplt.imshow(images['image'][3].squeeze().permute(1,2,0), cmap='Greys_r')","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:32:52.501534Z","iopub.execute_input":"2024-05-06T08:32:52.50184Z","iopub.status.idle":"2024-05-06T08:32:57.293652Z","shell.execute_reply.started":"2024-05-06T08:32:52.501804Z","shell.execute_reply":"2024-05-06T08:32:57.292789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Model\n

\n
","metadata":{}},{"cell_type":"code","source":"# \nclass EfficientNetB2(nn.Module):\n def __init__(self,num_classes:int=0):\n super(EfficientNetB2, self).__init__()\n self.num_classes = num_classes\n self.model = EfficientNet.from_pretrained('efficientnet-b2')\n in_features = self.model._fc.in_features\n self.model._fc = nn.Linear(in_features, self.num_classes)\n \n def forward(self, x):\n return self.model(x)\n\n# ","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:33:02.325089Z","iopub.execute_input":"2024-05-06T08:33:02.325867Z","iopub.status.idle":"2024-05-06T08:33:02.331704Z","shell.execute_reply.started":"2024-05-06T08:33:02.325826Z","shell.execute_reply":"2024-05-06T08:33:02.330862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class LitHerbarium(pl.LightningModule):\n def __init__(self, model: nn.Module):\n super(LitHerbarium, self).__init__()\n self.model = model # ResNet50\n # \n self.num_classes = self.model.num_classes\n self.train_accuracy = torchmetrics.Accuracy()\n self.valid_accuracy = torchmetrics.Accuracy()\n self.train_precision = torchmetrics.Precision(num_classes=self.num_classes, average='macro')\n self.valid_precision = torchmetrics.Precision(num_classes=self.num_classes, average='macro')\n self.train_recall = torchmetrics.Recall(num_classes=self.num_classes, average='macro')\n self.valid_recall = torchmetrics.Recall(num_classes=self.num_classes, average='macro')\n # \n self.metric = torchmetrics.F1(num_classes=CFG.num_classes) # F1 Score\n self.criterion = nn.CrossEntropyLoss() # Cross Entropy loss\n self.lr = CFG.lr # Learning Rate\n\n def forward(self, x, *args, **kwargs):\n return self.model(x)\n\n def configure_optimizers(self):\n \"\"\"Choose what optimizers and learning-rate schedulers to use in your optimization\"\"\"\n self.optimizer = torch.optim.Adam(self.model.parameters(), lr=self.lr)\n self.scheduler = torch.optim.lr_scheduler.StepLR(self.optimizer, step_size=2, gamma=0.1)\n\n return {'optimizer': self.optimizer, 'lr_scheduler': self.scheduler}\n\n def training_step(self, batch, batch_idx):\n \"\"\"Here you compute and return the training loss and some additional metrics for e.g. the progress bar or logger\"\"\"\n image,target = batch\n image = image['image']\n output = self.model(image)\n loss = self.criterion(output, target)\n accuracy = self.train_accuracy(output, target)\n precision = self.train_precision(output, target)\n recall = self.train_recall(output, target)\n score = self.metric(output.argmax(1), target)\n # \n logs = {'train_loss': loss, 'train_acc':accuracy,'train_precision': precision, 'train_recall': recall, 'train_f1': score, 'lr': self.optimizer.param_groups[0]['lr']}\n # \n self.log_dict(\n logs,\n on_step=False, on_epoch=True, prog_bar=True, logger=True\n )\n return loss\n\n def validation_step(self, batch, batch_idx):\n \"\"\"Operates on a single batch of data from the validation set .calculate anything of interest like accuracy.\"\"\"\n image,target = batch\n image = image['image']\n output = self.model(image)\n loss = self.criterion(output, target)\n accuracy = self.valid_accuracy(output, target)\n precision = self.valid_precision(output, target)\n recall = self.valid_recall(output, target)\n score = self.metric(output.argmax(1), target)\n # \n logs = {'valid_loss': loss, 'valid_acc':accuracy,'valid_precision': precision, 'valid_recall': recall, 'valid_f1': score}\n # \n self.log_dict(\n logs,\n on_step=False, on_epoch=True, prog_bar=True, logger=True\n )\n return loss","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:33:05.0422Z","iopub.execute_input":"2024-05-06T08:33:05.04283Z","iopub.status.idle":"2024-05-06T08:33:05.062031Z","shell.execute_reply.started":"2024-05-06T08:33:05.042796Z","shell.execute_reply":"2024-05-06T08:33:05.061076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### **To Learn more about** [LightningModule](https://pytorch-lightning.readthedocs.io/en/latest/common/lightning_module.html?highlight=training_step#)","metadata":{}},{"cell_type":"code","source":"model = EfficientNetB2(num_classes=CFG.num_classes)\nlit_model = LitHerbarium(model)\nlit_model","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:33:08.374432Z","iopub.execute_input":"2024-05-06T08:33:08.375031Z","iopub.status.idle":"2024-05-06T08:33:10.779555Z","shell.execute_reply.started":"2024-05-06T08:33:08.374995Z","shell.execute_reply":"2024-05-06T08:33:10.778674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model=model.cuda()","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:33:19.227666Z","iopub.execute_input":"2024-05-06T08:33:19.228258Z","iopub.status.idle":"2024-05-06T08:33:19.303268Z","shell.execute_reply.started":"2024-05-06T08:33:19.228219Z","shell.execute_reply":"2024-05-06T08:33:19.302631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pytorch computational Graph of ResNet50 \n\n# # !pip install torchviz\n# from torchviz import make_dot\n# batch = next(iter(train_loader))\n# images , labels = batch\n# label = model(images['image'])\n# make_dot(label, params=dict(list(lit_model.named_parameters())))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Trainer⚡\n

\n
","metadata":{}},{"cell_type":"markdown","source":"* **Max_epochs** : Stop training once this number of epochs is reached\n* **gpus** : Number of GPUs to train on (int)\n* **accumulate_grad_batches** : Accumulates grads every k batches or as set up in the dict. Trainer also calls optimizer.step() for the last indivisible step number.\n* **precision** : 16 bit floating points to reduce memory footprint during model training. This can result in improved performance.\n* **callbacks** : A callback is a self-contained program that can be reused across projects.\n* **checkpoint_callback / enable_checkpointing** : If True, enable checkpointing Else None.\n* **logger** : PyTorch TensorBoard logging under the hood, and stores the logs to a directory \n\nTo learn more about [**Trainer**](https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer)","metadata":{}},{"cell_type":"code","source":"logger = CSVLogger(save_dir='logs/', name=CFG.model_name)\nlogger.log_hyperparams(CFG.__dict__)\ncheckpoint_callback = ModelCheckpoint(monitor='valid_loss',\n save_top_k=1,\n save_last=True,\n save_weights_only=True,\n filename='checkpoint/{epoch:02d}-{valid_loss:.4f}-{valid_f1:.4f}',\n verbose=False,\n mode='min')\n\ntrainer = Trainer(\n max_epochs=CFG.num_epochs,\n gpus=1,\n accumulate_grad_batches=CFG.accum,\n precision=CFG.precision,\n callbacks=[EarlyStopping(monitor='valid_loss', patience=3, mode='min')],\n checkpoint_callback=checkpoint_callback,\n enable_checkpointing=True,\n logger=logger,\n)","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:33:21.632646Z","iopub.execute_input":"2024-05-06T08:33:21.632959Z","iopub.status.idle":"2024-05-06T08:33:21.655088Z","shell.execute_reply.started":"2024-05-06T08:33:21.632926Z","shell.execute_reply":"2024-05-06T08:33:21.654352Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer.fit(lit_model, train_dataloader=train_loader, val_dataloaders=valid_loader)","metadata":{"execution":{"iopub.status.busy":"2024-04-08T09:46:33.904809Z","iopub.execute_input":"2024-04-08T09:46:33.90512Z","iopub.status.idle":"2024-04-08T15:05:46.529351Z","shell.execute_reply.started":"2024-04-08T09:46:33.905086Z","shell.execute_reply":"2024-04-08T15:05:46.527941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### *LOADING CHECKPOINT FILES*","metadata":{}},{"cell_type":"code","source":"trainer.fit(lit_model, ckpt_path=\"/kaggle/input/epoch-10-presizer-effnet/version_1/checkpoints/epoch10-step288661.ckpt\", train_dataloader=train_loader, val_dataloaders=valid_loader)","metadata":{"execution":{"iopub.status.busy":"2024-05-06T08:33:44.425836Z","iopub.execute_input":"2024-05-06T08:33:44.426477Z","iopub.status.idle":"2024-05-06T12:06:41.177108Z","shell.execute_reply.started":"2024-05-06T08:33:44.426429Z","shell.execute_reply":"2024-05-06T12:06:41.175296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"aaaaaaaa","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n
\n\n

Plot Metrics\n

\n
","metadata":{}},{"cell_type":"code","source":"metrics = pd.read_csv(f'{trainer.logger.log_dir}/metrics.csv')\n\ntrain_acc = metrics['train_f1'].dropna().reset_index(drop=True)\nvalid_acc = metrics['valid_f1'].dropna().reset_index(drop=True)\n \nfig = plt.figure(figsize=(7, 6))\n\nplt.plot(train_acc, color=\"r\", label='train/f1')\nplt.plot(valid_acc, color=\"b\", label='valid/f1')\nplt.ylabel('F1', fontsize=24)\nplt.xlabel('Epoch', fontsize=24)\nplt.legend(loc='lower right', fontsize=18)\nplt.savefig(f'{trainer.logger.log_dir}/f1.png')\n\ntrain_loss = metrics['train_loss'].dropna().reset_index(drop=True)\nvalid_loss = metrics['valid_loss'].dropna().reset_index(drop=True)\n\nfig = plt.figure(figsize=(7, 6))\nplt.plot(train_loss, color=\"r\", label='train/loss')\nplt.plot(valid_loss, color=\"b\", label='valid/loss')\nplt.ylabel('Loss', fontsize=24)\nplt.xlabel('Epoch', fontsize=24)\nplt.legend(loc='upper right', fontsize=18)\nplt.savefig(f'{trainer.logger.log_dir}/loss.png')\\\n\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}