{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Simple baseline with ViT Transformers + Hugging Face + Lightning\n\nThis notebook shows the use of Hugging Face, Pytorch and Pytorch Lightning to train a classifier with ViT Transformers architecture.\n\nIt is based/inspired on the [HuggingPics](https://github.com/nateraw/huggingpics) project and uses a rezised and adjusted labels by folder dataset\n\nhttps://www.kaggle.com/ibombonato/sorghum-100-cultivar-512x512-png-imagefolder\n\n\nTO DO:\n- ~~Make inference faster~~\n- Find best learning rate\n- ~~Add augmentation~~\n- ~~Better split strategy~~\n- Add CrossValidation\n- ~~Make Wandb Work (I am getting an error right now with the logger in self.experiment.config.update(params, allow_val_change=True))~~\n- Load from checkpoint?\n\n**If it helps you in some manner, please upvote the dataset and the notebook :D**","metadata":{}},{"cell_type":"markdown","source":"### Load libs and minimal setup","metadata":{}},{"cell_type":"code","source":"!pip install -q timm\n!pip install -q --upgrade wandb wandb[service]","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:28:45.041432Z","iopub.execute_input":"2022-03-22T19:28:45.042095Z","iopub.status.idle":"2022-03-22T19:29:02.85839Z","shell.execute_reply.started":"2022-03-22T19:28:45.042051Z","shell.execute_reply":"2022-03-22T19:29:02.857434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom tqdm import tqdm\nfrom tqdm.auto import tqdm\nfrom sklearn.model_selection import ShuffleSplit\nfrom PIL import Image, UnidentifiedImageError\nfrom pathlib import Path\n\ntqdm.pandas()","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-03-22T19:29:02.860784Z","iopub.execute_input":"2022-03-22T19:29:02.861531Z","iopub.status.idle":"2022-03-22T19:29:02.868935Z","shell.execute_reply.started":"2022-03-22T19:29:02.861483Z","shell.execute_reply":"2022-03-22T19:29:02.867792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Confirm that a GPU is available\n!nvidia-smi","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:02.870468Z","iopub.execute_input":"2022-03-22T19:29:02.87069Z","iopub.status.idle":"2022-03-22T19:29:03.642095Z","shell.execute_reply.started":"2022-03-22T19:29:02.870655Z","shell.execute_reply":"2022-03-22T19:29:03.640952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ORIGIN_FOLDER = \"../input/sorghum-100-cultivar-512x512-png-imagefolder/images\"\nUSE_WANDB = True\nEPOCHS = 50\nMODEL_NAME = 'google/vit-base-patch16-224-in21k'","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:03.643881Z","iopub.execute_input":"2022-03-22T19:29:03.644131Z","iopub.status.idle":"2022-03-22T19:29:03.648835Z","shell.execute_reply.started":"2022-03-22T19:29:03.644103Z","shell.execute_reply":"2022-03-22T19:29:03.647632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_raw = pd.read_csv(\"../input/sorghum-id-fgvc-9/train_cultivar_mapping.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:03.6503Z","iopub.execute_input":"2022-03-22T19:29:03.650984Z","iopub.status.idle":"2022-03-22T19:29:03.700563Z","shell.execute_reply.started":"2022-03-22T19:29:03.65095Z","shell.execute_reply":"2022-03-22T19:29:03.699833Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport shutil\nimport torch\nimport pytorch_lightning as pl\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchmetrics import Accuracy\nfrom torchvision.datasets import ImageFolder\nfrom transformers import AutoFeatureExtractor, ViTForImageClassification\nfrom torchvision.transforms import ToTensor\nimport torchvision\nfrom torchvision.io import read_image\nimport random\nfrom timm.data import ImageDataset\nfrom sklearn.model_selection import StratifiedShuffleSplit","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:03.702019Z","iopub.execute_input":"2022-03-22T19:29:03.702282Z","iopub.status.idle":"2022-03-22T19:29:03.708513Z","shell.execute_reply.started":"2022-03-22T19:29:03.702244Z","shell.execute_reply":"2022-03-22T19:29:03.70769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Loading the data\n\nSince we are using a [dataset](https://www.kaggle.com/ibombonato/sorghum-100-cultivar-512x512-png-imagefolder) that has all imades grouped by folders/labels, we can use `ImageFolder` from `torchvision.datasets` to load the dataset and simplify the process.\n\n~~We will use `random_split` from Pytorch to split the Images into train and validation sets.~~","metadata":{}},{"cell_type":"code","source":"all_ds = ImageFolder(Path(ORIGIN_FOLDER, \"train\"))","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:03.709548Z","iopub.execute_input":"2022-03-22T19:29:03.709758Z","iopub.status.idle":"2022-03-22T19:29:03.959186Z","shell.execute_reply.started":"2022-03-22T19:29:03.709732Z","shell.execute_reply":"2022-03-22T19:29:03.958455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Lets add some transformation to the images, this will help our model to generalize better and also help with overfit","metadata":{}},{"cell_type":"code","source":"from torchvision import transforms\n# For training, we add some augmentation. Networks are too powerful and would overfit.\n\nfeature_extractor = AutoFeatureExtractor.from_pretrained(MODEL_NAME)\nnormalize = transforms.Normalize(mean=feature_extractor.image_mean, std=feature_extractor.image_std)\n\ntrain_transform = transforms.Compose(\n    [\n        transforms.RandomAffine(0.75),\n        transforms.ColorJitter(brightness=0.5, contrast=0.25),\n        transforms.RandomAutocontrast(0.25),\n        transforms.RandomRotation(0.15),\n        transforms.RandomResizedCrop(feature_extractor.size, scale=(0.1, 1), ratio=(0.5, 2)),\n        transforms.RandomHorizontalFlip(),\n    ]\n)\n\nval_transform = transforms.Compose(\n    [\n        transforms.Resize(feature_extractor.size),\n    ]\n)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:03.960493Z","iopub.execute_input":"2022-03-22T19:29:03.961259Z","iopub.status.idle":"2022-03-22T19:29:05.360884Z","shell.execute_reply.started":"2022-03-22T19:29:03.961215Z","shell.execute_reply":"2022-03-22T19:29:05.35996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will use `StratifiedShuffleSplit` from `sklearn` to split the Images into train and validation sets in a stratified way, label proportions are keept in the split.\n\nWe also need to create a `Subset` that we can use differente transforms for train and validation","metadata":{}},{"cell_type":"code","source":"# https://stackoverflow.com/questions/51782021/how-to-use-different-data-augmentation-for-subsets-in-pytorch\n# Subset with transform, so we can have a train and val transform\nclass Subset(Dataset):\n    r\"\"\"\n    Subset of a dataset at specified indices.\n\n    Arguments:\n        dataset (Dataset): The whole Dataset\n        indices (sequence): Indices in the whole set selected for subset\n        transform (Transformation): Vision Transforms to apply in the image\n    \"\"\"\n    def __init__(self, dataset, indices, transform):\n        self.dataset = dataset\n        self.indices = indices\n        self.transform = transform\n\n    def __getitem__(self, idx):\n        im, labels = self.dataset[self.indices[idx]]\n        if self.transform:\n            im = self.transform(im)\n        return im, labels\n\n    def __len__(self):\n        return len(self.indices)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:05.362314Z","iopub.execute_input":"2022-03-22T19:29:05.362627Z","iopub.status.idle":"2022-03-22T19:29:05.370493Z","shell.execute_reply.started":"2022-03-22T19:29:05.362589Z","shell.execute_reply":"2022-03-22T19:29:05.369755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def train_test_dataset(dataset, test_split, train_transform, val_transform):\n    X = dataset.imgs\n    y = dataset.targets\n    sss = StratifiedShuffleSplit(n_splits=1, test_size=test_split, random_state=42)\n    train_idx, val_idx = next(sss.split(X, y))\n    \n    train_ds = Subset(dataset, train_idx, train_transform)\n    val_ds = Subset(dataset, val_idx, val_transform)\n    return train_ds, val_ds\n\ntrain_ds, val_ds = train_test_dataset(all_ds, 0.2, train_transform, val_transform)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:05.371833Z","iopub.execute_input":"2022-03-22T19:29:05.372374Z","iopub.status.idle":"2022-03-22T19:29:06.460431Z","shell.execute_reply.started":"2022-03-22T19:29:05.372322Z","shell.execute_reply":"2022-03-22T19:29:06.459258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot a random image from train set\nimg, label = train_ds[random.randint(0, len(train_ds))]\nplt.imshow(img, cmap=\"gray\")","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:06.461683Z","iopub.execute_input":"2022-03-22T19:29:06.461934Z","iopub.status.idle":"2022-03-22T19:29:06.770104Z","shell.execute_reply.started":"2022-03-22T19:29:06.461905Z","shell.execute_reply":"2022-03-22T19:29:06.769291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot a random image from validation set\nimg, label = val_ds[random.randint(0, len(val_ds))]\nplt.imshow(img, cmap=\"gray\")","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:06.775713Z","iopub.execute_input":"2022-03-22T19:29:06.776421Z","iopub.status.idle":"2022-03-22T19:29:07.04772Z","shell.execute_reply.started":"2022-03-22T19:29:06.776381Z","shell.execute_reply":"2022-03-22T19:29:07.047146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since pytorch will convert targets to numeric, we will map ids to labels and labels to ids, so we can get/acess the class names in the future.","metadata":{}},{"cell_type":"code","source":"label2id = {}\nid2label = {}\n\nfor i, class_name in enumerate(all_ds.classes):\n    label2id[class_name] = str(i)\n    id2label[str(i)] = class_name","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:07.048736Z","iopub.execute_input":"2022-03-22T19:29:07.049442Z","iopub.status.idle":"2022-03-22T19:29:07.053924Z","shell.execute_reply.started":"2022-03-22T19:29:07.04941Z","shell.execute_reply":"2022-03-22T19:29:07.053104Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ImageClassificationCollator:\n    def __init__(self, feature_extractor):\n        self.feature_extractor = feature_extractor\n \n    def __call__(self, batch):\n        encodings = self.feature_extractor([x[0] for x in batch], return_tensors='pt')\n        encodings['labels'] = torch.tensor([x[1] for x in batch], dtype=torch.long)\n        return encodings","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:07.055163Z","iopub.execute_input":"2022-03-22T19:29:07.055428Z","iopub.status.idle":"2022-03-22T19:29:07.066753Z","shell.execute_reply.started":"2022-03-22T19:29:07.055397Z","shell.execute_reply":"2022-03-22T19:29:07.065848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#feature_extractor = AutoFeatureExtractor.from_pretrained(\"google/vit-base-patch16-224-in21k\")\nmodel = ViTForImageClassification.from_pretrained(\n    MODEL_NAME,\n    num_labels=len(label2id),\n    label2id=label2id,\n    id2label=id2label,\n    ignore_mismatched_sizes=True,\n)\n\ncollator = ImageClassificationCollator(feature_extractor)\ntrain_loader = DataLoader(train_ds, batch_size=8, collate_fn=collator, num_workers=2, shuffle=True)\nval_loader = DataLoader(val_ds, batch_size=16, collate_fn=collator, num_workers=2)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:07.068239Z","iopub.execute_input":"2022-03-22T19:29:07.068977Z","iopub.status.idle":"2022-03-22T19:29:09.617246Z","shell.execute_reply.started":"2022-03-22T19:29:07.068934Z","shell.execute_reply":"2022-03-22T19:29:09.616506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Classifier(pl.LightningModule):\n\n    def __init__(self, model, lr: float = 2e-5, **kwargs):\n        super().__init__()\n        self.save_hyperparameters('lr', *list(kwargs))\n        self.model = model\n        self.forward = self.model.forward\n        self.val_acc = Accuracy()\n\n    def training_step(self, batch, batch_idx):\n        outputs = self(**batch)\n        self.log(f\"train_loss\", outputs.loss)\n        return outputs.loss\n\n    def validation_step(self, batch, batch_idx):\n        outputs = self(**batch)\n        self.log(f\"val_loss\", outputs.loss)\n        acc = self.val_acc(outputs.logits.argmax(1), batch['labels'])\n        self.log(f\"val_acc\", acc, prog_bar=True)\n        return outputs.loss\n\n    def configure_optimizers(self):\n        return torch.optim.Adam(self.parameters(), lr=self.hparams.lr)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:09.618402Z","iopub.execute_input":"2022-03-22T19:29:09.618696Z","iopub.status.idle":"2022-03-22T19:29:09.628141Z","shell.execute_reply.started":"2022-03-22T19:29:09.618666Z","shell.execute_reply":"2022-03-22T19:29:09.627343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Setting up teh logger with Wandb","metadata":{}},{"cell_type":"code","source":"# FOR THIS TO WORK, YOU NEED TO SET YOUR API KEY IN THE KAGGLE SECRETS ENVIRONMENT!\nimport os\nfrom pytorch_lightning.loggers import WandbLogger\nfrom kaggle_secrets import UserSecretsClient\nimport wandb\n\nif USE_WANDB:\n    project_name = \"kaggle-sorghum-100-cultivar\"\n    user_secrets = UserSecretsClient()\n    wandb.require(experiment=\"service\")\n    wandb.login(key=user_secrets.get_secret(\"WANDB_API_KEY\"))\n       \n    model_logger = WandbLogger(project=project_name, log_model='all', config={\"epochs\": EPOCHS})\nelse:\n    model_logger=None","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:09.62973Z","iopub.execute_input":"2022-03-22T19:29:09.630071Z","iopub.status.idle":"2022-03-22T19:29:13.360707Z","shell.execute_reply.started":"2022-03-22T19:29:09.63003Z","shell.execute_reply":"2022-03-22T19:29:13.35948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_logger.experiment.config.update({\"epochs\": EPOCHS}, allow_val_change=True)\nmodel_logger.experiment.config","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:13.362706Z","iopub.execute_input":"2022-03-22T19:29:13.36304Z","iopub.status.idle":"2022-03-22T19:29:19.715287Z","shell.execute_reply.started":"2022-03-22T19:29:13.362988Z","shell.execute_reply":"2022-03-22T19:29:19.714686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Train the model","metadata":{}},{"cell_type":"code","source":"pl.seed_everything(42)\nclassifier = Classifier(model, lr=2e-5)\ntrainer = pl.Trainer(gpus=1, precision=16, max_epochs=EPOCHS, logger=model_logger)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:19.716371Z","iopub.execute_input":"2022-03-22T19:29:19.716776Z","iopub.status.idle":"2022-03-22T19:29:19.733317Z","shell.execute_reply.started":"2022-03-22T19:29:19.716747Z","shell.execute_reply":"2022-03-22T19:29:19.732637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer.fit(classifier, train_loader, val_loader)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T19:29:19.734989Z","iopub.execute_input":"2022-03-22T19:29:19.735587Z","iopub.status.idle":"2022-03-22T20:46:03.493617Z","shell.execute_reply.started":"2022-03-22T19:29:19.735542Z","shell.execute_reply":"2022-03-22T20:46:03.492546Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer.save_checkpoint(f\"cultivar_baseline_epoch_{EPOCHS}_vit_transformer.ckpt\")","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.567525Z","iopub.status.idle":"2022-03-22T20:46:05.567987Z","shell.execute_reply.started":"2022-03-22T20:46:05.567744Z","shell.execute_reply":"2022-03-22T20:46:05.567767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if USE_WANDB: wandb.finish()","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.569892Z","iopub.status.idle":"2022-03-22T20:46:05.570388Z","shell.execute_reply.started":"2022-03-22T20:46:05.570108Z","shell.execute_reply":"2022-03-22T20:46:05.570133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Make predictions\n\nNow we will make predictions on the test set.\n\nAfter make some adjustments, I could score it via batch and reduce the time **from 4 hours to 6 minutes :D**\n\n~~A working to improve here is to score the test set via batch and not one to one.~~","metadata":{}},{"cell_type":"markdown","source":"The old code are collapsed bellow for reference.","metadata":{}},{"cell_type":"code","source":"# OLD CODE, JUST FOR REFERENCE, DO NOT USE IT!\n# It takes over 4 hours to do inference on all test images :-/\n\n# def pred_image(img):\n    \n#     if not Path(img).exists(): return ''\n    \n#     im = Image.open(img)\n#     # Transform our image and pass it through the model\n#     inputs = feature_extractor(im, return_tensors='pt')\n#     with torch.no_grad():\n#         output = model(**inputs)\n\n#     # Predicted Class probabilities\n#     proba = output.logits.softmax(1)\n\n#     # Predicted Classes\n#     preds = proba.argmax(1)\n\n#     return model.config.id2label[str(preds.item())]\n\n# model.eval()\n\n# TEST_FOLDER = \"../input/sorghum-id-fgvc-9/test\"\n\n# test_df = pd.read_csv(\"../input/sorghum-id-fgvc-9/sample_submission.csv\")\n\n# test_df['cultivar'] = test_df.filename.progress_apply(lambda x: pred_image(f\"{TEST_FOLDER}/{x}\"))\n\n# test_df.to_csv(\"submission.csv\", index = False)\n","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.57193Z","iopub.status.idle":"2022-03-22T20:46:05.572434Z","shell.execute_reply.started":"2022-03-22T20:46:05.572145Z","shell.execute_reply":"2022-03-22T20:46:05.572172Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TEST_FOLDER = \"../input/sorghum-100-cultivar-512x512-png-imagefolder/images/test\"\n\ntest_ds = ImageDataset(Path(TEST_FOLDER))\ntest_dl = DataLoader(test_ds, batch_size=32, collate_fn=collator, num_workers=2)","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.573435Z","iopub.status.idle":"2022-03-22T20:46:05.573888Z","shell.execute_reply.started":"2022-03-22T20:46:05.57364Z","shell.execute_reply":"2022-03-22T20:46:05.573664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot a random image from test set\nimg, label = test_ds[random.randint(0, len(test_ds))]\nplt.imshow(img, cmap=\"gray\")","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.575637Z","iopub.status.idle":"2022-03-22T20:46:05.576092Z","shell.execute_reply.started":"2022-03-22T20:46:05.575845Z","shell.execute_reply":"2022-03-22T20:46:05.575869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.cuda()\nmodel.eval()\n\ndef batch_predictions(dl, ds, id2label):\n    predictions = []\n    for batch in tqdm(dl):\n        image = batch['pixel_values'].cuda()\n        with torch.no_grad():\n            outputs = model(image)\n            preds = outputs.logits.softmax(1).argmax(1).detach().cpu().numpy()\n            predictions.append(preds)\n        \n    all_preds = []\n    for batch in predictions:\n        for prediction in batch:\n            all_preds.append(id2label[str(prediction)])\n\n    return all_preds, ds.filenames()","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.577499Z","iopub.status.idle":"2022-03-22T20:46:05.577947Z","shell.execute_reply.started":"2022-03-22T20:46:05.577703Z","shell.execute_reply":"2022-03-22T20:46:05.577727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batch_preds, batch_filenames = batch_predictions(test_dl, test_ds, id2label)\ndf_preds = pd.DataFrame({'filename': batch_filenames, \"cultivar\": batch_preds})\ndf_preds.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.579147Z","iopub.status.idle":"2022-03-22T20:46:05.579704Z","shell.execute_reply.started":"2022-03-22T20:46:05.579352Z","shell.execute_reply":"2022-03-22T20:46:05.579442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submisson\n\nAt the moment, the testset or the sample_submission are broken and its not possible to submit. As soon as the organizers fix it, I will update with the submission.\n","metadata":{}},{"cell_type":"code","source":"test_df = pd.read_csv(\"../input/sorghum-id-fgvc-9/sample_submission.csv\")\n\nsubmission_df = pd.merge(test_df[['filename']], df_preds, how='inner', on='filename')\n\nsubmission_df.to_csv(\"submission.csv\", index = False)\n\nsubmission_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-22T20:46:05.581272Z","iopub.status.idle":"2022-03-22T20:46:05.581934Z","shell.execute_reply.started":"2022-03-22T20:46:05.581726Z","shell.execute_reply":"2022-03-22T20:46:05.581753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## TO DO:\n\n- ~~Make inference faster~~\n- Find best learning rate\n- ~~Add augmentation~~\n- ~~Better split strategy~~\n- Add CrossValidation\n- ~~Make Wandb Work (I am getting an error right now with the logger in `self.experiment.config.update(params, allow_val_change=True)`)~~\n- Load from checkpoint?","metadata":{}},{"cell_type":"markdown","source":"## If it helps you in some manner, please upvote the dataset and the notebook :D","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}