{"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":7781194,"sourceType":"datasetVersion","datasetId":4553461}],"dockerImageVersionId":30665,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.13"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Setup","metadata":{}},{"cell_type":"code","source":"#!pip install timm huggingface_hub kaggle -Uqq\n# !kaggle datasets download -d vishalbakshi/hms-hbac-training-spectrogram-images\n# zipfile.ZipFile('hms-hbac-training-spectrogram-images.zip').extractall('hms-hbac-training-spectrogram-images')","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:11:50.145862Z","iopub.status.busy":"2024-03-09T23:11:50.145278Z","iopub.status.idle":"2024-03-09T23:11:55.343951Z","shell.execute_reply":"2024-03-09T23:11:55.343079Z","shell.execute_reply.started":"2024-03-09T23:11:50.145797Z"},"_kg_hide-input":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nimport timm\nimport gc\n\nfrom fastai.vision.all import *\nfrom fastcore.parallel import *\n\npath = Path('/kaggle/input/hms-hbac-training-spectrogram-images/train_spectrograms')\n#path = Path('/notebooks/hms-hbac-training-spectrogram-images/train_spectrograms')\npath.ls()","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2024-03-09T23:11:56.769977Z","iopub.status.busy":"2024-03-09T23:11:56.769324Z","iopub.status.idle":"2024-03-09T23:12:00.867070Z","shell.execute_reply":"2024-03-09T23:12:00.866466Z","shell.execute_reply.started":"2024-03-09T23:11:56.769951Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Background\n\nThis is the second notebook of a series of 5 notebooks where I train different `convnext` image classifiers on training spectrogram images using the fastai library:\n\n- [Part 0](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-planning-small-model-experiments): I plan out my initial small model experiments and create a [Kaggle dataset](https://www.kaggle.com/datasets/vishalbakshi/hms-hbac-training-spectrogram-images) with training spectrogram images.\n- **Part 1 [Train] (You are here): I train 48 variants of `convnext_small_in22k` using different `ImageDataLoaders`.**\n- [Part 1 Analysis](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-convnext-small-pt-1-analysis): I analyze the results from Part 1, run a few more trainings and pick the top convnext models for submission.**\n- [Part 2 [Train]](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-convnext-small-pt-2-train): I train those top convnext models and export them to Kaggle.\n- [Part 2 [Submit]](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-convnext-small-pt-2-submit): I submit those models individually and as ensembles, and document their Kaggle Public Score.\n\nI'll follow the same approach (experiment -> train and export top models -> submit -> document Kaggle Public Score) for three other families: `vit`, `swin` and `swinv2`. Once I have identified the best `small` models, I'll train their `large` versions, submit them and document the results. I have taken this general approach from Jeremy Howard's [Road to the Top](https://www.kaggle.com/code/jhoward/first-steps-road-to-the-top-part-1) notebook series (although his notebooks and presentation is much more efficient).\n\n---\n\nI have created a `train` helper function similar to what Jeremy Howard did in his [Road to the Top](https://www.kaggle.com/code/jhoward/scaling-up-road-to-the-top-part-3?scriptVersionId=99313071&cellId=11) notebook series. I'll run the first training line-by-line to make sure each step looks good, then for the following trainings I'll use my helper function to expedite the process.\n\nPlease note that I ran out of Kaggle GPU quota so I have run this notebook in Paperspace with faster GPUs so the training time will be longer if you run this on Kaggle. I used the A4000 with 45GB RAM, 8GB CPU and 16GB GPU (Paperspace Pro subscription).","metadata":{}},{"cell_type":"markdown","source":"## Create `DataLoaders`","metadata":{}},{"cell_type":"markdown","source":"Instead of processing the parquet files, I'm now using images directly so I'll use the `ImageDataLoaders.from_path` factory function instead of building my own `DataBlock`.","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n    path, \n    valid_pct=0.2, \n    item_tfms=Resize(256), \n    batch_tfms=aug_transforms(size=128, min_scale=0.75))\n\ndls.show_batch()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T19:00:09.144504Z","iopub.status.busy":"2024-03-08T19:00:09.143872Z","iopub.status.idle":"2024-03-08T19:00:11.939961Z","shell.execute_reply":"2024-03-08T19:00:11.939254Z","shell.execute_reply.started":"2024-03-08T19:00:09.144482Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Notice how the images are all skewed/rotated/brighter/darker because I have applied [`aug_transforms`](https://docs.fast.ai/vision.augment.html#aug_transforms) to each batch. I'm not sure if this will improve the model's performance so I'll also do a run where `batch_tfms=None`.","metadata":{}},{"cell_type":"markdown","source":"I'll see if I can use the same convnext model that Jeremy used in his Road to the Top series: `convnext_small_in22k`. My two previous training runs were overfitting the model at around 7 epochs.","metadata":{}},{"cell_type":"code","source":"learn = vision_learner(dls, 'convnext_small_in22k', metrics=error_rate).to_fp16()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T19:00:16.151333Z","iopub.status.busy":"2024-03-08T19:00:16.150520Z","iopub.status.idle":"2024-03-08T19:00:16.952821Z","shell.execute_reply":"2024-03-08T19:00:16.952256Z","shell.execute_reply.started":"2024-03-08T19:00:16.151310Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fine_tune(7, 0.01)","metadata":{"execution":{"iopub.execute_input":"2024-03-08T19:00:21.942979Z","iopub.status.busy":"2024-03-08T19:00:21.942531Z","iopub.status.idle":"2024-03-08T19:04:19.835493Z","shell.execute_reply":"2024-03-08T19:04:19.834822Z","shell.execute_reply.started":"2024-03-08T19:00:21.942958Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.recorder.plot_loss()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T19:04:28.585017Z","iopub.status.busy":"2024-03-08T19:04:28.584349Z","iopub.status.idle":"2024-03-08T19:04:28.683681Z","shell.execute_reply":"2024-03-08T19:04:28.683234Z","shell.execute_reply.started":"2024-03-08T19:04:28.584992Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I could probably push it another epoch or two but for now I'll keepy it at 7 epochs since the validation loss is definitely starting to slow down.","metadata":{}},{"cell_type":"code","source":"error_rate(*learn.tta(dl=dls.valid))","metadata":{"execution":{"iopub.execute_input":"2024-03-08T19:05:19.855506Z","iopub.status.busy":"2024-03-08T19:05:19.854684Z","iopub.status.idle":"2024-03-08T19:05:36.284759Z","shell.execute_reply":"2024-03-08T19:05:36.283880Z","shell.execute_reply.started":"2024-03-08T19:05:19.855481Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The TTA validation error rate is noticeably better than regular validation error rate. I'll certainly use it in this notebook and for predictions in the eventual submission notebook for my `convnext_small` models.\n\nI'll use Jeremy's `train` function, set the random `seed` so I am comparing error rates on the same validation set, and also add a bit of logic to handle situations where I don't want to use `aug_transforms`.","metadata":{}},{"cell_type":"code","source":"def train(arch, size, item, accum=4, epochs=7):\n    if size is None: \n        batch = None\n    else: \n        batch = aug_transforms(size=size, min_scale=0.75)\n        \n    dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=item,\n        batch_tfms=batch,\n        bs=64//accum)\n    \n    cbs = GradientAccumulation(64) if accum else []\n    learn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\n    learn.fine_tune(epochs, 0.01)\n    print(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:12:08.408087Z","iopub.status.busy":"2024-03-09T23:12:08.407011Z","iopub.status.idle":"2024-03-09T23:12:08.413590Z","shell.execute_reply":"2024-03-09T23:12:08.412916Z","shell.execute_reply.started":"2024-03-09T23:12:08.408046Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I was hoping to use a `models` dictionary to store all of my training parameters and `for`-loop to run all trainings consecutively, as done in [Road to the Top, Part 3](https://www.kaggle.com/code/jhoward/scaling-up-road-to-the-top-part-3?scriptVersionId=99313071&cellId=34), but my notebook kept stalling before getting through all 20 runs (both on Kaggle and in Paperspace). So, instead, I'll run each training one-at-a-time. ","metadata":{}},{"cell_type":"markdown","source":"### `method='squish'`","metadata":{}},{"cell_type":"code","source":"warnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:12:13.273615Z","iopub.status.busy":"2024-03-09T23:12:13.272971Z","iopub.status.idle":"2024-03-09T23:12:13.276824Z","shell.execute_reply":"2024-03-09T23:12:13.276021Z","shell.execute_reply.started":"2024-03-09T23:12:13.273591Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"arch = 'convnext_small_in22k'","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:12:16.249081Z","iopub.status.busy":"2024-03-09T23:12:16.248816Z","iopub.status.idle":"2024-03-09T23:12:16.252503Z","shell.execute_reply":"2024-03-09T23:12:16.251731Z","shell.execute_reply.started":"2024-03-09T23:12:16.249063Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize(400, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T20:57:51.832807Z","iopub.status.busy":"2024-03-08T20:57:51.831948Z","iopub.status.idle":"2024-03-08T21:16:30.634902Z","shell.execute_reply":"2024-03-08T21:16:30.634288Z","shell.execute_reply.started":"2024-03-08T20:57:51.832779Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize(400, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T21:21:51.955464Z","iopub.status.busy":"2024-03-08T21:21:51.954639Z","iopub.status.idle":"2024-03-08T21:40:38.983822Z","shell.execute_reply":"2024-03-08T21:40:38.982687Z","shell.execute_reply.started":"2024-03-08T21:21:51.955424Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize(400, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T21:42:09.870144Z","iopub.status.busy":"2024-03-08T21:42:09.869868Z","iopub.status.idle":"2024-03-08T21:54:13.907525Z","shell.execute_reply":"2024-03-08T21:54:13.906884Z","shell.execute_reply.started":"2024-03-08T21:42:09.870123Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize(400, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T21:54:13.908809Z","iopub.status.busy":"2024-03-08T21:54:13.908615Z","iopub.status.idle":"2024-03-08T22:09:20.505464Z","shell.execute_reply":"2024-03-08T22:09:20.504662Z","shell.execute_reply.started":"2024-03-08T21:54:13.908790Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize(311, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T22:09:20.506793Z","iopub.status.busy":"2024-03-08T22:09:20.506595Z","iopub.status.idle":"2024-03-08T22:21:17.341957Z","shell.execute_reply":"2024-03-08T22:21:17.341418Z","shell.execute_reply.started":"2024-03-08T22:09:20.506773Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize(311, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T22:21:17.343984Z","iopub.status.busy":"2024-03-08T22:21:17.343247Z","iopub.status.idle":"2024-03-08T22:40:01.847526Z","shell.execute_reply":"2024-03-08T22:40:01.847014Z","shell.execute_reply.started":"2024-03-08T22:21:17.343959Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize(311, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T22:40:01.848885Z","iopub.status.busy":"2024-03-08T22:40:01.848394Z","iopub.status.idle":"2024-03-08T22:52:04.980291Z","shell.execute_reply":"2024-03-08T22:52:04.979489Z","shell.execute_reply.started":"2024-03-08T22:40:01.848861Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize(311, 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T22:52:04.982709Z","iopub.status.busy":"2024-03-08T22:52:04.982430Z","iopub.status.idle":"2024-03-08T23:07:22.271962Z","shell.execute_reply":"2024-03-08T23:07:22.269895Z","shell.execute_reply.started":"2024-03-08T22:52:04.982685Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize((311,400), 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T17:12:10.687842Z","iopub.status.busy":"2024-03-09T17:12:10.687351Z","iopub.status.idle":"2024-03-09T17:27:44.300680Z","shell.execute_reply":"2024-03-09T17:27:44.299815Z","shell.execute_reply.started":"2024-03-09T17:12:10.687820Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((311,400), 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T17:27:44.302441Z","iopub.status.busy":"2024-03-09T17:27:44.302222Z","iopub.status.idle":"2024-03-09T17:46:08.391777Z","shell.execute_reply":"2024-03-09T17:46:08.390676Z","shell.execute_reply.started":"2024-03-09T17:27:44.302423Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((311,400), 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T17:46:08.393909Z","iopub.status.busy":"2024-03-09T17:46:08.393588Z","iopub.status.idle":"2024-03-09T17:58:05.648005Z","shell.execute_reply":"2024-03-09T17:58:05.647299Z","shell.execute_reply.started":"2024-03-09T17:46:08.393860Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((311,400), 'squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T17:58:05.650958Z","iopub.status.busy":"2024-03-09T17:58:05.650042Z","iopub.status.idle":"2024-03-09T18:13:08.018193Z","shell.execute_reply":"2024-03-09T18:13:08.016167Z","shell.execute_reply.started":"2024-03-09T17:58:05.650928Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `method='crop'`","metadata":{}},{"cell_type":"code","source":"train(arch, size=None, item=Resize(400))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T23:15:21.107396Z","iopub.status.busy":"2024-03-08T23:15:21.106631Z","iopub.status.idle":"2024-03-08T23:34:24.099829Z","shell.execute_reply":"2024-03-08T23:34:24.098869Z","shell.execute_reply.started":"2024-03-08T23:15:21.107360Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize(400))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T23:34:24.102214Z","iopub.status.busy":"2024-03-08T23:34:24.101555Z","iopub.status.idle":"2024-03-08T23:53:18.653280Z","shell.execute_reply":"2024-03-08T23:53:18.652675Z","shell.execute_reply.started":"2024-03-08T23:34:24.102188Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize(400))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-08T23:53:18.654848Z","iopub.status.busy":"2024-03-08T23:53:18.654138Z","iopub.status.idle":"2024-03-09T00:05:15.923486Z","shell.execute_reply":"2024-03-09T00:05:15.922785Z","shell.execute_reply.started":"2024-03-08T23:53:18.654822Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize(400))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T00:05:15.925266Z","iopub.status.busy":"2024-03-09T00:05:15.924789Z","iopub.status.idle":"2024-03-09T00:20:19.454958Z","shell.execute_reply":"2024-03-09T00:20:19.454309Z","shell.execute_reply.started":"2024-03-09T00:05:15.925245Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize(311))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T03:18:26.272464Z","iopub.status.busy":"2024-03-09T03:18:26.271978Z","iopub.status.idle":"2024-03-09T03:30:07.279417Z","shell.execute_reply":"2024-03-09T03:30:07.278457Z","shell.execute_reply.started":"2024-03-09T03:18:26.272438Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize(311))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T03:30:07.281346Z","iopub.status.busy":"2024-03-09T03:30:07.281126Z","iopub.status.idle":"2024-03-09T03:47:53.970667Z","shell.execute_reply":"2024-03-09T03:47:53.969679Z","shell.execute_reply.started":"2024-03-09T03:30:07.281326Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize(311))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T03:47:53.972141Z","iopub.status.busy":"2024-03-09T03:47:53.971898Z","iopub.status.idle":"2024-03-09T03:59:16.361650Z","shell.execute_reply":"2024-03-09T03:59:16.360786Z","shell.execute_reply.started":"2024-03-09T03:47:53.972121Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize(311))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T03:59:16.363514Z","iopub.status.busy":"2024-03-09T03:59:16.363258Z","iopub.status.idle":"2024-03-09T04:13:42.468846Z","shell.execute_reply":"2024-03-09T04:13:42.468009Z","shell.execute_reply.started":"2024-03-09T03:59:16.363493Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize((311,400)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T18:13:08.021541Z","iopub.status.busy":"2024-03-09T18:13:08.020199Z","iopub.status.idle":"2024-03-09T18:27:56.135185Z","shell.execute_reply":"2024-03-09T18:27:56.134095Z","shell.execute_reply.started":"2024-03-09T18:13:08.021510Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((311,400)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T18:27:56.136687Z","iopub.status.busy":"2024-03-09T18:27:56.136471Z","iopub.status.idle":"2024-03-09T18:46:20.393317Z","shell.execute_reply":"2024-03-09T18:46:20.392228Z","shell.execute_reply.started":"2024-03-09T18:27:56.136668Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((311,400)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T18:46:20.395040Z","iopub.status.busy":"2024-03-09T18:46:20.394729Z","iopub.status.idle":"2024-03-09T18:58:13.889692Z","shell.execute_reply":"2024-03-09T18:58:13.888939Z","shell.execute_reply.started":"2024-03-09T18:46:20.395018Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((311,400)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T18:58:13.891121Z","iopub.status.busy":"2024-03-09T18:58:13.890909Z","iopub.status.idle":"2024-03-09T19:13:08.418285Z","shell.execute_reply":"2024-03-09T19:13:08.417357Z","shell.execute_reply.started":"2024-03-09T18:58:13.891103Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `method=ResizeMethod.Pad`","metadata":{}},{"cell_type":"code","source":"train(arch, size=None, item=Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T04:16:25.828455Z","iopub.status.busy":"2024-03-09T04:16:25.828156Z","iopub.status.idle":"2024-03-09T04:30:49.769278Z","shell.execute_reply":"2024-03-09T04:30:49.768618Z","shell.execute_reply.started":"2024-03-09T04:16:25.828433Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T04:30:49.770957Z","iopub.status.busy":"2024-03-09T04:30:49.770635Z","iopub.status.idle":"2024-03-09T04:48:43.314452Z","shell.execute_reply":"2024-03-09T04:48:43.313765Z","shell.execute_reply.started":"2024-03-09T04:30:49.770935Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T04:48:43.315741Z","iopub.status.busy":"2024-03-09T04:48:43.315546Z","iopub.status.idle":"2024-03-09T05:00:09.744262Z","shell.execute_reply":"2024-03-09T05:00:09.743282Z","shell.execute_reply.started":"2024-03-09T04:48:43.315723Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T05:00:09.746123Z","iopub.status.busy":"2024-03-09T05:00:09.745859Z","iopub.status.idle":"2024-03-09T05:14:36.528028Z","shell.execute_reply":"2024-03-09T05:14:36.527300Z","shell.execute_reply.started":"2024-03-09T05:00:09.746102Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize((400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T19:13:08.420395Z","iopub.status.busy":"2024-03-09T19:13:08.419644Z","iopub.status.idle":"2024-03-09T19:31:19.913858Z","shell.execute_reply":"2024-03-09T19:31:19.912679Z","shell.execute_reply.started":"2024-03-09T19:13:08.420367Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T19:31:19.916279Z","iopub.status.busy":"2024-03-09T19:31:19.916012Z","iopub.status.idle":"2024-03-09T19:49:45.975299Z","shell.execute_reply":"2024-03-09T19:49:45.974385Z","shell.execute_reply.started":"2024-03-09T19:31:19.916258Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T19:49:45.977051Z","iopub.status.busy":"2024-03-09T19:49:45.976845Z","iopub.status.idle":"2024-03-09T20:01:44.233696Z","shell.execute_reply":"2024-03-09T20:01:44.232746Z","shell.execute_reply.started":"2024-03-09T19:49:45.977051Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T20:01:44.235530Z","iopub.status.busy":"2024-03-09T20:01:44.235007Z","iopub.status.idle":"2024-03-09T20:16:39.659362Z","shell.execute_reply":"2024-03-09T20:16:39.658238Z","shell.execute_reply.started":"2024-03-09T20:01:44.235509Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=None, item=Resize((311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T20:16:39.660953Z","iopub.status.busy":"2024-03-09T20:16:39.660729Z","iopub.status.idle":"2024-03-09T20:28:19.543993Z","shell.execute_reply":"2024-03-09T20:28:19.543364Z","shell.execute_reply.started":"2024-03-09T20:16:39.660935Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T20:28:19.545681Z","iopub.status.busy":"2024-03-09T20:28:19.545039Z","iopub.status.idle":"2024-03-09T20:46:21.726461Z","shell.execute_reply":"2024-03-09T20:46:21.725763Z","shell.execute_reply.started":"2024-03-09T20:28:19.545653Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T20:46:21.728210Z","iopub.status.busy":"2024-03-09T20:46:21.727804Z","iopub.status.idle":"2024-03-09T20:57:54.287005Z","shell.execute_reply":"2024-03-09T20:57:54.286412Z","shell.execute_reply.started":"2024-03-09T20:46:21.728189Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T20:57:54.288188Z","iopub.status.busy":"2024-03-09T20:57:54.287869Z","iopub.status.idle":"2024-03-09T21:12:28.158657Z","shell.execute_reply":"2024-03-09T21:12:28.157699Z","shell.execute_reply.started":"2024-03-09T20:57:54.288170Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"After running these trainings, I realized that I might have messed up the order of height and width for image dimensions in `item_tfms` and `batch_tfms`. I should've done a more thorough job of ensuring the right order at the start, but better late than never I suppose. I'll create a couple of `DataLoaders` to make sure I understand the order of the image dimensions.\nSince it's hard to tell whether a spectrogram is skewed or not, I'll use the PETS dataset.","metadata":{}},{"cell_type":"code","source":"pets_path = untar_data(URLs.PETS)","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:29:02.627125Z","iopub.status.busy":"2024-03-09T21:29:02.626427Z","iopub.status.idle":"2024-03-09T21:29:27.568417Z","shell.execute_reply":"2024-03-09T21:29:27.567803Z","shell.execute_reply.started":"2024-03-09T21:29:02.627098Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n    pets_path, \n    valid_pct=0.2, \n    item_tfms=Resize((100,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\n\ndls.show_batch(nrows=1, ncols=3)","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:30:35.933034Z","iopub.status.busy":"2024-03-09T21:30:35.932191Z","iopub.status.idle":"2024-03-09T21:30:37.372795Z","shell.execute_reply":"2024-03-09T21:30:37.372122Z","shell.execute_reply.started":"2024-03-09T21:30:35.933001Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Okay so the first dimension, `100` is the height, and `400` is the width. In my experiments I erroneously used `Resize((311,400))` whereas what I really want is `311` to be the width and `400` to be the height, mkaing it `Resize((400,311))`","metadata":{}},{"cell_type":"markdown","source":"I'll next look at `batch_tfms`:","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n    pets_path, \n    valid_pct=0.2, \n    item_tfms=Resize(256),\n    batch_tfms=aug_transforms(size=(100,400), min_scale=0.75))\n\ndls.show_batch(nrows=1, ncols=3)","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:32:49.824968Z","iopub.status.busy":"2024-03-09T21:32:49.824318Z","iopub.status.idle":"2024-03-09T21:32:50.844019Z","shell.execute_reply":"2024-03-09T21:32:50.843579Z","shell.execute_reply.started":"2024-03-09T21:32:49.824938Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The same order applies here: `(height, width)`. This is inverse of `PILImage.size`:","metadata":{}},{"cell_type":"code","source":"PILImage.create((pets_path/'images').ls()[0])","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:34:00.105825Z","iopub.status.busy":"2024-03-09T21:34:00.104840Z","iopub.status.idle":"2024-03-09T21:34:00.176945Z","shell.execute_reply":"2024-03-09T21:34:00.176419Z","shell.execute_reply.started":"2024-03-09T21:34:00.105800Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PILImage.create((pets_path/'images').ls()[0]).size","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:34:10.219601Z","iopub.status.busy":"2024-03-09T21:34:10.219296Z","iopub.status.idle":"2024-03-09T21:34:10.237700Z","shell.execute_reply":"2024-03-09T21:34:10.236965Z","shell.execute_reply.started":"2024-03-09T21:34:10.219581Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here, the order is `width, height`.\n\nI'll train a series of models with `(400,311)` as the `height, width` tuple, which is what I meant to do since all spectrograms have a height of 400 px and a median width of 311.","metadata":{}},{"cell_type":"markdown","source":"### `method='squish'`","metadata":{}},{"cell_type":"code","source":"train(arch, size=None, item=Resize((400,311), method='squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:38:43.768081Z","iopub.status.busy":"2024-03-09T21:38:43.767776Z","iopub.status.idle":"2024-03-09T21:53:19.110955Z","shell.execute_reply":"2024-03-09T21:53:19.110361Z","shell.execute_reply.started":"2024-03-09T21:38:43.768061Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((400,311), method='squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T21:53:19.112452Z","iopub.status.busy":"2024-03-09T21:53:19.112169Z","iopub.status.idle":"2024-03-09T22:11:30.465681Z","shell.execute_reply":"2024-03-09T22:11:30.464826Z","shell.execute_reply.started":"2024-03-09T21:53:19.112435Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((400,311), method='squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T22:11:30.468043Z","iopub.status.busy":"2024-03-09T22:11:30.467531Z","iopub.status.idle":"2024-03-09T22:23:35.966262Z","shell.execute_reply":"2024-03-09T22:23:35.965042Z","shell.execute_reply.started":"2024-03-09T22:11:30.468015Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((400,311), method='squish'))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T22:24:44.758385Z","iopub.status.busy":"2024-03-09T22:24:44.757993Z","iopub.status.idle":"2024-03-09T22:39:39.840287Z","shell.execute_reply":"2024-03-09T22:39:39.839686Z","shell.execute_reply.started":"2024-03-09T22:24:44.758358Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `method='crop'`","metadata":{}},{"cell_type":"code","source":"train(arch, size=None, item=Resize((400,311)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T22:42:00.247731Z","iopub.status.busy":"2024-03-09T22:42:00.247089Z","iopub.status.idle":"2024-03-09T22:56:46.740084Z","shell.execute_reply":"2024-03-09T22:56:46.739144Z","shell.execute_reply.started":"2024-03-09T22:42:00.247703Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((400,311)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:12:35.745209Z","iopub.status.busy":"2024-03-09T23:12:35.744933Z","iopub.status.idle":"2024-03-09T23:31:25.298224Z","shell.execute_reply":"2024-03-09T23:31:25.297177Z","shell.execute_reply.started":"2024-03-09T23:12:35.745190Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((400,311)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:31:25.299926Z","iopub.status.busy":"2024-03-09T23:31:25.299731Z","iopub.status.idle":"2024-03-09T23:43:08.852346Z","shell.execute_reply":"2024-03-09T23:43:08.851318Z","shell.execute_reply.started":"2024-03-09T23:31:25.299908Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((400,311)))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:43:08.853832Z","iopub.status.busy":"2024-03-09T23:43:08.853647Z","iopub.status.idle":"2024-03-09T23:57:56.670188Z","shell.execute_reply":"2024-03-09T23:57:56.669667Z","shell.execute_reply.started":"2024-03-09T23:43:08.853812Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `method='pad'`","metadata":{}},{"cell_type":"code","source":"train(arch, size=None, item=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-09T23:57:56.672184Z","iopub.status.busy":"2024-03-09T23:57:56.671517Z","iopub.status.idle":"2024-03-10T00:12:42.176074Z","shell.execute_reply":"2024-03-10T00:12:42.175383Z","shell.execute_reply.started":"2024-03-09T23:57:56.672182Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=384, item=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-10T00:12:42.177580Z","iopub.status.busy":"2024-03-10T00:12:42.177227Z","iopub.status.idle":"2024-03-10T00:31:02.063317Z","shell.execute_reply":"2024-03-10T00:31:02.062391Z","shell.execute_reply.started":"2024-03-10T00:12:42.177560Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=288, item=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-10T00:31:02.064999Z","iopub.status.busy":"2024-03-10T00:31:02.064427Z","iopub.status.idle":"2024-03-10T00:42:46.871217Z","shell.execute_reply":"2024-03-10T00:42:46.870438Z","shell.execute_reply.started":"2024-03-10T00:31:02.064978Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train(arch, size=(384,299), item=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))\ngc.collect()\ntorch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2024-03-10T00:42:46.872510Z","iopub.status.busy":"2024-03-10T00:42:46.872308Z","iopub.status.idle":"2024-03-10T01:01:11.549485Z","shell.execute_reply":"2024-03-10T01:01:11.548942Z","shell.execute_reply.started":"2024-03-10T00:42:46.872494Z"}},"execution_count":null,"outputs":[]}]}