{"metadata":{"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.13"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"!pip install timm huggingface_hub kaggle -Uqq\n!jupyter notebook --ServerApp.iopub_data_rate_limit=1.0e10\n# !kaggle datasets download -d vishalbakshi/hms-hbac-training-spectrogram-images\n# zipfile.ZipFile('hms-hbac-training-spectrogram-images.zip').extractall('hms-hbac-training-spectrogram-images')","metadata":{"execution":{"iopub.execute_input":"2024-03-18T03:48:34.467953Z","iopub.status.busy":"2024-03-18T03:48:34.467390Z","iopub.status.idle":"2024-03-18T03:48:41.551031Z","shell.execute_reply":"2024-03-18T03:48:41.550186Z","shell.execute_reply.started":"2024-03-18T03:48:34.467890Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nimport timm\nimport gc\n\nfrom fastai.vision.all import *\nfrom fastcore.parallel import *\n\n#path = Path('/kaggle/input/hms-hbac-training-spectrogram-images/train_spectrograms')\npath = Path('/notebooks/hms-hbac-training-spectrogram-images/train_spectrograms')\npath.ls()","metadata":{"execution":{"iopub.execute_input":"2024-03-18T03:54:33.653747Z","iopub.status.busy":"2024-03-18T03:54:33.653101Z","iopub.status.idle":"2024-03-18T03:54:39.267139Z","shell.execute_reply":"2024-03-18T03:54:39.266327Z","shell.execute_reply.started":"2024-03-18T03:54:33.653717Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Background","metadata":{}},{"cell_type":"markdown","source":"In this notebook I train variants of the `vit` model and document the results (validation TTA error rate and final epoch validation error rate). In the next notebook I'll train and submit the top-5 performing `vit` models.\n\nAs Jeremy Howard has done in his [Road to the Top](https://www.kaggle.com/code/jhoward/scaling-up-road-to-the-top-part-3) series, I'll be using the `vit_small_patch16_224` architecture which requires the image size to be 224 x 224 pixels.\n\nI'll train a model for 15 epochs and see when the validation loss starts to increase (indicating overfitting).","metadata":{}},{"cell_type":"code","source":"arch = 'vit_small_patch16_224'","metadata":{"execution":{"iopub.execute_input":"2024-03-18T03:54:43.527306Z","iopub.status.busy":"2024-03-18T03:54:43.526585Z","iopub.status.idle":"2024-03-18T03:54:43.530776Z","shell.execute_reply":"2024-03-18T03:54:43.530078Z","shell.execute_reply.started":"2024-03-18T03:54:43.527279Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n    path, \n    valid_pct=0.2, \n    item_tfms=Resize(400), \n    batch_tfms=aug_transforms(size=224, min_scale=0.75))\n\ndls.show_batch()","metadata":{"execution":{"iopub.execute_input":"2024-03-16T18:06:22.827858Z","iopub.status.busy":"2024-03-16T18:06:22.826639Z","iopub.status.idle":"2024-03-16T18:06:31.485617Z","shell.execute_reply":"2024-03-16T18:06:31.484544Z","shell.execute_reply.started":"2024-03-16T18:06:22.827814Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn = vision_learner(dls, arch, metrics=error_rate).to_fp16()\nlearn.fine_tune(15, 0.01)","metadata":{"execution":{"iopub.execute_input":"2024-03-16T18:06:58.031758Z","iopub.status.busy":"2024-03-16T18:06:58.030684Z","iopub.status.idle":"2024-03-16T18:29:38.346291Z","shell.execute_reply":"2024-03-16T18:29:38.344003Z","shell.execute_reply.started":"2024-03-16T18:06:58.031717Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.recorder.plot_loss()","metadata":{"execution":{"iopub.execute_input":"2024-03-16T18:31:17.143336Z","iopub.status.busy":"2024-03-16T18:31:17.142896Z","iopub.status.idle":"2024-03-16T18:31:17.361247Z","shell.execute_reply":"2024-03-16T18:31:17.360206Z","shell.execute_reply.started":"2024-03-16T18:31:17.143306Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The model starts overfitting after 11 epochs so I'll train the variants for 10 epochs.","metadata":{}},{"cell_type":"markdown","source":"## Training `vit` Variants","metadata":{}},{"cell_type":"markdown","source":"The final size of the image has to be 224 x 224,  I'll train the following combinations, labeled as Model A, B, C, etc. Something I didn't try with my `convnext` models was to turn of `aug_transforms` and just use something like `Resize` or `RandomResizeCrop` for `batch_tfms`. I'll try that here which will greatly expand my list of options.\n\n- 3 item `method`s\n    - crop\n    - squish\n    - pad\n- 3 `batch` types\n    - None (item `size` must be `224`)\n    - `RandomResizedCropGPU(size=224, min_scale=1.0)` (no other transforms but a crop)\n    - `aug_transforms(224, min_scale=0.75)`\n- 7 item `size`s\n    - 224\n    - 400\n    - 311\n    - (400, 311)\n    - (311, 400)\n    - (320, 512)\n    - (384, 768)\n- 3 progressive resizing\n    - 256 --> (311,400) --> (320,512)\n    - 256 --> (400,311) --> (320,512)\n    - 256 --> 400 --> (320,512)\n\n\nThis will give me a total of 3 item methods x 2 batch type x 6 item sizes + 3 (batch size `None`) + 3 item methods x 2 batch types x 2 progressive resizing types = 57 variants.","metadata":{}},{"cell_type":"markdown","source":"Something I came across while trying to figure out which `batch_tfms` to apply was [this fastai forums post (sign-in required)](https://forums.fast.ai/t/need-help-understanding-datablock-s-batch-tfms/62655) where Zach Mueller explains that a transform will only be applied to an input object `x` if `x` is of the appropriate type. So, for example, `Resize` only works with input types `Image.Image`, `TensorBBox` or `TensorPoint`. My understanding is that by default, an `item_tfms` will include a `IntToFloatTensor` which will convert the `PILImage` to a `TensorImage` before it gets to `batch_tfms` so only transforms that work on `TensorImage`, such as `RandomResizedCropGPU` will be applied. Of course you could define your own custom `item_tfms` and `batch_tfms` to handle whatever however you'd like. ","metadata":{}},{"cell_type":"markdown","source":"I'll set the `seed` so that I can compare validation error rates across models.","metadata":{}},{"cell_type":"code","source":"def train(batch, item, arch='vit_small_patch16_224', accum=4, epochs=10):        \n    dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=item,\n        batch_tfms=batch,\n        bs=64//accum)\n    \n    cbs = GradientAccumulation(64) if accum else []\n    learn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\n    learn.fine_tune(epochs, 0.01)\n    print(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T03:54:51.087486Z","iopub.status.busy":"2024-03-18T03:54:51.086830Z","iopub.status.idle":"2024-03-18T03:54:51.091581Z","shell.execute_reply":"2024-03-18T03:54:51.090895Z","shell.execute_reply.started":"2024-03-18T03:54:51.087461Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model A","metadata":{}},{"cell_type":"code","source":"train(batch=None, item=Resize(224))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T18:36:29.423342Z","iopub.status.busy":"2024-03-16T18:36:29.422797Z","iopub.status.idle":"2024-03-16T18:52:12.640313Z","shell.execute_reply":"2024-03-16T18:52:12.639412Z","shell.execute_reply.started":"2024-03-16T18:36:29.423302Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model B","metadata":{}},{"cell_type":"code","source":"train(batch=None, item=Resize(224, method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T18:52:12.643219Z","iopub.status.busy":"2024-03-16T18:52:12.641992Z","iopub.status.idle":"2024-03-16T19:07:59.827892Z","shell.execute_reply":"2024-03-16T19:07:59.826812Z","shell.execute_reply.started":"2024-03-16T18:52:12.643152Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model C","metadata":{}},{"cell_type":"code","source":"train(batch=None, item=Resize(224, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T19:07:59.829679Z","iopub.status.busy":"2024-03-16T19:07:59.829328Z","iopub.status.idle":"2024-03-16T19:23:44.121242Z","shell.execute_reply":"2024-03-16T19:23:44.120343Z","shell.execute_reply.started":"2024-03-16T19:07:59.829634Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model D","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize(400))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T19:27:00.003528Z","iopub.status.busy":"2024-03-16T19:27:00.003035Z","iopub.status.idle":"2024-03-16T19:45:27.440423Z","shell.execute_reply":"2024-03-16T19:45:27.439153Z","shell.execute_reply.started":"2024-03-16T19:27:00.003491Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model E","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize(400, method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T19:45:27.442855Z","iopub.status.busy":"2024-03-16T19:45:27.442393Z","iopub.status.idle":"2024-03-16T20:03:53.690160Z","shell.execute_reply":"2024-03-16T20:03:53.688748Z","shell.execute_reply.started":"2024-03-16T19:45:27.442823Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model F","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize(400, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T20:03:53.692038Z","iopub.status.busy":"2024-03-16T20:03:53.691701Z","iopub.status.idle":"2024-03-16T20:22:15.387016Z","shell.execute_reply":"2024-03-16T20:22:15.386008Z","shell.execute_reply.started":"2024-03-16T20:03:53.692006Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model G","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize(311))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T20:22:15.390252Z","iopub.status.busy":"2024-03-16T20:22:15.389408Z","iopub.status.idle":"2024-03-16T20:40:17.920751Z","shell.execute_reply":"2024-03-16T20:40:17.919535Z","shell.execute_reply.started":"2024-03-16T20:22:15.390209Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model H","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize(311, method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T20:40:17.922579Z","iopub.status.busy":"2024-03-16T20:40:17.922230Z","iopub.status.idle":"2024-03-16T20:58:22.891231Z","shell.execute_reply":"2024-03-16T20:58:22.889900Z","shell.execute_reply.started":"2024-03-16T20:40:17.922526Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model I","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize(311, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T20:58:22.893376Z","iopub.status.busy":"2024-03-16T20:58:22.893010Z","iopub.status.idle":"2024-03-16T21:16:30.133290Z","shell.execute_reply":"2024-03-16T21:16:30.131620Z","shell.execute_reply.started":"2024-03-16T20:58:22.893335Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model J","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((400, 311)))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T21:16:30.137000Z","iopub.status.busy":"2024-03-16T21:16:30.135326Z","iopub.status.idle":"2024-03-16T21:34:39.409324Z","shell.execute_reply":"2024-03-16T21:34:39.408269Z","shell.execute_reply.started":"2024-03-16T21:16:30.136948Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model K","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((400, 311), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T21:34:39.411442Z","iopub.status.busy":"2024-03-16T21:34:39.410569Z","iopub.status.idle":"2024-03-16T21:52:49.367396Z","shell.execute_reply":"2024-03-16T21:52:49.366559Z","shell.execute_reply.started":"2024-03-16T21:34:39.411398Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model L","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((400, 311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T21:52:49.369079Z","iopub.status.busy":"2024-03-16T21:52:49.368613Z","iopub.status.idle":"2024-03-16T22:10:56.427529Z","shell.execute_reply":"2024-03-16T22:10:56.426523Z","shell.execute_reply.started":"2024-03-16T21:52:49.369048Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model M","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((311, 400)))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T22:10:56.431518Z","iopub.status.busy":"2024-03-16T22:10:56.431214Z","iopub.status.idle":"2024-03-16T22:28:57.654587Z","shell.execute_reply":"2024-03-16T22:28:57.653421Z","shell.execute_reply.started":"2024-03-16T22:10:56.431488Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model N","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((311, 400), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T22:28:57.656339Z","iopub.status.busy":"2024-03-16T22:28:57.656031Z","iopub.status.idle":"2024-03-16T22:47:00.420226Z","shell.execute_reply":"2024-03-16T22:47:00.418676Z","shell.execute_reply.started":"2024-03-16T22:28:57.656309Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model O","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((311, 400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T22:47:00.421981Z","iopub.status.busy":"2024-03-16T22:47:00.421686Z","iopub.status.idle":"2024-03-16T23:05:03.158968Z","shell.execute_reply":"2024-03-16T23:05:03.157533Z","shell.execute_reply.started":"2024-03-16T22:47:00.421950Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model P","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((320, 512)))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T23:05:03.161076Z","iopub.status.busy":"2024-03-16T23:05:03.160638Z","iopub.status.idle":"2024-03-16T23:23:22.090153Z","shell.execute_reply":"2024-03-16T23:23:22.089244Z","shell.execute_reply.started":"2024-03-16T23:05:03.161035Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model Q","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((320, 512), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-16T23:23:22.091809Z","iopub.status.busy":"2024-03-16T23:23:22.091362Z","iopub.status.idle":"2024-03-16T23:41:49.697962Z","shell.execute_reply":"2024-03-16T23:41:49.696543Z","shell.execute_reply.started":"2024-03-16T23:23:22.091777Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model R","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((320, 512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T02:05:36.322877Z","iopub.status.busy":"2024-03-17T02:05:36.321942Z","iopub.status.idle":"2024-03-17T02:25:37.583253Z","shell.execute_reply":"2024-03-17T02:25:37.581601Z","shell.execute_reply.started":"2024-03-17T02:05:36.322877Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model S","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((384, 768)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T02:25:37.587988Z","iopub.status.busy":"2024-03-17T02:25:37.587476Z","iopub.status.idle":"2024-03-17T02:46:22.159631Z","shell.execute_reply":"2024-03-17T02:46:22.157871Z","shell.execute_reply.started":"2024-03-17T02:25:37.587944Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model T","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((384, 768), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T02:46:22.162073Z","iopub.status.busy":"2024-03-17T02:46:22.161637Z","iopub.status.idle":"2024-03-17T03:07:13.739449Z","shell.execute_reply":"2024-03-17T03:07:13.738093Z","shell.execute_reply.started":"2024-03-17T02:46:22.162073Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model U","metadata":{}},{"cell_type":"code","source":"train(batch=aug_transforms(size=224, min_scale=0.75), item=Resize((384, 768), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T03:07:13.743552Z","iopub.status.busy":"2024-03-17T03:07:13.743078Z","iopub.status.idle":"2024-03-17T03:28:10.210370Z","shell.execute_reply":"2024-03-17T03:28:10.209281Z","shell.execute_reply.started":"2024-03-17T03:07:13.743507Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model V","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize(400))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T05:01:15.276519Z","iopub.status.busy":"2024-03-17T05:01:15.275661Z","iopub.status.idle":"2024-03-17T05:18:41.728616Z","shell.execute_reply":"2024-03-17T05:18:41.727083Z","shell.execute_reply.started":"2024-03-17T05:01:15.276519Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model W","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize(400, method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T05:18:41.730444Z","iopub.status.busy":"2024-03-17T05:18:41.730170Z","iopub.status.idle":"2024-03-17T05:36:18.505947Z","shell.execute_reply":"2024-03-17T05:36:18.504994Z","shell.execute_reply.started":"2024-03-17T05:18:41.730413Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model X","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize(400, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T05:36:18.508928Z","iopub.status.busy":"2024-03-17T05:36:18.507977Z","iopub.status.idle":"2024-03-17T05:53:52.920162Z","shell.execute_reply":"2024-03-17T05:53:52.917983Z","shell.execute_reply.started":"2024-03-17T05:36:18.508928Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model Y","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize(311))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T18:48:51.048180Z","iopub.status.busy":"2024-03-17T18:48:51.047503Z","iopub.status.idle":"2024-03-17T18:54:34.915159Z","shell.execute_reply":"2024-03-17T18:54:34.914448Z","shell.execute_reply.started":"2024-03-17T18:48:51.048158Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model Z","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize(311, method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T18:54:34.916957Z","iopub.status.busy":"2024-03-17T18:54:34.916328Z","iopub.status.idle":"2024-03-17T19:00:11.830823Z","shell.execute_reply":"2024-03-17T19:00:11.830240Z","shell.execute_reply.started":"2024-03-17T18:54:34.916907Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AA","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize(311, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:00:11.832720Z","iopub.status.busy":"2024-03-17T19:00:11.831823Z","iopub.status.idle":"2024-03-17T19:05:46.647524Z","shell.execute_reply":"2024-03-17T19:05:46.646787Z","shell.execute_reply.started":"2024-03-17T19:00:11.832692Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AB","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((400,311)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:05:46.652428Z","iopub.status.busy":"2024-03-17T19:05:46.651027Z","iopub.status.idle":"2024-03-17T19:11:17.608339Z","shell.execute_reply":"2024-03-17T19:11:17.607512Z","shell.execute_reply.started":"2024-03-17T19:05:46.652388Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AC","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((400,311), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:11:17.610230Z","iopub.status.busy":"2024-03-17T19:11:17.609674Z","iopub.status.idle":"2024-03-17T19:16:51.533550Z","shell.execute_reply":"2024-03-17T19:16:51.532683Z","shell.execute_reply.started":"2024-03-17T19:11:17.610195Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AD","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:16:51.534795Z","iopub.status.busy":"2024-03-17T19:16:51.534602Z","iopub.status.idle":"2024-03-17T19:22:23.822155Z","shell.execute_reply":"2024-03-17T19:22:23.821282Z","shell.execute_reply.started":"2024-03-17T19:16:51.534775Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AE","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((311, 400)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:22:23.824279Z","iopub.status.busy":"2024-03-17T19:22:23.824009Z","iopub.status.idle":"2024-03-17T19:27:58.490810Z","shell.execute_reply":"2024-03-17T19:27:58.490013Z","shell.execute_reply.started":"2024-03-17T19:22:23.824256Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AF","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((311, 400), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:27:58.492087Z","iopub.status.busy":"2024-03-17T19:27:58.491869Z","iopub.status.idle":"2024-03-17T19:33:40.847676Z","shell.execute_reply":"2024-03-17T19:33:40.846899Z","shell.execute_reply.started":"2024-03-17T19:27:58.492064Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AG","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((311, 400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:33:40.849085Z","iopub.status.busy":"2024-03-17T19:33:40.848902Z","iopub.status.idle":"2024-03-17T19:39:24.286108Z","shell.execute_reply":"2024-03-17T19:39:24.285338Z","shell.execute_reply.started":"2024-03-17T19:33:40.849065Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AH","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((320, 512)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:39:24.290990Z","iopub.status.busy":"2024-03-17T19:39:24.290615Z","iopub.status.idle":"2024-03-17T19:45:20.833400Z","shell.execute_reply":"2024-03-17T19:45:20.832200Z","shell.execute_reply.started":"2024-03-17T19:39:24.290966Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AI","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((320, 512), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:45:20.841311Z","iopub.status.busy":"2024-03-17T19:45:20.841096Z","iopub.status.idle":"2024-03-17T19:51:16.159507Z","shell.execute_reply":"2024-03-17T19:51:16.158700Z","shell.execute_reply.started":"2024-03-17T19:45:20.841283Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AJ","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((320, 512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:51:16.160809Z","iopub.status.busy":"2024-03-17T19:51:16.160569Z","iopub.status.idle":"2024-03-17T19:57:09.214572Z","shell.execute_reply":"2024-03-17T19:57:09.213868Z","shell.execute_reply.started":"2024-03-17T19:51:16.160786Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AK","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((384, 768)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T19:57:09.215720Z","iopub.status.busy":"2024-03-17T19:57:09.215519Z","iopub.status.idle":"2024-03-17T20:03:19.982184Z","shell.execute_reply":"2024-03-17T20:03:19.981562Z","shell.execute_reply.started":"2024-03-17T19:57:09.215701Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AL","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((384, 768), method='squish'))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:03:19.983625Z","iopub.status.busy":"2024-03-17T20:03:19.983057Z","iopub.status.idle":"2024-03-17T20:09:44.267837Z","shell.execute_reply":"2024-03-17T20:09:44.266777Z","shell.execute_reply.started":"2024-03-17T20:03:19.983602Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AM","metadata":{}},{"cell_type":"code","source":"train(batch=RandomResizedCropGPU(size=224, min_scale=1.0), item=Resize((384, 768), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:09:44.269838Z","iopub.status.busy":"2024-03-17T20:09:44.269600Z","iopub.status.idle":"2024-03-17T20:16:05.625202Z","shell.execute_reply":"2024-03-17T20:16:05.624638Z","shell.execute_reply.started":"2024-03-17T20:09:44.269815Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AN","metadata":{}},{"cell_type":"code","source":"arch = 'vit_small_patch16_224'","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:25:53.334061Z","iopub.status.busy":"2024-03-17T20:25:53.333792Z","iopub.status.idle":"2024-03-17T20:25:53.337932Z","shell.execute_reply":"2024-03-17T20:25:53.337215Z","shell.execute_reply.started":"2024-03-17T20:25:53.334043Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:25:56.813609Z","iopub.status.busy":"2024-03-17T20:25:56.813089Z","iopub.status.idle":"2024-03-17T20:28:37.923776Z","shell.execute_reply":"2024-03-17T20:28:37.923009Z","shell.execute_reply.started":"2024-03-17T20:25:56.813586Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((311,400)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:32:40.433111Z","iopub.status.busy":"2024-03-17T20:32:40.432759Z","iopub.status.idle":"2024-03-17T20:37:07.799562Z","shell.execute_reply":"2024-03-17T20:37:07.798759Z","shell.execute_reply.started":"2024-03-17T20:32:40.433084Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:38:51.134317Z","iopub.status.busy":"2024-03-17T20:38:51.133729Z","iopub.status.idle":"2024-03-17T20:45:13.225049Z","shell.execute_reply":"2024-03-17T20:45:13.224315Z","shell.execute_reply.started":"2024-03-17T20:38:51.134283Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:45:28.437787Z","iopub.status.busy":"2024-03-17T20:45:28.436861Z","iopub.status.idle":"2024-03-17T20:45:52.767023Z","shell.execute_reply":"2024-03-17T20:45:52.766247Z","shell.execute_reply.started":"2024-03-17T20:45:28.437743Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AO","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((311,400), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T20:46:56.832066Z","iopub.status.busy":"2024-03-17T20:46:56.830916Z","iopub.status.idle":"2024-03-17T21:01:17.117541Z","shell.execute_reply":"2024-03-17T21:01:17.116634Z","shell.execute_reply.started":"2024-03-17T20:46:56.832019Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AP","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T21:06:50.343395Z","iopub.status.busy":"2024-03-17T21:06:50.342719Z","iopub.status.idle":"2024-03-17T21:20:53.432475Z","shell.execute_reply":"2024-03-17T21:20:53.431684Z","shell.execute_reply.started":"2024-03-17T21:06:50.343364Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AQ","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((311,400)),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T21:22:53.823185Z","iopub.status.busy":"2024-03-17T21:22:53.822627Z","iopub.status.idle":"2024-03-17T21:34:44.906051Z","shell.execute_reply":"2024-03-17T21:34:44.905486Z","shell.execute_reply.started":"2024-03-17T21:22:53.823155Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AR","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((311,400), method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T21:36:02.869209Z","iopub.status.busy":"2024-03-17T21:36:02.868767Z","iopub.status.idle":"2024-03-17T21:48:06.406775Z","shell.execute_reply":"2024-03-17T21:48:06.405804Z","shell.execute_reply.started":"2024-03-17T21:36:02.869182Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AS","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T21:52:12.169156Z","iopub.status.busy":"2024-03-17T21:52:12.168528Z","iopub.status.idle":"2024-03-17T22:04:35.683580Z","shell.execute_reply":"2024-03-17T22:04:35.682758Z","shell.execute_reply.started":"2024-03-17T21:52:12.169126Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AT","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T22:33:10.620660Z","iopub.status.busy":"2024-03-17T22:33:10.619970Z","iopub.status.idle":"2024-03-17T22:47:06.811200Z","shell.execute_reply":"2024-03-17T22:47:06.810328Z","shell.execute_reply.started":"2024-03-17T22:33:10.620636Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T06:19:45.905426Z","iopub.status.busy":"2024-03-18T06:19:45.905200Z","iopub.status.idle":"2024-03-18T06:33:36.789969Z","shell.execute_reply":"2024-03-18T06:33:36.789327Z","shell.execute_reply.started":"2024-03-18T06:19:45.905426Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AU","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-17T22:47:06.813540Z","iopub.status.busy":"2024-03-17T22:47:06.812757Z","iopub.status.idle":"2024-03-17T23:01:12.500099Z","shell.execute_reply":"2024-03-17T23:01:12.499317Z","shell.execute_reply.started":"2024-03-17T22:47:06.813516Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T06:05:44.611469Z","iopub.status.busy":"2024-03-18T06:05:44.611119Z","iopub.status.idle":"2024-03-18T06:19:45.904226Z","shell.execute_reply":"2024-03-18T06:19:45.903671Z","shell.execute_reply.started":"2024-03-18T06:05:44.611447Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AV","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T05:51:44.388378Z","iopub.status.busy":"2024-03-18T05:51:44.388163Z","iopub.status.idle":"2024-03-18T06:05:44.610051Z","shell.execute_reply":"2024-03-18T06:05:44.609525Z","shell.execute_reply.started":"2024-03-18T05:51:44.388356Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AW","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311)),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T05:40:01.304206Z","iopub.status.busy":"2024-03-18T05:40:01.303724Z","iopub.status.idle":"2024-03-18T05:51:44.387153Z","shell.execute_reply":"2024-03-18T05:51:44.386419Z","shell.execute_reply.started":"2024-03-18T05:40:01.304183Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AX","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311), method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T05:28:05.705631Z","iopub.status.busy":"2024-03-18T05:28:05.705379Z","iopub.status.idle":"2024-03-18T05:40:01.302629Z","shell.execute_reply":"2024-03-18T05:40:01.301761Z","shell.execute_reply.started":"2024-03-18T05:28:05.705612Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AY","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((400,311), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T05:16:13.057397Z","iopub.status.busy":"2024-03-18T05:16:13.056865Z","iopub.status.idle":"2024-03-18T05:28:05.704567Z","shell.execute_reply":"2024-03-18T05:28:05.704055Z","shell.execute_reply.started":"2024-03-18T05:16:13.057371Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model AZ","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(400),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T05:02:14.744525Z","iopub.status.busy":"2024-03-18T05:02:14.743918Z","iopub.status.idle":"2024-03-18T05:16:13.055190Z","shell.execute_reply":"2024-03-18T05:16:13.054434Z","shell.execute_reply.started":"2024-03-18T05:02:14.744499Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model BA","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(400, method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T04:48:06.566904Z","iopub.status.busy":"2024-03-18T04:48:06.566464Z","iopub.status.idle":"2024-03-18T05:02:14.742966Z","shell.execute_reply":"2024-03-18T05:02:14.742402Z","shell.execute_reply.started":"2024-03-18T04:48:06.566882Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model BB","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(400, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=aug_transforms(size=224, min_scale=0.75),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T04:34:05.009606Z","iopub.status.busy":"2024-03-18T04:34:05.009019Z","iopub.status.idle":"2024-03-18T04:48:06.565177Z","shell.execute_reply":"2024-03-18T04:48:06.564490Z","shell.execute_reply.started":"2024-03-18T04:34:05.009578Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model BC","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(400),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512)),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T04:19:30.570179Z","iopub.status.busy":"2024-03-18T04:19:30.569496Z","iopub.status.idle":"2024-03-18T04:31:21.974145Z","shell.execute_reply":"2024-03-18T04:31:21.973297Z","shell.execute_reply.started":"2024-03-18T04:19:30.570154Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model BD","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(400, method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method='squish'),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T03:55:23.104112Z","iopub.status.busy":"2024-03-18T03:55:23.103471Z","iopub.status.idle":"2024-03-18T04:07:35.187228Z","shell.execute_reply":"2024-03-18T04:07:35.186600Z","shell.execute_reply.started":"2024-03-18T03:55:23.104086Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Model BE","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(256, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n    \ncbs = GradientAccumulation(64)\nlearn = vision_learner(dls, arch, metrics=error_rate, cbs=cbs).to_fp16()\nlearn.fine_tune(4, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize(400, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(7, 0.01)\n\ndls = ImageDataLoaders.from_folder(\n        path, \n        valid_pct=0.2, \n        seed=42,\n        item_tfms=Resize((320,512), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros),\n        batch_tfms=RandomResizedCropGPU(size=224, min_scale=1.0),\n        bs=64//4)\n\nlearn.dls = dls\nlearn.fine_tune(10, 0.01)\nprint(error_rate(*learn.tta(dl=dls.valid)))","metadata":{"execution":{"iopub.execute_input":"2024-03-18T04:07:35.188623Z","iopub.status.busy":"2024-03-18T04:07:35.188431Z","iopub.status.idle":"2024-03-18T04:19:30.568620Z","shell.execute_reply":"2024-03-18T04:19:30.567954Z","shell.execute_reply.started":"2024-03-18T04:07:35.188604Z"}},"execution_count":null,"outputs":[]}]}