{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"}],"dockerImageVersionId":30665,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import warnings\nimport timm\nfrom fastai.vision.all import *\nfrom fastcore.parallel import *\n\npath = Path('/kaggle/input/hms-harmful-brain-activity-classification')\n\npath.ls()","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-03-07T02:53:48.470373Z","iopub.execute_input":"2024-03-07T02:53:48.470766Z","iopub.status.idle":"2024-03-07T02:53:48.477991Z","shell.execute_reply.started":"2024-03-07T02:53:48.470741Z","shell.execute_reply":"2024-03-07T02:53:48.477135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Background\n\nIn this notebook, I will plan out a series of experiments using small architectures. \n\nI will heavily reference Jeremy Howard's approach in [Small models: Road to the Top, Part 2](https://www.kaggle.com/code/jhoward/small-models-road-to-the-top-part-2) where he experiments with different small architectures and image processing techniques to determine the best set of training parameters to move forward with. I'll also reference [my experiments as I followed along his Live Coding 10 video](https://vishalbakshi.github.io/blog/posts/2024-02-05-paddy-part-3/). Finally, I'll export a dataset of images that I can reuse in these experiments so I don't have to spend 5-ish minutes recreating the images every time I restart the kernel.\n\nI am also referencing the following notebooks:\n\n- [HMS - HBAC - Fastai Starter](https://www.kaggle.com/code/sonujha090/hms-hbac-fastai-starter)\n- [HMS-HBAC: KerasCV Starter Notebook](https://www.kaggle.com/code/awsaf49/hms-hbac-kerascv-starter-notebook)\n\nSome of my previous experimentation can be seen in the following notebooks I've created:\n- fastai resnet34 Starter ([Train](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-resnet34-starter-train) and [Submit](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-resnet34-starter-submit))---I average the spectrograms to create training images and get a Public score of 2.02.\n- fastai Stacked Images ([Train](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-stacked-images-train) and [Submit](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-stacked-images-submit))---instead of averaging the spectrograms, I stack them together as done in the two notebooks referenced above. This improves my Public score from 2.09 to 2.02.\n- fastai TTA ([Submit](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-tta-submit))---I use Test Time Augmentation during inference. My Public score stayed at 2.02.","metadata":{}},{"cell_type":"markdown","source":"## Architectures and Image Processing","metadata":{}},{"cell_type":"markdown","source":"Jeremy has put together this fantastic notebook---[The best vision models for fine-tuning](https://www.kaggle.com/code/jhoward/the-best-vision-models-for-fine-tuning)---in which he shows the performance of different models (that are pretrained on ImageNet) after fine-tuning them on the PETS and PLANET datasets.\n\nI'll use the same four architecture families as Jeremy has done in his Road to the Top series:\n\n- convnext \n- vit\n- swin\n- swinv2\n\nI'll create a new notebook for each one so my notebooks stay bite-sized and don't get bloated.\n\nFor each architecture, I'll try different `item_tfms`, trying both rectangular and square images before they are batched together:\n\n- `Resize(400, 'squish')` (400 px is the fixed height for all images)\n- `Resize(311, 'squish')` (311 px is the median width)\n- `Resize(400)` (crop)\n- `Resize(311)` (crop)\n- `Resize((311,400), method=ResizeMethod.Pad, pad_mode=PadMode.Zeros)`\n\nI'll try the following batch transforms for models with fixed image dimension (like vit and swinv2):\n\n- `aug_transforms(size=#, min_scale=0.75)` \n\nFor convnext, which Jeremy recommends to use multiples of 32 for at least one of the dimensions, I'll try the following batch transforms:\n\n- `aug_transforms(size=384, min_scale=0.75)` (384 is the closest multiple of 32 to 400)\n- `aug_transforms(size=288, min_scale=0.75)` (288 is the closest multiple of 32 to 311)\n- `aug_transforms(size=(384,323, min_scale=0.75)` (I'll double check, but I think fastai's `aug_transforms` takes (height, width) as the `size` tuple).","metadata":{}},{"cell_type":"markdown","source":"## Randomization","metadata":{}},{"cell_type":"markdown","source":"Generally speaking, I don't set a random seed during training since as Jeremy as recommended in the [fastai course](https://course.fast.ai/), I like to see how the model varies between trainings. Sometimes I'll train a model 3-4 times and view the validation and training loss plots to ensure that the training is stable and somewhat consistent. I'm still building intuition around what stable and consistent means.\n\nHowever, there's a time and a place for setting the random seed and this is it! I want to compare the performance of different architectures and item/batch transforms so I'll set the seed so that I get the same training/validation splits each run. That way, I'm measuring the metric on the same validation set.","metadata":{}},{"cell_type":"markdown","source":"## Switching to Error Rate","metadata":{}},{"cell_type":"markdown","source":"In the previous notebooks I used accuracy as my validation metric. I'll switch to error rate since I expect the differences in performance to be relatively small, and small differences in error rate can still be significant---for example, an accuracy of 99% and 99.9% have an error rate of 1% and 0.1% respectively, a 10x difference.","metadata":{}},{"cell_type":"markdown","source":"## Exporting an Image Dataset","metadata":{}},{"cell_type":"markdown","source":"Since I'm going to be creating 4+ more notebooks for these experiments, I'm going to export a dataset of training images, with separate folders for each class.","metadata":{}},{"cell_type":"markdown","source":"Note that in the following code, I have removed the `eeg_id` column for now since there are cases with multiple rows with the same `spectrogram_id` for a given `eeg_id`. In a future notebook I'll use EEGs as inputs. For now, I'm just using unique `spectrogram_id` values and associated `target` values.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(path/'train.csv')\n    \ncols = ['spectrogram_id', 'seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']\nagg_funcs = {c: 'sum' for c in cols if 'vote' in c}\n\nunique_df = df[cols].groupby(['spectrogram_id'], as_index=False).agg(agg_funcs)\nunique_df['target'] = unique_df[[c for c in cols if 'vote' in c]].idxmax(axis=1)\n\nunique_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:06:22.741229Z","iopub.execute_input":"2024-03-07T03:06:22.742226Z","iopub.status.idle":"2024-03-07T03:06:22.931064Z","shell.execute_reply.started":"2024-03-07T03:06:22.742181Z","shell.execute_reply":"2024-03-07T03:06:22.929773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:10:04.692785Z","iopub.execute_input":"2024-03-07T03:10:04.693660Z","iopub.status.idle":"2024-03-07T03:10:04.702630Z","shell.execute_reply.started":"2024-03-07T03:10:04.693626Z","shell.execute_reply":"2024-03-07T03:10:04.701501Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create output folders to hold spectrograms\nSPEC_DIR = \"/kaggle/working\"\nos.makedirs(SPEC_DIR+'/train_spectrograms', exist_ok=True)\n\nfor targ in ['seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']:\n    os.makedirs(SPEC_DIR+'/train_spectrograms'+'/'+targ, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:20:26.180945Z","iopub.execute_input":"2024-03-07T03:20:26.181699Z","iopub.status.idle":"2024-03-07T03:20:26.188146Z","shell.execute_reply.started":"2024-03-07T03:20:26.181663Z","shell.execute_reply":"2024-03-07T03:20:26.186944Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Path(\"/kaggle/working/train_spectrograms\").ls()","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:20:58.445080Z","iopub.execute_input":"2024-03-07T03:20:58.445457Z","iopub.status.idle":"2024-03-07T03:20:58.453329Z","shell.execute_reply.started":"2024-03-07T03:20:58.445429Z","shell.execute_reply":"2024-03-07T03:20:58.452248Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_spec(spec_id, split=\"train\"):\n    # read the data\n    data = pd.read_parquet(path/f'{split}_spectrograms'/f'{spec_id}.parquet')\n    \n    # read the label\n    label = unique_df[unique_df.spectrogram_id == spec_id][\"target\"].item()\n    \n    # replace NA with 0\n    data = data.fillna(0)\n    \n    # convert DataFrame to array\n    data = data.values[:, 1:]\n    \n    # transpose\n    data = data.T\n    data = data.astype(\"float32\")\n    \n    # convert array to PILImage\n    im = PILImage.create(Image.fromarray((data * 255).astype(np.uint8)))\n    im.save(f\"{SPEC_DIR}/{split}_spectrograms/{label}/{spec_id}.png\")","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:21:24.456697Z","iopub.execute_input":"2024-03-07T03:21:24.457095Z","iopub.status.idle":"2024-03-07T03:21:24.464434Z","shell.execute_reply.started":"2024-03-07T03:21:24.457066Z","shell.execute_reply":"2024-03-07T03:21:24.463317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spec_ids = df[\"spectrogram_id\"].unique()\nlen(spec_ids)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:21:26.824581Z","iopub.execute_input":"2024-03-07T03:21:26.825711Z","iopub.status.idle":"2024-03-07T03:21:26.840023Z","shell.execute_reply.started":"2024-03-07T03:21:26.825664Z","shell.execute_reply":"2024-03-07T03:21:26.838907Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"warnings.filterwarnings(\"ignore\")\nparallel(process_spec, spec_ids, split='train', n_workers=4)\nwarnings.filterwarnings(\"default\")","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:22:09.059742Z","iopub.execute_input":"2024-03-07T03:22:09.060365Z","iopub.status.idle":"2024-03-07T03:26:49.192944Z","shell.execute_reply.started":"2024-03-07T03:22:09.060324Z","shell.execute_reply":"2024-03-07T03:26:49.191728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll spot-check the images to make sure that they are in the correct folders (class labels).","metadata":{}},{"cell_type":"code","source":"for vote in ['seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']:\n    for fpath in (Path(\"/kaggle/working/train_spectrograms\")/vote).ls()[:3]:\n        print(vote, vote == unique_df[unique_df.spectrogram_id == int(fpath.stem)]['target'].item())","metadata":{"execution":{"iopub.status.busy":"2024-03-07T03:39:13.622864Z","iopub.execute_input":"2024-03-07T03:39:13.623248Z","iopub.status.idle":"2024-03-07T03:39:13.666573Z","shell.execute_reply.started":"2024-03-07T03:39:13.623221Z","shell.execute_reply":"2024-03-07T03:39:13.665482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll also make sure that I captured all of the images in the `DataFrame`:","metadata":{}},{"cell_type":"code","source":"files = get_image_files(Path(\"/kaggle/working/train_spectrograms\"))\nlen(files)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T05:06:51.087689Z","iopub.execute_input":"2024-03-07T05:06:51.088541Z","iopub.status.idle":"2024-03-07T05:06:51.185099Z","shell.execute_reply.started":"2024-03-07T05:06:51.088498Z","shell.execute_reply":"2024-03-07T05:06:51.183994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(unique_df)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T05:06:58.999781Z","iopub.execute_input":"2024-03-07T05:06:59.000146Z","iopub.status.idle":"2024-03-07T05:06:59.006383Z","shell.execute_reply.started":"2024-03-07T05:06:59.000116Z","shell.execute_reply":"2024-03-07T05:06:59.005473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'm also curious if all of the images are sized the same, I'll check that now (interesting to note---using a for-loop took a few seconds _less_ than using `fastcore.parallel`). As Jeremy mentioned in [Live Coding 8](https://vishalbakshi.github.io/blog/posts/2024-02-05-paddy-part-1/#setup), if most of the time is spent reading the image from the disk, doing it in parallel won’t be much faster.","metadata":{}},{"cell_type":"code","source":"def f(o): return (o, PILImage.create(o).size)\nsizes = parallel(f, files, n_workers=4)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:07:45.112264Z","iopub.execute_input":"2024-03-07T04:07:45.112632Z","iopub.status.idle":"2024-03-07T04:08:15.301273Z","shell.execute_reply.started":"2024-03-07T04:07:45.112603Z","shell.execute_reply":"2024-03-07T04:08:15.300001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_sizes = pd.DataFrame({'sizes': sizes})\nimg_sizes['img_path'] = img_sizes['sizes'].apply(lambda x: x[0])\nimg_sizes['width'] = img_sizes['sizes'].apply(lambda x: x[1][0])\nimg_sizes['height'] = img_sizes['sizes'].apply(lambda x: x[1][1])\nimg_sizes = img_sizes.drop('sizes', axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:09:31.387781Z","iopub.execute_input":"2024-03-07T04:09:31.388479Z","iopub.status.idle":"2024-03-07T04:09:31.423904Z","shell.execute_reply.started":"2024-03-07T04:09:31.388446Z","shell.execute_reply":"2024-03-07T04:09:31.422719Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_sizes.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:09:32.322338Z","iopub.execute_input":"2024-03-07T04:09:32.323124Z","iopub.status.idle":"2024-03-07T04:09:32.333629Z","shell.execute_reply.started":"2024-03-07T04:09:32.323091Z","shell.execute_reply":"2024-03-07T04:09:32.332480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"All of the images are 400 pixels in height:","metadata":{}},{"cell_type":"code","source":"img_sizes.height.describe()","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:09:37.377883Z","iopub.execute_input":"2024-03-07T04:09:37.378841Z","iopub.status.idle":"2024-03-07T04:09:37.389754Z","shell.execute_reply.started":"2024-03-07T04:09:37.378809Z","shell.execute_reply":"2024-03-07T04:09:37.388698Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is considerable variance in the width, with some images being up to 9116 pixels wide:","metadata":{}},{"cell_type":"code","source":"img_sizes.width.describe()","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:09:40.961055Z","iopub.execute_input":"2024-03-07T04:09:40.962004Z","iopub.status.idle":"2024-03-07T04:09:40.974710Z","shell.execute_reply.started":"2024-03-07T04:09:40.961957Z","shell.execute_reply":"2024-03-07T04:09:40.973624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here's the widest image:","metadata":{}},{"cell_type":"code","source":"PILImage.create(img_sizes.query(\"width == 9116\").img_path.item())","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:10:13.861355Z","iopub.execute_input":"2024-03-07T04:10:13.862186Z","iopub.status.idle":"2024-03-07T04:10:15.374482Z","shell.execute_reply.started":"2024-03-07T04:10:13.862147Z","shell.execute_reply":"2024-03-07T04:10:15.373163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And here is the narrowest image:","metadata":{}},{"cell_type":"code","source":"PILImage.create(img_sizes.query(\"width == 300\").img_path.unique()[0])","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:11:40.304877Z","iopub.execute_input":"2024-03-07T04:11:40.305258Z","iopub.status.idle":"2024-03-07T04:11:40.370753Z","shell.execute_reply.started":"2024-03-07T04:11:40.305231Z","shell.execute_reply":"2024-03-07T04:11:40.369754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'm honestly not sure what to make of these width differences right now, as I haven't spent too much time understanding the details of the spectrogram frequencies. If I get to a point where my image classification models are competing well, I'll dig into data cleaning then. \n\nHowever, this does make me question whether randomly selecting a validation set is appropriate for this problem. Let's see how these image widths are distributed:","metadata":{}},{"cell_type":"code","source":"import plotly.express as px\npx.box(img_sizes, 'width')","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:42:10.794333Z","iopub.execute_input":"2024-03-07T04:42:10.795091Z","iopub.status.idle":"2024-03-07T04:42:12.951277Z","shell.execute_reply.started":"2024-03-07T04:42:10.795057Z","shell.execute_reply":"2024-03-07T04:42:12.950390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(img_sizes[img_sizes.width > 433])/len(img_sizes)","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:43:27.981404Z","iopub.execute_input":"2024-03-07T04:43:27.981769Z","iopub.status.idle":"2024-03-07T04:43:27.991193Z","shell.execute_reply.started":"2024-03-07T04:43:27.981742Z","shell.execute_reply":"2024-03-07T04:43:27.990092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"16.5% of the images are outliers. That's significant enough that it could saturate my validation set with images different than the training data. I will consider the following approaches in my experiments, documenting the erorr rate and Public score at each step to understand its impact:\n\n- Keep images with an outlier width. Randomly create validation set.\n- Remove images with an outlier width. Randomly create validation set.\n- Keep images with an outlier width, ensure equal percentage of outliers in training and validation set.\n\nThe last approach is tricky---what if the outliers are grouped together in one particular category? I'll look at that next:","metadata":{}},{"cell_type":"code","source":"img_sizes['target'] = img_sizes.img_path.apply(lambda x: x.parts[-2])","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:52:08.279993Z","iopub.execute_input":"2024-03-07T04:52:08.280353Z","iopub.status.idle":"2024-03-07T04:52:08.329766Z","shell.execute_reply.started":"2024-03-07T04:52:08.280325Z","shell.execute_reply":"2024-03-07T04:52:08.328704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_sizes.groupby('target').agg({'width': ['mean', 'median', 'max', 'std']})","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:53:50.184494Z","iopub.execute_input":"2024-03-07T04:53:50.184877Z","iopub.status.idle":"2024-03-07T04:53:50.211653Z","shell.execute_reply.started":"2024-03-07T04:53:50.184847Z","shell.execute_reply":"2024-03-07T04:53:50.210633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"All of the medians are relatively close to each other, which is a good sign. `seizure_vote` has the narrowest maximum width at 2631 px. Unsurpsingly, it has the smallest standard deviation.","metadata":{}},{"cell_type":"markdown","source":"There are only 16 images with a width larger than 2631 px.","metadata":{}},{"cell_type":"code","source":"len(img_sizes[img_sizes.width > 2631])","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:56:00.457141Z","iopub.execute_input":"2024-03-07T04:56:00.457499Z","iopub.status.idle":"2024-03-07T04:56:00.465757Z","shell.execute_reply.started":"2024-03-07T04:56:00.457471Z","shell.execute_reply":"2024-03-07T04:56:00.464592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If I remove just those images, the standard deviation comes down for the other 5 classes, and the `max` widths are now pretty close.","metadata":{}},{"cell_type":"code","source":"img_sizes[img_sizes.width <= 2631].groupby('target').agg({'width': ['mean', 'median', 'max', 'std']})","metadata":{"execution":{"iopub.status.busy":"2024-03-07T04:56:44.267023Z","iopub.execute_input":"2024-03-07T04:56:44.267780Z","iopub.status.idle":"2024-03-07T04:56:44.289218Z","shell.execute_reply.started":"2024-03-07T04:56:44.267748Z","shell.execute_reply":"2024-03-07T04:56:44.288153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, I'll look at how the images with a width larger than the upper fence of 433 px are distributed across classes:","metadata":{}},{"cell_type":"code","source":"img_sizes.groupby('target').agg({'width': 'count'})","metadata":{"execution":{"iopub.status.busy":"2024-03-07T05:03:56.503080Z","iopub.execute_input":"2024-03-07T05:03:56.504062Z","iopub.status.idle":"2024-03-07T05:03:56.517294Z","shell.execute_reply.started":"2024-03-07T05:03:56.504021Z","shell.execute_reply":"2024-03-07T05:03:56.516238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_sizes[img_sizes.width > 433].groupby('target').agg({'width': 'count'})","metadata":{"execution":{"iopub.status.busy":"2024-03-07T05:01:27.052983Z","iopub.execute_input":"2024-03-07T05:01:27.053379Z","iopub.status.idle":"2024-03-07T05:01:27.067390Z","shell.execute_reply.started":"2024-03-07T05:01:27.053342Z","shell.execute_reply":"2024-03-07T05:01:27.066404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is a significant difference between the lowest count (`grda_vote` at 176) and the highest count (`other_vote` at 748) for images with a greater than 433 px width. In general there is a disproportionate amount of `other_vote` classes. After I run my experiments with small models (and large models) I'll look into weighing the classes accordingly and seeing how that impacts error rate and Public score.\n\n---","metadata":{}}]}