{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":59093,"databundleVersionId":7469972,"sourceType":"competition"},{"sourceId":13831,"sourceType":"modelInstanceVersion","isSourceIdPinned":true,"modelInstanceId":11450}],"dockerImageVersionId":30665,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Setup","metadata":{}},{"cell_type":"code","source":"import warnings\nimport timm\nimport io\nimport joblib\nfrom tqdm import tqdm\nfrom fastai.vision.all import *\n\npath = Path('/kaggle/input/hms-harmful-brain-activity-classification')\n\npath.ls()","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2024-03-05T16:27:14.015207Z","iopub.execute_input":"2024-03-05T16:27:14.015715Z","iopub.status.idle":"2024-03-05T16:27:25.569896Z","shell.execute_reply.started":"2024-03-05T16:27:14.015684Z","shell.execute_reply":"2024-03-05T16:27:25.569012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Background\n\nIn this notebook, I will train (and export) a resnet34 image classification model and make some of my [first starter notebook's](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-starter-notebook) code more efficient.\n\nI also want to share that I found [a discussion post](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/475467) which confirmed that you can train and export a model in one notebook (with internet access **enabled**) and import it and use it for submission in another notebook (with internet access **disabled**), and that you are allowed to use freely and publicly available pretrained models (such as those in the `timm` library). That is how I will move forward in this competition. My training notebook names will end with [Train] and my submission notebooks with [Submit]. For example, the submission notebook that complements this one is titled [HMS-HBAC fastai Resnet34 Starter [Submit]](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-resnet34-starter-submit).\n\nI also came across a [request form](https://www.kaggle.com/request-to-make-your-model-public) to publish your model publicly on Kaggle. I'm unclear if finetuned models for this competition have to be publicly available in order to follow competition rules, so to err on the side of caution, I'll submit a request to publicly publish the model I train in this starter notebook. You only have to request once, after which you can publish future models without making requests.\n\nOther than training a resnet34, I want to make more efficient my process of converting spectrogram parquet files into images, referencing [Sonu Jha's fastai Starter Notebook](https://www.kaggle.com/code/sonujha090/hms-hbac-fastai-starter/notebook).","metadata":{}},{"cell_type":"markdown","source":"## Converting Training and Test Data to Images","metadata":{}},{"cell_type":"markdown","source":"I'll start by comparing the difference in execution time between my code and the reference code for converting spectrograms to images.","metadata":{}},{"cell_type":"markdown","source":"Here is my code to average the spectrogram parquet data across the four regions:","metadata":{}},{"cell_type":"code","source":"def remove_col_prefix(df):\n    df.columns = df.columns.str.replace(r'^[A-Z]+_', '', regex=True)\n    return df\n\ndef avg_sgram(sgram_path):\n    # read the parquet file and separate into four DataFrames\n    sample_spect = pd.read_parquet(sgram_path)\n    \n    split_spect = {\n        \"LL\": sample_spect.filter(regex='^LL', axis=1),\n        \"RL\": sample_spect.filter(regex='^RL', axis=1),\n        \"RP\": sample_spect.filter(regex='^RP', axis=1),\n        \"LP\": sample_spect.filter(regex='^LP', axis=1),\n    }\n\n    # concanate the four DataFrames with column prefixes removed\n    sgram_avg_df = pd.concat([remove_col_prefix(df) for df in split_spect.values()])\n    sgram_avg_df = sgram_avg_df.groupby(sgram_avg_df.index).mean()\n    \n    return sgram_avg_df","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:30.886336Z","iopub.execute_input":"2024-03-05T16:27:30.887270Z","iopub.status.idle":"2024-03-05T16:27:30.894165Z","shell.execute_reply.started":"2024-03-05T16:27:30.887236Z","shell.execute_reply":"2024-03-05T16:27:30.893250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And here is a new function to save that data as an image","metadata":{}},{"cell_type":"code","source":"def save_sgram_to_image(sgram_path):\n    # get average spectrogram \n    data = avg_sgram(sgram_path)\n    \n    # plot the spectrogram\n    fig,ax = plt.subplots(1, figsize=(2, 2))\n    fig.subplots_adjust(left=0,right=1,bottom=0,top=1)\n\n    ax.axis('off')\n    ax.imshow(np.log(data).T, cmap='viridis', aspect='auto', origin='lower');\n    \n    img_buf = io.BytesIO()\n    fig.savefig(img_buf, format='png')\n    im = PILImage.create(img_buf)\n    \n    plt.close(fig)\n    \n    return im","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:31.934782Z","iopub.execute_input":"2024-03-05T16:27:31.935495Z","iopub.status.idle":"2024-03-05T16:27:31.942235Z","shell.execute_reply.started":"2024-03-05T16:27:31.935461Z","shell.execute_reply":"2024-03-05T16:27:31.940985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here's an example of an image it generates:","metadata":{}},{"cell_type":"code","source":"sgram_path = path/'train_spectrograms'/'1111500860.parquet'","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:34.075838Z","iopub.execute_input":"2024-03-05T16:27:34.076469Z","iopub.status.idle":"2024-03-05T16:27:34.080614Z","shell.execute_reply.started":"2024-03-05T16:27:34.076439Z","shell.execute_reply":"2024-03-05T16:27:34.079673Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_sgram_to_image(sgram_path)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:34.559202Z","iopub.execute_input":"2024-03-05T16:27:34.560009Z","iopub.status.idle":"2024-03-05T16:27:34.917860Z","shell.execute_reply.started":"2024-03-05T16:27:34.559976Z","shell.execute_reply":"2024-03-05T16:27:34.916995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_sgram_to_image(sgram_path).size","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:37.739303Z","iopub.execute_input":"2024-03-05T16:27:37.740077Z","iopub.status.idle":"2024-03-05T16:27:37.839934Z","shell.execute_reply.started":"2024-03-05T16:27:37.740044Z","shell.execute_reply":"2024-03-05T16:27:37.839079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll time it to see how long it takes on average to create an image from a filepath:","metadata":{}},{"cell_type":"code","source":"warnings.filterwarnings(\"ignore\")\n%timeit save_sgram_to_image(sgram_path)\nwarnings.filterwarnings(\"default\")","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:41.610559Z","iopub.execute_input":"2024-03-05T16:27:41.610910Z","iopub.status.idle":"2024-03-05T16:27:48.659509Z","shell.execute_reply.started":"2024-03-05T16:27:41.610885Z","shell.execute_reply":"2024-03-05T16:27:48.658555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll reference [Sonu Jha's notebook](https://www.kaggle.com/code/sonujha090/hms-hbac-fastai-starter/notebook) and apply that concept to my average spectrogram:","metadata":{}},{"cell_type":"code","source":"def process_spec(spec_id, split=\"train\"):\n    # take the average value of four regions' spectrogram data\n    data = avg_sgram(path/f'{split}_spectrograms'/f'{spec_id}.parquet')\n    \n    # replace NA with 0\n    data = data.fillna(0)\n    \n    # convert DataFrame to array\n    data = data.values[:, 1:]\n    data = data.astype(\"float32\")\n    \n    # convert array to PILImage\n    im = PILImage.create(Image.fromarray((data * 255).astype(np.uint8)))\n    return im","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:50.895339Z","iopub.execute_input":"2024-03-05T16:27:50.896058Z","iopub.status.idle":"2024-03-05T16:27:50.901954Z","shell.execute_reply.started":"2024-03-05T16:27:50.896023Z","shell.execute_reply":"2024-03-05T16:27:50.900951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here's one image:","metadata":{}},{"cell_type":"code","source":"process_spec('1111500860')","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:52.634257Z","iopub.execute_input":"2024-03-05T16:27:52.635007Z","iopub.status.idle":"2024-03-05T16:27:52.694664Z","shell.execute_reply.started":"2024-03-05T16:27:52.634975Z","shell.execute_reply":"2024-03-05T16:27:52.693730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here's how long it takes:","metadata":{}},{"cell_type":"code","source":"warnings.filterwarnings(\"ignore\")\n%timeit process_spec('1111500860')\nwarnings.filterwarnings(\"default\")","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:54.250641Z","iopub.execute_input":"2024-03-05T16:27:54.251676Z","iopub.status.idle":"2024-03-05T16:27:57.475249Z","shell.execute_reply.started":"2024-03-05T16:27:54.251640Z","shell.execute_reply":"2024-03-05T16:27:57.474271Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This method takes about half the time as my method, and it creates a differently sized image (456 x 99) than mine (200 x 200). I'll go with the time savings for now as I can use `item_tfms` in the fastai `DataBlock` to resize the image as needed. As done in the reference code, I'll create temporary folders to store the images:","metadata":{}},{"cell_type":"code","source":"# create temporary folders to hold spectrograms\nSPEC_DIR = \"/tmp/dataset/hms-hbac\"\nos.makedirs(SPEC_DIR+'/train_spectrograms', exist_ok=True)\nos.makedirs(SPEC_DIR+'/test_spectrograms', exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:27:59.826991Z","iopub.execute_input":"2024-03-05T16:27:59.827393Z","iopub.status.idle":"2024-03-05T16:27:59.834147Z","shell.execute_reply.started":"2024-03-05T16:27:59.827344Z","shell.execute_reply":"2024-03-05T16:27:59.833292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And add a line to the `process_spec` function to save the image:","metadata":{}},{"cell_type":"code","source":"def process_spec(spec_id, split=\"train\"):\n    # take the average value of four regions' spectrogram data\n    data = avg_sgram(path/f'{split}_spectrograms'/f'{spec_id}.parquet')\n    \n    # replace NA with 0\n    data = data.fillna(0)\n    \n    # convert DataFrame to array\n    data = data.values[:, 1:]\n    data = data.astype(\"float32\")\n    \n    # convert array to PILImage and save\n    im = PILImage.create(Image.fromarray((data * 255).astype(np.uint8)))\n    im.save(f\"{SPEC_DIR}/{split}_spectrograms/{spec_id}.png\")","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:28:01.998841Z","iopub.execute_input":"2024-03-05T16:28:01.999695Z","iopub.status.idle":"2024-03-05T16:28:02.005690Z","shell.execute_reply.started":"2024-03-05T16:28:01.999659Z","shell.execute_reply":"2024-03-05T16:28:02.004708Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll load the training data so I can get all of the `spectrogram_id` values:","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(path/'train.csv')\ndf.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:28:03.922477Z","iopub.execute_input":"2024-03-05T16:28:03.923133Z","iopub.status.idle":"2024-03-05T16:28:04.178114Z","shell.execute_reply.started":"2024-03-05T16:28:03.923102Z","shell.execute_reply":"2024-03-05T16:28:04.177136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spec_ids = df[\"spectrogram_id\"].unique()","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:28:04.274052Z","iopub.execute_input":"2024-03-05T16:28:04.274683Z","iopub.status.idle":"2024-03-05T16:28:04.281666Z","shell.execute_reply.started":"2024-03-05T16:28:04.274654Z","shell.execute_reply":"2024-03-05T16:28:04.280908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(spec_ids)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:28:06.230361Z","iopub.execute_input":"2024-03-05T16:28:06.230735Z","iopub.status.idle":"2024-03-05T16:28:06.236773Z","shell.execute_reply.started":"2024-03-05T16:28:06.230707Z","shell.execute_reply":"2024-03-05T16:28:06.235846Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And then save them as image files using `fastcore.parallel` which takes about four and a half minutes.","metadata":{}},{"cell_type":"code","source":"from fastcore.parallel import *","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:28:07.218279Z","iopub.execute_input":"2024-03-05T16:28:07.219079Z","iopub.status.idle":"2024-03-05T16:28:07.223175Z","shell.execute_reply.started":"2024-03-05T16:28:07.219048Z","shell.execute_reply":"2024-03-05T16:28:07.222176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"warnings.filterwarnings(\"ignore\")\nparallel(process_spec, spec_ids, split='train', n_workers=4)\nwarnings.filterwarnings(\"default\")","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:28:39.987421Z","iopub.execute_input":"2024-03-05T16:28:39.987942Z","iopub.status.idle":"2024-03-05T16:34:05.892943Z","shell.execute_reply.started":"2024-03-05T16:28:39.987906Z","shell.execute_reply":"2024-03-05T16:34:05.891610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll load a training spectrogram image to make sure:","metadata":{}},{"cell_type":"code","source":"Path('/tmp/dataset/hms-hbac/train_spectrograms').ls()","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:21.016315Z","iopub.execute_input":"2024-03-05T16:34:21.017299Z","iopub.status.idle":"2024-03-05T16:34:21.050698Z","shell.execute_reply.started":"2024-03-05T16:34:21.017258Z","shell.execute_reply":"2024-03-05T16:34:21.049790Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PILImage.create('/tmp/dataset/hms-hbac/train_spectrograms/1829953376.png')","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:22.142253Z","iopub.execute_input":"2024-03-05T16:34:22.143147Z","iopub.status.idle":"2024-03-05T16:34:22.163458Z","shell.execute_reply.started":"2024-03-05T16:34:22.143112Z","shell.execute_reply":"2024-03-05T16:34:22.162407Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looks good! I'll do the test spectrograms next (there's only one):","metadata":{}},{"cell_type":"code","source":"test_df = pd.read_csv(path/'test.csv')\ntest_df","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:25.070156Z","iopub.execute_input":"2024-03-05T16:34:25.071056Z","iopub.status.idle":"2024-03-05T16:34:25.086872Z","shell.execute_reply.started":"2024-03-05T16:34:25.071022Z","shell.execute_reply":"2024-03-05T16:34:25.085787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spec_ids = test_df['spectrogram_id'].unique()\nspec_ids","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:27.032175Z","iopub.execute_input":"2024-03-05T16:34:27.032866Z","iopub.status.idle":"2024-03-05T16:34:27.039655Z","shell.execute_reply.started":"2024-03-05T16:34:27.032834Z","shell.execute_reply":"2024-03-05T16:34:27.038660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"parallel(process_spec, spec_ids, split='test', n_workers=4)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:28.915226Z","iopub.execute_input":"2024-03-05T16:34:28.915980Z","iopub.status.idle":"2024-03-05T16:34:29.164400Z","shell.execute_reply.started":"2024-03-05T16:34:28.915948Z","shell.execute_reply":"2024-03-05T16:34:29.163299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PILImage.create('/tmp/dataset/hms-hbac/test_spectrograms/853520.png')","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:31.658551Z","iopub.execute_input":"2024-03-05T16:34:31.659568Z","iopub.status.idle":"2024-03-05T16:34:31.686556Z","shell.execute_reply.started":"2024-03-05T16:34:31.659525Z","shell.execute_reply":"2024-03-05T16:34:31.685521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Awesome! I now have the images I need for training.","metadata":{}},{"cell_type":"markdown","source":"## Finetuning a Pretrained ResNet34","metadata":{}},{"cell_type":"markdown","source":"As done in the reference notebook, I'll add the newly generated image path to the trainig `DataFrame`.","metadata":{}},{"cell_type":"code","source":"df['img_path'] = '/tmp/dataset/hms-hbac/train_spectrograms/' + df['spectrogram_id'].astype(str) + '.png'","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:39.398590Z","iopub.execute_input":"2024-03-05T16:34:39.399451Z","iopub.status.idle":"2024-03-05T16:34:39.491611Z","shell.execute_reply.started":"2024-03-05T16:34:39.399407Z","shell.execute_reply":"2024-03-05T16:34:39.490600Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:39.924176Z","iopub.execute_input":"2024-03-05T16:34:39.925123Z","iopub.status.idle":"2024-03-05T16:34:39.947919Z","shell.execute_reply.started":"2024-03-05T16:34:39.925081Z","shell.execute_reply":"2024-03-05T16:34:39.946972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, I'll create a `DataFrame` with unique `eeg_id` and `spectrogram_id` and average votes for eac","metadata":{}},{"cell_type":"code","source":"cols = ['eeg_id', 'spectrogram_id', 'img_path', 'seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']\nagg_funcs = {c: 'sum' for c in cols if 'vote' in c}\n\nunique_df = df[cols].groupby(['eeg_id', 'spectrogram_id', 'img_path'], as_index=False).agg(agg_funcs)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:42.169892Z","iopub.execute_input":"2024-03-05T16:34:42.170274Z","iopub.status.idle":"2024-03-05T16:34:42.258053Z","shell.execute_reply.started":"2024-03-05T16:34:42.170248Z","shell.execute_reply":"2024-03-05T16:34:42.257066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll set the `target` as the column with the largest number of votes for a given spectrogram.","metadata":{}},{"cell_type":"code","source":"unique_df['target'] = unique_df[[c for c in cols if 'vote' in c]].idxmax(axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:44.597847Z","iopub.execute_input":"2024-03-05T16:34:44.598179Z","iopub.status.idle":"2024-03-05T16:34:44.610229Z","shell.execute_reply.started":"2024-03-05T16:34:44.598155Z","shell.execute_reply":"2024-03-05T16:34:44.609291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And finally, an `is_valid` column which indicates whether a row is set aside for the training or validation set:","metadata":{}},{"cell_type":"code","source":"train_bool = [False for _ in range(int(0.8 * len(unique_df)))]\nvalid_bool = [True for _ in range(len(unique_df) - int(0.8 * len(unique_df)))]\nis_valid_bool = pd.Series(train_bool + valid_bool).sample(frac=1).reset_index(drop=True)\nlen(is_valid_bool)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:45.905716Z","iopub.execute_input":"2024-03-05T16:34:45.906685Z","iopub.status.idle":"2024-03-05T16:34:45.921523Z","shell.execute_reply.started":"2024-03-05T16:34:45.906652Z","shell.execute_reply":"2024-03-05T16:34:45.920318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_df[\"is_valid\"] = is_valid_bool","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:47.614036Z","iopub.execute_input":"2024-03-05T16:34:47.614868Z","iopub.status.idle":"2024-03-05T16:34:47.620385Z","shell.execute_reply.started":"2024-03-05T16:34:47.614827Z","shell.execute_reply":"2024-03-05T16:34:47.619238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:48.382496Z","iopub.execute_input":"2024-03-05T16:34:48.383346Z","iopub.status.idle":"2024-03-05T16:34:48.399822Z","shell.execute_reply.started":"2024-03-05T16:34:48.383306Z","shell.execute_reply":"2024-03-05T16:34:48.398571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now I can create my `DataBlock`. Since I now have paths to images ready to train on, I can use `ImageBlock` for my inputs. I'll also make sure to `Resize` my images so they are 224 x 224 squares.","metadata":{}},{"cell_type":"code","source":"dblock = DataBlock(\n            blocks=(ImageBlock, CategoryBlock),\n            splitter=ColSplitter(),\n            get_x=ColReader('img_path'),\n            get_y=ColReader('target'),\n            item_tfms=Resize(224, method='squish'))\n\ndls = dblock.dataloaders(unique_df)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:51.429501Z","iopub.execute_input":"2024-03-05T16:34:51.430424Z","iopub.status.idle":"2024-03-05T16:34:52.954401Z","shell.execute_reply.started":"2024-03-05T16:34:51.430383Z","shell.execute_reply":"2024-03-05T16:34:52.953421Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls.show_batch(nrows=1, ncols=3)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:55.906252Z","iopub.execute_input":"2024-03-05T16:34:55.906630Z","iopub.status.idle":"2024-03-05T16:34:56.946478Z","shell.execute_reply.started":"2024-03-05T16:34:55.906603Z","shell.execute_reply":"2024-03-05T16:34:56.945213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check dls vocab\ndls.vocab","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:34:56.948403Z","iopub.execute_input":"2024-03-05T16:34:56.948781Z","iopub.status.idle":"2024-03-05T16:34:56.956448Z","shell.execute_reply.started":"2024-03-05T16:34:56.948742Z","shell.execute_reply":"2024-03-05T16:34:56.954889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"fastai uses `timm` when you specify the architecture as a string:","metadata":{}},{"cell_type":"code","source":"learn = vision_learner(dls, 'resnet34', metrics=accuracy).to_fp16()","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:35:00.694584Z","iopub.execute_input":"2024-03-05T16:35:00.695433Z","iopub.status.idle":"2024-03-05T16:35:02.208590Z","shell.execute_reply.started":"2024-03-05T16:35:00.695391Z","shell.execute_reply":"2024-03-05T16:35:02.207608Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"type(learn.model[0])","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:35:04.774578Z","iopub.execute_input":"2024-03-05T16:35:04.774949Z","iopub.status.idle":"2024-03-05T16:35:04.781338Z","shell.execute_reply.started":"2024-03-05T16:35:04.774921Z","shell.execute_reply":"2024-03-05T16:35:04.780426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll use the same number of epochs and learning rate as Jeremy did in the Paddy Doctor competition Live Coding videos:","metadata":{}},{"cell_type":"code","source":"learn.fine_tune(12, 0.01)","metadata":{"execution":{"iopub.status.busy":"2024-03-05T16:35:08.022157Z","iopub.execute_input":"2024-03-05T16:35:08.022749Z","iopub.status.idle":"2024-03-05T16:45:39.104776Z","shell.execute_reply.started":"2024-03-05T16:35:08.022708Z","shell.execute_reply":"2024-03-05T16:45:39.103626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This model is overfitting (the validation loss starts to increase each epoch after the 7th epoch) so in subsequent trainings I'll limit the number of epochs to 6 or 7.\n\nNext, I'll export the model so that I can upload it to Kaggle and use it for my submission notebook with internet access disabled.","metadata":{}},{"cell_type":"code","source":"# learn.save('/kaggle/working/hms_hbac_resnet34', with_opt=False)","metadata":{"execution":{"iopub.status.busy":"2024-03-04T22:18:10.122202Z","iopub.execute_input":"2024-03-04T22:18:10.122996Z","iopub.status.idle":"2024-03-04T22:18:10.274302Z","shell.execute_reply.started":"2024-03-04T22:18:10.122955Z","shell.execute_reply":"2024-03-04T22:18:10.273317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That's it for this notebook, next, I'll make sure I can successfully execute a submission from my [submission notebook](https://www.kaggle.com/code/vishalbakshi/hms-hbac-fastai-resnet34-starter-submit).\n","metadata":{}}]}