{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Oversampling for Multi-Label Classification with fastai\n\nThis is a simplified approach to oversampling for multilabel classification, inspired by @iafoss and his [notebook on the previous HPA challenge](https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb).\n\nTo summarize, I will:\n- count the number of examples for each class\n- calculate the oversampling ratio to match the average number of examples per class\n- set manually the oversampling ratio for each class (in case I prefer that vs the calculated ratio)\n- copy each example in my training dataframe the same number of times as the oversampling ratio I determined above\n\nFrom then on, I will follow the regular model training approach. ","attachments":{}},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!pip install iterative_stratification -q\n\nfrom fastai.vision.all import *\nimport numpy as np\nimport pandas as pd\nimport torch\nfrom iterstrat.ml_stratifiers import MultilabelStratifiedKFold\nimport warnings\nwarnings.filterwarnings('ignore')\ntorch.set_printoptions(precision=3, sci_mode=False)\n\nsample_size = 1\nseed = 42\nstats = ([0.07237246, 0.04476176, 0.07661699], [0.17179589, 0.10284516, 0.14199627])\nitem_tfms = RandomResizedCrop(448, min_scale=0.75, ratio=(1.,1.))\nbatch_tfms = [*aug_transforms(flip_vert=True, max_warp=0), Normalize.from_stats(*stats)]\nbs = 32\nlr = 3e-2\nepochs = 2\ncbs = None\n\ndf = pd.read_csv('../input/hpa-single-cell-image-classification/train.csv')\npath = Path('../input/hpa-512x512-jpg-images-dataset/512x512jpgs')\n\nlabels = [str(i) for i in range(19)]\nfor x in labels: df[x] = df['Label'].apply(lambda r: int(x in r.split('|')))\n\ndfs = df.sample(frac=sample_size, random_state=seed).reset_index(drop=True)\ny = dfs[labels].values\nX = dfs['ID'].values\ndfs['fold'] = np.nan\n\nmskf = MultilabelStratifiedKFold(n_splits=5)\nfor i, (_, test_index) in enumerate(mskf.split(X, y)):\n    dfs.iloc[test_index, -1] = i\n   \ndfs['fold'] = dfs['fold'].astype('int')\ndfs['is_valid'] = False\ndfs['is_valid'][dfs.fold == 0] = True\n\ndef get_x(r): return path/f'{r[\"ID\"]}.jpg'\ndef get_y(r): return list(set(r['Label'].split('|')))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the snippet below, I calculate the number of examples for each class in my dataset, and the average. "},{"metadata":{"trusted":true},"cell_type":"code","source":"full_counts = {}\nfsum = 0\nfor lbl in labels:\n    count = 0\n    for row_label in dfs['Label']:\n        if lbl in row_label.split('|'): count += 1\n    full_counts[lbl] = count\n    fsum += count\nfull_counts['avg'] = int(fsum/(len(labels)))\n\ncounts = list(zip(full_counts.keys(), full_counts.values()))\ncounts = np.array(sorted(counts, key=lambda x:-x[1]))\ncounts = pd.DataFrame(counts, columns=['label', 'full_count'])\ncounts.set_index('label', inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here I define the function to set an integer oversampling ratio so the number of examples gets close to the average. I apply that function in my `counts` dataframe, and I also show how to set this manually. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def set_sample_ratio(x):\n    avg = int(counts['full_count'].loc['avg'])\n    x = int(x)\n    if x >= avg: return 1\n    else: return int(np.round(avg / x))\n\ncounts['calculated_oversampling_ratio'] = counts['full_count'].apply(set_sample_ratio)\ncounts['manual_oversapling_ratio'] = [ 1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  2,  3, 4, 8, 16]\ncounts.T","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here I define the function that given an example, will check the classes it belongs to, and then return the highest ratio across these classes. Then, we split our dataset into train and valid, iterate through the train rows and copy each row as many times, as determined by this function. Finally, we merge train and valid again so that we can pass it into fastai `DataBlock`. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_sample_ratio(row):\n    ratio = 1\n    labels = row[1].split('|')\n    for l in labels:\n        r = counts.manual_oversapling_ratio.loc[l]\n        if r > ratio: ratio = r\n    return ratio\n\ndf_valid = dfs[dfs['is_valid'] == True]\ndf_train = dfs[dfs['is_valid'] == False]\n\nrows = df_train.values.tolist()\nprint(len(rows))\noversampled_rows = [row for row in rows for _ in range(get_sample_ratio(row))]\nprint(len(oversampled_rows))\n\ndf_train_oversampled = pd.DataFrame(oversampled_rows, columns=df_train.columns)\n\ndfs = pd.concat([df_valid, df_train_oversampled], ignore_index=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From here on, we can train as usual. "},{"metadata":{"trusted":true},"cell_type":"code","source":"dblock = DataBlock(blocks=(ImageBlock, MultiCategoryBlock(vocab=labels)),\n                    splitter=ColSplitter(col='is_valid'),\n                    get_x=get_x,\n                    get_y=get_y,\n                    item_tfms=item_tfms,\n                    batch_tfms=batch_tfms\n                    )\ndls = dblock.dataloaders(dfs, bs=bs)\n\nlearn = cnn_learner(dls, resnet18, metrics=[accuracy_multi, APScoreMulti()]).to_fp16()\nlearn.fine_tune(epochs, base_lr=lr, cbs=cbs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# End."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}