{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Update\n\n**Bug found**: I intended to use the whole clip of `site_3`, but I found that current implementation only uses the last 5 seconds. This version (v3) fixes that.","metadata":{}},{"cell_type":"markdown","source":"## About\n\nI've spent several days to make a successful submission and finally got a way to do that after 3 days of struggle. \nI want Kaggle competitors to feel easy to participate in this competition, therefore I decided to share my Notebook.\n\nI would like to thank [@radek1](https://www.kaggle.com/radek1) for creating [a good starter notebook](https://www.kaggle.com/c/birdsong-recognition/discussion/160222), [@shonenkov](https://www.kaggle.com/shonenkov) for [a nice checking dataset and notebook](https://www.kaggle.com/shonenkov/sample-submission-using-custom-check) and several discussions to make this competition better, [@cwthompson](https://www.kaggle.com/cwthompson) for [showing the way to submit](https://www.kaggle.com/cwthompson/birdsong-making-a-prediction) using `test_audio`.\n\nI also would like to thank [@stefankahl](https://www.kaggle.com/stefankahl), [@tomdenton](https://www.kaggle.com/tomdenton), [@sohier](https://www.kaggle.com/sohier) for hosting a really interesting competition.","metadata":{}},{"cell_type":"markdown","source":"In this notebook I tried to make submission using ResNet based model trained with log melspectrogram. I will create a notebook to show the way I trained the model but here I briefly describe my approach.\n\n* Randomly crop 5 seconds for each train audio clip each epoch.\n* No augmentation.\n* Use pretrained weight of `torchvision.models.resnet50`.\n* Used `BCELoss`.\n* Trained 100 epoch and used the weight which got best F1 (at 92epoch).\n* `Adam` optimizer (`lr=0.001`) with `CosineAnnealingLR` (`T_max=10`).\n* Use `StratifiedKFold(n_splits=5)` to split dataset and used only first fold\n\nHere are the parameter details.\n\n* `batch_size`: 100 (on V100, took 2 ~ 3hrs to run 100epochs)\n* melspectrogram parameters\n  - `n_mels`: 128\n  - `fmin`: 20\n  - `fmax`: 16000\n* image size: 224 x 541 (I don't remember the exact width)","metadata":{}},{"cell_type":"markdown","source":"### Future direction\n\nThere are a lot many to do to make improvement. It was a big challenge for me to make successful submission with very few feedback signal (like `Submission CSV Not Found` or `Notebook Exceeded Allowed Compute`), but this is just a beginning of the real challenge.\nAs described in https://www.kaggle.com/c/birdsong-recognition/discussion/160222#895234 , data augmentation is a key. I worked on [Freesound Audio Tagging 2019](https://www.kaggle.com/c/freesound-audio-tagging-2019) last year, which was also an audio competition (which is comparatively rare in Kaggle), and at that time data augmentation like pitch shift or reverb effect gave us a boost. This competition is not about environmental sound but about bird song, therefore we need to check what augmentation works best on this data by experiment. Maybe we can get a boost with different augmentation for different audio class.\n\nMixup / BClearning or mixing different audio class may give us a rise I believe, since the test set has multiple sounds in the clip whereas train set has basically one class for one clip (of course we can use background sound information to treat train set as multilabel problem).\n\nTraining procedure also has an important role, whether we use a procedure for multilabel problem (by using background sound) or for multiclass problem. The challenge of this competition can also be treated as *Domain Adaptation* problem, so we can use techniques for that.\n\nModel selection is also important, deeper model may give us a rise, but from my experience, *too deep* model are sometimes defeated by shallower model in audio classification.","metadata":{}},{"cell_type":"markdown","source":"## Libraries","metadata":{}},{"cell_type":"code","source":"import cv2\nimport audioread\nimport logging\nimport os\nimport random\nimport time\nimport warnings\n\nimport librosa\nimport numpy as np\nimport pandas as pd\nimport soundfile as sf\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.utils.data as data\n\nfrom contextlib import contextmanager\nfrom pathlib import Path\nfrom typing import Optional\n\n\nfrom sklearn.metrics import f1_score\nfrom torchvision import models\n\nfrom tqdm import tqdm\nimport warnings","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-11-03T14:40:58.907281Z","iopub.execute_input":"2023-11-03T14:40:58.907620Z","iopub.status.idle":"2023-11-03T14:41:02.567598Z","shell.execute_reply.started":"2023-11-03T14:40:58.907591Z","shell.execute_reply":"2023-11-03T14:41:02.566736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Utilities","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0"}},{"cell_type":"code","source":"def set_seed(seed: int = 42):\n    random.seed(seed)\n    np.random.seed(seed)\n    os.environ[\"PYTHONHASHSEED\"] = str(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)  # type: ignore\n    torch.backends.cudnn.deterministic = True  # type: ignore\n    torch.backends.cudnn.benchmark = True  # type: ignore\n    \n    \ndef get_logger(out_file=None):\n    logger = logging.getLogger()\n    formatter = logging.Formatter(\"%(asctime)s - %(levelname)s - %(message)s\")\n    logger.handlers = []\n    logger.setLevel(logging.INFO)\n\n    handler = logging.StreamHandler()\n    handler.setFormatter(formatter)\n    handler.setLevel(logging.INFO)\n    logger.addHandler(handler)\n\n    if out_file is not None:\n        fh = logging.FileHandler(out_file)\n        fh.setFormatter(formatter)\n        fh.setLevel(logging.INFO)\n        logger.addHandler(fh)\n    logger.info(\"logger set up\")\n    return logger\n    \n    \n@contextmanager\ndef timer(name: str, logger: Optional[logging.Logger] = None):\n    t0 = time.time()\n    msg = f\"[{name}] start\"\n    if logger is None:\n        print(msg)\n    else:\n        logger.info(msg)\n    yield\n\n    msg = f\"[{name}] done in {time.time() - t0:.2f} s\"\n    if logger is None:\n        print(msg)\n    else:\n        logger.info(msg)","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:02.569566Z","iopub.execute_input":"2023-11-03T14:41:02.569916Z","iopub.status.idle":"2023-11-03T14:41:02.584463Z","shell.execute_reply.started":"2023-11-03T14:41:02.569884Z","shell.execute_reply":"2023-11-03T14:41:02.583538Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"logger = get_logger(\"main.log\")\nset_seed(1213)","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:02.586198Z","iopub.execute_input":"2023-11-03T14:41:02.586494Z","iopub.status.idle":"2023-11-03T14:41:02.606070Z","shell.execute_reply.started":"2023-11-03T14:41:02.586465Z","shell.execute_reply":"2023-11-03T14:41:02.605188Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Loading","metadata":{}},{"cell_type":"code","source":"TARGET_SR = 32000\nTEST = Path(\"../kaggle/input/data-test\").exists()","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:02.607576Z","iopub.execute_input":"2023-11-03T14:41:02.607914Z","iopub.status.idle":"2023-11-03T14:41:02.613951Z","shell.execute_reply.started":"2023-11-03T14:41:02.607883Z","shell.execute_reply":"2023-11-03T14:41:02.613158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sound_array = [['site_4', 'site_4_Adfly_5', 5, 'Aldfly']\n              ,['site_4', 'site_4_Adfly_10', 10, 'Aldfly']\n              ,['site_4', 'site_4_Adfly_15', 15, 'Aldfly']\n              ,['site_5', 'site_5_perfal_5', 5, 'perfal']\n              ,['site_5', 'site_5_perfal_10', 10, 'perfal']\n              ,['site_5', 'site_5_perfal_15', 15, 'perfal']]\nex_test = pd.DataFrame(sound_array, columns =['site', 'row_id', 'seconds', 'audio_id']) \nex_test","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:02.617856Z","iopub.execute_input":"2023-11-03T14:41:02.618176Z","iopub.status.idle":"2023-11-03T14:41:02.641078Z","shell.execute_reply.started":"2023-11-03T14:41:02.618145Z","shell.execute_reply":"2023-11-03T14:41:02.640226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_path = '/kaggle/input/birdsong-recognition/example_test_audio_summary.csv'\n# ex_test = pd.read_csv(test_path)\n\n# ex_test[['site', 'num', 'audio_id']] = ex_test['filename_seconds'].str.split('-', n=2, expand=True)\n# ex_test.site.replace(['BLKFR', 'ORANGE'], ['site_1', 'site_2'], inplace=True)\n# ex_test['row_id'] = ex_test[['site', 'audio_id']].apply(lambda row: '_'.join(row.values.astype(str)), axis=1)\n# ex_test[['name', 'date', 'id', 'sec']] = ex_test['audio_id'].str.split('_', n=3, expand=True)\n# ex_test.name.replace(['CPL', 'CAP'], ['BLKFR-10-CPL_20190611_093000.pt540', 'ORANGE-7-CAP_20190606_093000.pt623'], inplace=True)\n\n# column = ['site', 'row_id', 'seconds', 'name']\n# ex_test = ex_test.reindex(column, axis ='columns')\n# ex_test = ex_test.rename(columns={'name':'audio_id'})\n\n# ex_test","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:02.644091Z","iopub.execute_input":"2023-11-03T14:41:02.644432Z","iopub.status.idle":"2023-11-03T14:41:02.648664Z","shell.execute_reply.started":"2023-11-03T14:41:02.644402Z","shell.execute_reply":"2023-11-03T14:41:02.647645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ex_audio_path = ['/kaggle/input/birdsong-recognition/example_test_audio/BLKFR-10-CPL_20190611_093000.pt540.mp3',\n                 '/kaggle/input/birdsong-recognition/example_test_audio/ORANGE-7-CAP_20190606_093000.pt623.mp3']\n\naudio, rate = librosa.load(ex_audio_path[0], sr=None, offset=465, duration=5)\n# audio, rate = librosa.load(ex_audio_path[0], sr=None)\nprint(librosa.get_duration(audio))\n# for path in ex_audio_path:\n#     print(path)","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:02.650126Z","iopub.execute_input":"2023-11-03T14:41:02.650557Z","iopub.status.idle":"2023-11-03T14:41:03.342095Z","shell.execute_reply.started":"2023-11-03T14:41:02.650489Z","shell.execute_reply":"2023-11-03T14:41:03.341023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import shutil\nif os.path.exists('/kaggle/working/test_audio'):\n    shutil.rmtree('/kaggle/working/test_audio')\nos.mkdir('/kaggle/working/test_audio')","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.343650Z","iopub.execute_input":"2023-11-03T14:41:03.343983Z","iopub.status.idle":"2023-11-03T14:41:03.348674Z","shell.execute_reply.started":"2023-11-03T14:41:03.343945Z","shell.execute_reply":"2023-11-03T14:41:03.347910Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y, sr = librosa.load('/kaggle/input/birdcall-check/test_audio/07ab324c602e4afab65ddbcc746c31b5.mp3', sr=None)\nlibrosa.get_duration(y)","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.349954Z","iopub.execute_input":"2023-11-03T14:41:03.350242Z","iopub.status.idle":"2023-11-03T14:41:03.554177Z","shell.execute_reply.started":"2023-11-03T14:41:03.350215Z","shell.execute_reply":"2023-11-03T14:41:03.553119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# warnings.filterwarnings(\"ignore\", category=UserWarning, module=\"librosa.core.audio\")\n# for i in range(len(ex_audio_path)):\n#     progress_bar = tqdm(range(len(ex_test[ex_test['site']==f'site_{i+1}']['seconds'])))\n#     df = ex_test[ex_test['site']==f'site_{i+1}']['seconds'].reset_index(drop=True)\n    \n#     for j in progress_bar:\n#         offset = df[j] - 5\n#         name = ex_test['audio_id'][j]\n        \n#         audio, rate = librosa.load(ex_audio_path[i], sr=None, offset=offset, duration=5)\n#         librosa.output.write_wav(f'/kaggle/working/test_audio/{name}.wav', audio, rate)\n        \n#         progress_bar.set_postfix({'offset': '{:.3f}'.format(offset)})","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.556190Z","iopub.execute_input":"2023-11-03T14:41:03.556664Z","iopub.status.idle":"2023-11-03T14:41:03.561399Z","shell.execute_reply.started":"2023-11-03T14:41:03.556616Z","shell.execute_reply":"2023-11-03T14:41:03.560388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# del progress_bar","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.563246Z","iopub.execute_input":"2023-11-03T14:41:03.563661Z","iopub.status.idle":"2023-11-03T14:41:03.576311Z","shell.execute_reply.started":"2023-11-03T14:41:03.563618Z","shell.execute_reply":"2023-11-03T14:41:03.575197Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from fastprogress import progress_bar","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.577644Z","iopub.execute_input":"2023-11-03T14:41:03.578009Z","iopub.status.idle":"2023-11-03T14:41:03.591402Z","shell.execute_reply.started":"2023-11-03T14:41:03.577976Z","shell.execute_reply":"2023-11-03T14:41:03.590540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"zeroCheck = 0\nzeroAudio = []\nfor i in os.listdir('/kaggle/working/test_audio'):\n    audio, rate = librosa.load(f'/kaggle/working/test_audio/{i}', sr=None)\n    if len(audio) == 0:\n        zeroCheck+=1\n        zeroAudio.append(i)","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.592955Z","iopub.execute_input":"2023-11-03T14:41:03.593354Z","iopub.status.idle":"2023-11-03T14:41:03.600171Z","shell.execute_reply.started":"2023-11-03T14:41:03.593314Z","shell.execute_reply":"2023-11-03T14:41:03.599348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"zeroAudio.sort()\nzeroAudio","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.602520Z","iopub.execute_input":"2023-11-03T14:41:03.602983Z","iopub.status.idle":"2023-11-03T14:41:03.612151Z","shell.execute_reply.started":"2023-11-03T14:41:03.602936Z","shell.execute_reply":"2023-11-03T14:41:03.611147Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# if TEST:\n#     DATA_DIR = Path(\"../input/birdsong-recognition/\")\n# else:\n#     # dataset created by @shonenkov, thanks!\n#     DATA_DIR = Path(\"../input/birdcall-check/\")\n# DATA_DIR = Path(\"../input/birdsong-recognition/\")    \nDATA_DIR = Path(\"../input/\") \n\n# test = pd.read_csv(DATA_DIR / \"test.csv\")\n# test_audio = DATA_DIR / \"test_audio\"\ntest = ex_test\ntest_audio = DATA_DIR / \"data-test\"\n\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.613369Z","iopub.execute_input":"2023-11-03T14:41:03.613763Z","iopub.status.idle":"2023-11-03T14:41:03.631383Z","shell.execute_reply.started":"2023-11-03T14:41:03.613722Z","shell.execute_reply":"2023-11-03T14:41:03.630496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv(\"../input/birdsong-recognition/sample_submission.csv\")\nsub.to_csv(\"submission.csv\", index=False)  # this will be overwritten if everything goes well","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:03.632801Z","iopub.execute_input":"2023-11-03T14:41:03.633197Z","iopub.status.idle":"2023-11-03T14:41:04.021239Z","shell.execute_reply.started":"2023-11-03T14:41:03.633156Z","shell.execute_reply":"2023-11-03T14:41:04.020515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define Model","metadata":{}},{"cell_type":"markdown","source":"### Model","metadata":{}},{"cell_type":"code","source":"class ResNet(nn.Module):\n    def __init__(self, base_model_name: str, pretrained=False,\n                 num_classes=264):\n        super().__init__()\n        base_model = models.__getattribute__(base_model_name)(\n            pretrained=pretrained)\n        layers = list(base_model.children())[:-2]\n        layers.append(nn.AdaptiveMaxPool2d(1))\n        self.encoder = nn.Sequential(*layers)\n\n        in_features = base_model.fc.in_features\n\n        self.classifier = nn.Sequential(\n            nn.Linear(in_features, 1024), nn.ReLU(), nn.Dropout(p=0.2),\n            nn.Linear(1024, 1024), nn.ReLU(), nn.Dropout(p=0.2),\n            nn.Linear(1024, num_classes))\n\n    def forward(self, x):\n        batch_size = x.size(0)\n        x = self.encoder(x).view(batch_size, -1)\n        x = self.classifier(x)\n        multiclass_proba = F.softmax(x, dim=1)\n        multilabel_proba = F.sigmoid(x)\n        return {\n            \"logits\": x,\n            \"multiclass_proba\": multiclass_proba,\n            \"multilabel_proba\": multilabel_proba\n        }","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.022782Z","iopub.execute_input":"2023-11-03T14:41:04.023167Z","iopub.status.idle":"2023-11-03T14:41:04.036261Z","shell.execute_reply.started":"2023-11-03T14:41:04.023126Z","shell.execute_reply":"2023-11-03T14:41:04.035187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Parameters","metadata":{}},{"cell_type":"code","source":"model_config = {\n    \"base_model_name\": \"resnet50\",\n    \"pretrained\": False,\n    \"num_classes\": 264\n}\n\nmelspectrogram_parameters = {\n    \"n_mels\": 128,\n    \"fmin\": 20,\n    \"fmax\": 16000\n}\n\nweights_path = \"../input/birdcall-resnet50-init-weights/best.pth\"","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.037692Z","iopub.execute_input":"2023-11-03T14:41:04.038097Z","iopub.status.idle":"2023-11-03T14:41:04.049456Z","shell.execute_reply.started":"2023-11-03T14:41:04.038057Z","shell.execute_reply":"2023-11-03T14:41:04.048747Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BIRD_CODE = {\n    'aldfly': 0, 'ameavo': 1, 'amebit': 2, 'amecro': 3, 'amegfi': 4,\n    'amekes': 5, 'amepip': 6, 'amered': 7, 'amerob': 8, 'amewig': 9,\n    'amewoo': 10, 'amtspa': 11, 'annhum': 12, 'astfly': 13, 'baisan': 14,\n    'baleag': 15, 'balori': 16, 'banswa': 17, 'barswa': 18, 'bawwar': 19,\n    'belkin1': 20, 'belspa2': 21, 'bewwre': 22, 'bkbcuc': 23, 'bkbmag1': 24,\n    'bkbwar': 25, 'bkcchi': 26, 'bkchum': 27, 'bkhgro': 28, 'bkpwar': 29,\n    'bktspa': 30, 'blkpho': 31, 'blugrb1': 32, 'blujay': 33, 'bnhcow': 34,\n    'boboli': 35, 'bongul': 36, 'brdowl': 37, 'brebla': 38, 'brespa': 39,\n    'brncre': 40, 'brnthr': 41, 'brthum': 42, 'brwhaw': 43, 'btbwar': 44,\n    'btnwar': 45, 'btywar': 46, 'buffle': 47, 'buggna': 48, 'buhvir': 49,\n    'bulori': 50, 'bushti': 51, 'buwtea': 52, 'buwwar': 53, 'cacwre': 54,\n    'calgul': 55, 'calqua': 56, 'camwar': 57, 'cangoo': 58, 'canwar': 59,\n    'canwre': 60, 'carwre': 61, 'casfin': 62, 'caster1': 63, 'casvir': 64,\n    'cedwax': 65, 'chispa': 66, 'chiswi': 67, 'chswar': 68, 'chukar': 69,\n    'clanut': 70, 'cliswa': 71, 'comgol': 72, 'comgra': 73, 'comloo': 74,\n    'commer': 75, 'comnig': 76, 'comrav': 77, 'comred': 78, 'comter': 79,\n    'comyel': 80, 'coohaw': 81, 'coshum': 82, 'cowscj1': 83, 'daejun': 84,\n    'doccor': 85, 'dowwoo': 86, 'dusfly': 87, 'eargre': 88, 'easblu': 89,\n    'easkin': 90, 'easmea': 91, 'easpho': 92, 'eastow': 93, 'eawpew': 94,\n    'eucdov': 95, 'eursta': 96, 'evegro': 97, 'fiespa': 98, 'fiscro': 99,\n    'foxspa': 100, 'gadwal': 101, 'gcrfin': 102, 'gnttow': 103, 'gnwtea': 104,\n    'gockin': 105, 'gocspa': 106, 'goleag': 107, 'grbher3': 108, 'grcfly': 109,\n    'greegr': 110, 'greroa': 111, 'greyel': 112, 'grhowl': 113, 'grnher': 114,\n    'grtgra': 115, 'grycat': 116, 'gryfly': 117, 'haiwoo': 118, 'hamfly': 119,\n    'hergul': 120, 'herthr': 121, 'hoomer': 122, 'hoowar': 123, 'horgre': 124,\n    'horlar': 125, 'houfin': 126, 'houspa': 127, 'houwre': 128, 'indbun': 129,\n    'juntit1': 130, 'killde': 131, 'labwoo': 132, 'larspa': 133, 'lazbun': 134,\n    'leabit': 135, 'leafly': 136, 'leasan': 137, 'lecthr': 138, 'lesgol': 139,\n    'lesnig': 140, 'lesyel': 141, 'lewwoo': 142, 'linspa': 143, 'lobcur': 144,\n    'lobdow': 145, 'logshr': 146, 'lotduc': 147, 'louwat': 148, 'macwar': 149,\n    'magwar': 150, 'mallar3': 151, 'marwre': 152, 'merlin': 153, 'moublu': 154,\n    'mouchi': 155, 'moudov': 156, 'norcar': 157, 'norfli': 158, 'norhar2': 159,\n    'normoc': 160, 'norpar': 161, 'norpin': 162, 'norsho': 163, 'norwat': 164,\n    'nrwswa': 165, 'nutwoo': 166, 'olsfly': 167, 'orcwar': 168, 'osprey': 169,\n    'ovenbi1': 170, 'palwar': 171, 'pasfly': 172, 'pecsan': 173, 'perfal': 174,\n    'phaino': 175, 'pibgre': 176, 'pilwoo': 177, 'pingro': 178, 'pinjay': 179,\n    'pinsis': 180, 'pinwar': 181, 'plsvir': 182, 'prawar': 183, 'purfin': 184,\n    'pygnut': 185, 'rebmer': 186, 'rebnut': 187, 'rebsap': 188, 'rebwoo': 189,\n    'redcro': 190, 'redhea': 191, 'reevir1': 192, 'renpha': 193, 'reshaw': 194,\n    'rethaw': 195, 'rewbla': 196, 'ribgul': 197, 'rinduc': 198, 'robgro': 199,\n    'rocpig': 200, 'rocwre': 201, 'rthhum': 202, 'ruckin': 203, 'rudduc': 204,\n    'rufgro': 205, 'rufhum': 206, 'rusbla': 207, 'sagspa1': 208, 'sagthr': 209,\n    'savspa': 210, 'saypho': 211, 'scatan': 212, 'scoori': 213, 'semplo': 214,\n    'semsan': 215, 'sheowl': 216, 'shshaw': 217, 'snobun': 218, 'snogoo': 219,\n    'solsan': 220, 'sonspa': 221, 'sora': 222, 'sposan': 223, 'spotow': 224,\n    'stejay': 225, 'swahaw': 226, 'swaspa': 227, 'swathr': 228, 'treswa': 229,\n    'truswa': 230, 'tuftit': 231, 'tunswa': 232, 'veery': 233, 'vesspa': 234,\n    'vigswa': 235, 'warvir': 236, 'wesblu': 237, 'wesgre': 238, 'weskin': 239,\n    'wesmea': 240, 'wessan': 241, 'westan': 242, 'wewpew': 243, 'whbnut': 244,\n    'whcspa': 245, 'whfibi': 246, 'whtspa': 247, 'whtswi': 248, 'wilfly': 249,\n    'wilsni1': 250, 'wiltur': 251, 'winwre3': 252, 'wlswar': 253, 'wooduc': 254,\n    'wooscj2': 255, 'woothr': 256, 'y00475': 257, 'yebfly': 258, 'yebsap': 259,\n    'yehbla': 260, 'yelwar': 261, 'yerwar': 262, 'yetvir': 263\n}\n\nINV_BIRD_CODE = {v: k for k, v in BIRD_CODE.items()}","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.050861Z","iopub.execute_input":"2023-11-03T14:41:04.051195Z","iopub.status.idle":"2023-11-03T14:41:04.092664Z","shell.execute_reply.started":"2023-11-03T14:41:04.051165Z","shell.execute_reply":"2023-11-03T14:41:04.091435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define Dataset\n\nFor `site_3`, I decided to use the same procedure as I did for `site_1` and `site_2`, which is, crop 5 seconds out of the clip and provide prediction on that short clip.\nThe only difference is that I crop 5 seconds short clip from start to the end of the `site_3` clip and aggeregate predictions for each short clip after I did prediction for all those short clips.","metadata":{}},{"cell_type":"code","source":"def mono_to_color(X: np.ndarray,\n                  mean=None,\n                  std=None,\n                  norm_max=None,\n                  norm_min=None,\n                  eps=1e-6):\n    \"\"\"\n    Code from https://www.kaggle.com/daisukelab/creating-fat2019-preprocessed-data\n    \"\"\"\n    # Stack X as [X,X,X]\n    X = np.stack([X, X, X], axis=-1)\n\n    # Standardize\n    mean = mean or X.mean()\n    X = X - mean\n    std = std or X.std()\n    Xstd = X / (std + eps)\n    _min, _max = Xstd.min(), Xstd.max()\n    norm_max = norm_max or _max\n    norm_min = norm_min or _min\n    if (_max - _min) > eps:\n        # Normalize to [0, 255]\n        V = Xstd\n        V[V < norm_min] = norm_min\n        V[V > norm_max] = norm_max\n        V = 255 * (V - norm_min) / (norm_max - norm_min)\n        V = V.astype(np.uint8)\n    else:\n        # Just zero\n        V = np.zeros_like(Xstd, dtype=np.uint8)\n    return V\n\n\nclass TestDataset(data.Dataset):\n    def __init__(self, df: pd.DataFrame, clip: np.ndarray,\n                 img_size=224, melspectrogram_parameters={}):\n        self.df = df\n        self.clip = clip\n        self.img_size = img_size\n        self.melspectrogram_parameters = melspectrogram_parameters\n        \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, idx: int):\n        SR = 32000\n        sample = self.df.loc[idx, :]\n        site = sample.site\n        row_id = sample.row_id\n        \n        if site == \"site_3\":\n            y = self.clip.astype(np.float32)\n            len_y = len(y)\n            start = 0\n            end = SR * 5\n            images = []\n            while len_y > start:\n                y_batch = y[start:end].astype(np.float32)\n                if len(y_batch) != (SR * 5):\n                    break\n                start = end\n                end = end + SR * 5\n                \n                melspec = librosa.feature.melspectrogram(y_batch,\n                                                         sr=SR,\n                                                         **self.melspectrogram_parameters)\n                melspec = librosa.power_to_db(melspec).astype(np.float32)\n                image = mono_to_color(melspec)\n                height, width, _ = image.shape\n                image = cv2.resize(image, (int(width * self.img_size / height), self.img_size))\n                image = np.moveaxis(image, 2, 0)\n                image = (image / 255.0).astype(np.float32)\n                images.append(image)\n            images = np.asarray(images)\n            return images, row_id, site\n        else:\n            end_seconds = int(sample.seconds)\n            start_seconds = int(end_seconds - 5)\n            \n            start_index = SR * start_seconds\n            end_index = SR * end_seconds\n            \n            y = self.clip[start_index:end_index].astype(np.float32)\n\n            melspec = librosa.feature.melspectrogram(y, sr=SR, **self.melspectrogram_parameters)\n            melspec = librosa.power_to_db(melspec).astype(np.float32)\n\n            image = mono_to_color(melspec)\n            height, width, _ = image.shape\n            image = cv2.resize(image, (int(width * self.img_size / height), self.img_size))\n            image = np.moveaxis(image, 2, 0)\n            image = (image / 255.0).astype(np.float32)\n\n            return image, row_id, site","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.094291Z","iopub.execute_input":"2023-11-03T14:41:04.094734Z","iopub.status.idle":"2023-11-03T14:41:04.124228Z","shell.execute_reply.started":"2023-11-03T14:41:04.094670Z","shell.execute_reply":"2023-11-03T14:41:04.123302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Prediction loop","metadata":{}},{"cell_type":"code","source":"def get_model(config: dict, weights_path: str):\n    model = ResNet(**config)\n    checkpoint = torch.load(weights_path)\n    model.load_state_dict(checkpoint[\"model_state_dict\"])\n    device = torch.device(\"cuda\")\n    model.to(device)\n    model.eval()\n    return model","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.125500Z","iopub.execute_input":"2023-11-03T14:41:04.125888Z","iopub.status.idle":"2023-11-03T14:41:04.137562Z","shell.execute_reply.started":"2023-11-03T14:41:04.125856Z","shell.execute_reply":"2023-11-03T14:41:04.136795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prediction_for_clip(test_df: pd.DataFrame, \n                        clip: np.ndarray, \n                        model: ResNet, \n                        mel_params: dict, \n                        threshold=0.5):\n\n    dataset = TestDataset(df=test_df, \n                          clip=clip,\n                          img_size=224,\n                          melspectrogram_parameters=mel_params)\n    loader = data.DataLoader(dataset, batch_size=1, shuffle=False)\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    \n    model.eval()\n    prediction_dict = {}\n    for image, row_id, site in progress_bar(loader):\n        site = site[0]\n        row_id = row_id[0]\n        if site in {\"site_1\", \"site_2\"}:\n            image = image.to(device)\n\n            with torch.no_grad():\n                prediction = model(image)\n                proba = prediction[\"multilabel_proba\"].detach().cpu().numpy().reshape(-1)\n\n            events = proba >= threshold\n            labels = np.argwhere(events).reshape(-1).tolist()\n\n        else:\n            # to avoid prediction on large batch\n            image = image.squeeze(0)\n            batch_size = 16\n            whole_size = image.size(0)\n            if whole_size % batch_size == 0:\n                n_iter = whole_size // batch_size\n            else:\n                n_iter = whole_size // batch_size + 1\n                \n            all_events = set()\n            for batch_i in range(n_iter):\n                batch = image[batch_i * batch_size:(batch_i + 1) * batch_size]\n                if batch.ndim == 3:\n                    batch = batch.unsqueeze(0)\n\n                batch = batch.to(device)\n                with torch.no_grad():\n                    prediction = model(batch)\n                    proba = prediction[\"multilabel_proba\"].detach().cpu().numpy()\n                    \n                events = proba >= threshold\n                for i in range(len(events)):\n                    event = events[i, :]\n                    labels = np.argwhere(event).reshape(-1).tolist()\n                    for label in labels:\n                        all_events.add(label)\n                        \n            labels = list(all_events)\n        if len(labels) == 0:\n            prediction_dict[row_id] = \"nocall\"\n        else:\n            labels_str_list = list(map(lambda x: INV_BIRD_CODE[x], labels))\n            label_string = \" \".join(labels_str_list)\n            prediction_dict[row_id] = label_string\n    return prediction_dict","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.139091Z","iopub.execute_input":"2023-11-03T14:41:04.139400Z","iopub.status.idle":"2023-11-03T14:41:04.161408Z","shell.execute_reply.started":"2023-11-03T14:41:04.139372Z","shell.execute_reply":"2023-11-03T14:41:04.160231Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prediction(test_df: pd.DataFrame,\n               test_audio: Path,\n               model_config: dict,\n               mel_params: dict,\n               weights_path: str,\n               threshold=0.5):\n    model = get_model(model_config, weights_path)\n    unique_audio_id = test_df.audio_id.unique()\n\n    warnings.filterwarnings(\"ignore\")\n    prediction_dfs = []\n    for audio_id in unique_audio_id:\n        with timer(f\"Loading {audio_id}\", logger):\n#             clip, _ = librosa.load(test_audio / (audio_id + \".mp3\"),\n#                                    sr=TARGET_SR,\n#                                    mono=True,\n#                                    res_type=\"kaiser_fast\")\n            clip, _ = librosa.load(test_audio / (audio_id + \".wav\"),\n                                   sr=TARGET_SR,\n                                   mono=True,\n                                   res_type=\"kaiser_fast\")\n        test_df_for_audio_id = test_df.query(\n            f\"audio_id == '{audio_id}'\").reset_index(drop=True)\n        with timer(f\"Prediction on {audio_id}\", logger):\n            prediction_dict = prediction_for_clip(test_df_for_audio_id,\n                                                  clip=clip,\n                                                  model=model,\n                                                  mel_params=mel_params,\n                                                  threshold=threshold)\n        row_id = list(prediction_dict.keys())\n        birds = list(prediction_dict.values())\n        prediction_df = pd.DataFrame({\n            \"row_id\": row_id,\n            \"birds\": birds\n        })\n        prediction_dfs.append(prediction_df)\n    \n    prediction_df = pd.concat(prediction_dfs, axis=0, sort=False).reset_index(drop=True)\n    return prediction_df","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.162787Z","iopub.execute_input":"2023-11-03T14:41:04.163104Z","iopub.status.idle":"2023-11-03T14:41:04.177374Z","shell.execute_reply.started":"2023-11-03T14:41:04.163073Z","shell.execute_reply":"2023-11-03T14:41:04.176402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Prediction","metadata":{}},{"cell_type":"code","source":"os.path.exists('/kaggle/working/test_audio/CPL_20190611_093000_5.wav')","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.178808Z","iopub.execute_input":"2023-11-03T14:41:04.179228Z","iopub.status.idle":"2023-11-03T14:41:04.191456Z","shell.execute_reply.started":"2023-11-03T14:41:04.179185Z","shell.execute_reply":"2023-11-03T14:41:04.190616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = prediction(test_df=test,\n                        test_audio=test_audio,\n                        model_config=model_config,\n                        mel_params=melspectrogram_parameters,\n                        weights_path=weights_path,\n                        threshold=0.5)\nsubmission.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:04.192788Z","iopub.execute_input":"2023-11-03T14:41:04.193115Z","iopub.status.idle":"2023-11-03T14:41:13.429075Z","shell.execute_reply.started":"2023-11-03T14:41:04.193084Z","shell.execute_reply":"2023-11-03T14:41:13.428180Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission","metadata":{"execution":{"iopub.status.busy":"2023-11-03T14:41:13.430326Z","iopub.execute_input":"2023-11-03T14:41:13.430622Z","iopub.status.idle":"2023-11-03T14:41:13.440561Z","shell.execute_reply.started":"2023-11-03T14:41:13.430592Z","shell.execute_reply":"2023-11-03T14:41:13.439787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}