{"cells": [{"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.009691, "end_time": "2020-08-20T08:22:41.640032", "exception": false, "start_time": "2020-08-20T08:22:41.630341", "status": "completed"}, "tags": []}, "source": ["# NOTE\n", "\n", "I'm just trying to understand [this](https://www.kaggle.com/radek1/esp-starter-pack-from-training-to-submission/notebook) excellent notebook and so am trying to refactor it / clean it up in a way that makes sense to me.\n", "\n", "Am using [nbQA](https://github.com/nbQA-dev/nbQA) for a few code-quality checks (currently `black`, `flake8`, `isort` and `mypy`).\n", "\n", "Please upvote the original notebook (linked above) if you find this useful."]}, {"cell_type": "markdown", "metadata": {"_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0", "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a", "papermill": {"duration": 0.010673, "end_time": "2020-08-20T08:22:41.659463", "exception": false, "start_time": "2020-08-20T08:22:41.648790", "status": "completed"}, "tags": []}, "source": ["In this notebook we will start with training and go all the way to submission \ud83d\ude42 This will put some constraint on our code - the plan is for it to be fairly easy to read and to lend itself well to modifications. Let's get started!"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 34.229392, "end_time": "2020-08-20T08:23:15.898099", "exception": false, "start_time": "2020-08-20T08:22:41.668707", "status": "completed"}, "tags": []}, "outputs": [], "source": ["!pip install /kaggle/input/pyfftw0120-wheel/pyFFTW-0.12.0-cp37-cp37m-manylinux1_x86_64.whl\n", "\n", "\n", "import multiprocessing\n", "import os\n", "import typing as tp\n", "import warnings\n", "from functools import reduce\n", "from pathlib import Path\n", "\n", "import librosa\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "import torch\n", "import torchvision\n", "from pyfftw import empty_aligned\n", "from pyfftw.builders import rfft as rfft_builder"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.007449, "end_time": "2020-08-20T08:23:15.913943", "exception": false, "start_time": "2020-08-20T08:23:15.906494", "status": "completed"}, "tags": []}, "source": ["For training, we will use audio files I preprocessed to spectrograms and saved as numpy arrays. They are not compressed and should be very fast to read. Each spectrogram captures under 30 initial seconds of one of the files in train."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.016177, "end_time": "2020-08-20T08:23:15.939461", "exception": false, "start_time": "2020-08-20T08:23:15.923284", "status": "completed"}, "tags": []}, "outputs": [], "source": ["TRAIN_DIR = Path(\"../input/birdcalldatasetnpy/train_resampled_npy\")\n", "SAMPLE_RATE = 32_000"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.007936, "end_time": "2020-08-20T08:23:15.955982", "exception": false, "start_time": "2020-08-20T08:23:15.948046", "status": "completed"}, "tags": []}, "source": ["This is what ~2 seconds of audio converted to a spectrogram looks like."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.225218, "end_time": "2020-08-20T08:23:16.189028", "exception": false, "start_time": "2020-08-20T08:23:15.963810", "status": "completed"}, "tags": []}, "outputs": [], "source": ["plt.imshow(\n", "    np.load(\"../input/birdcalldatasetnpy/train_resampled_npy/aldfly/XC135454.npy\")[:, :256]\n", ")"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.007997, "end_time": "2020-08-20T08:23:16.205392", "exception": false, "start_time": "2020-08-20T08:23:16.197395", "status": "completed"}, "tags": []}, "source": ["To generate the spectrograms, I used [code](https://github.com/f0k/birdclef2018/blob/master/experiments/audio.py) shared by Jan Schl\u00fcter.\n", "\n", "\n", "Let me copy it over here, we will need it down the road when we predict on the test set."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.049054, "end_time": "2020-08-20T08:23:16.262524", "exception": false, "start_time": "2020-08-20T08:23:16.213470", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def spectrogram(\n", "    samples, sample_rate, frame_len, fps, batch=48, dtype=None, bins=None, plans=None\n", "):\n", "    \"\"\"\n", "    Computes a magnitude spectrogram for a given vector of samples at a given\n", "    sample rate (in Hz), frame length (in samples) and frame rate (in Hz).\n", "    Allows to transform multiple frames at once for improved performance (with\n", "    a default value of 48, more is not always better). Returns a numpy array.\n", "    Allows to return a limited number of bins only, with improved performance\n", "    over discarding them afterwards. Optionally accepts a set of precomputed\n", "    plans created with spectrogram_plans(), required when multi-threading.\n", "    \"\"\"\n", "    if dtype is None:\n", "        dtype = samples.dtype\n", "    if bins is None:\n", "        bins = frame_len // 2 + 1\n", "    if len(samples) < frame_len:\n", "        return np.empty((0, bins), dtype=dtype)\n", "    if plans is None:\n", "        plans = spectrogram_plans(frame_len, batch, dtype)\n", "    rfft1, rfft, win = plans\n", "    hopsize = int(sample_rate // fps)\n", "    num_frames = (len(samples) - frame_len) // hopsize + 1\n", "    nabs = np.abs\n", "    naa = np.asanyarray\n", "    if batch > 1 and num_frames >= batch and samples.flags.c_contiguous:\n", "        frames = np.lib.stride_tricks.as_strided(\n", "            samples,\n", "            shape=(num_frames, frame_len),\n", "            strides=(samples.strides[0] * hopsize, samples.strides[0]),\n", "        )\n", "        spect = [\n", "            nabs(rfft(naa(frames[pos : pos + batch :], dtype) * win)[:, :bins])\n", "            for pos in range(0, num_frames - batch + 1, batch)\n", "        ]\n", "        samples = samples[(num_frames // batch * batch) * hopsize : :]\n", "        num_frames = num_frames % batch\n", "    else:\n", "        spect = []\n", "    if num_frames:\n", "        spect.append(\n", "            np.vstack(\n", "                [\n", "                    nabs(rfft1(naa(samples[pos : pos + frame_len :], dtype) * win)[:bins:])\n", "                    for pos in range(0, len(samples) - frame_len + 1, hopsize)\n", "                ]\n", "            )\n", "        )\n", "    return np.vstack(spect) if len(spect) > 1 else spect[0]\n", "\n", "\n", "def create_mel_filterbank(sample_rate, frame_len, num_bands, min_freq, max_freq):\n", "    \"\"\"\n", "    Creates a mel filterbank of `num_bands` triangular filters, with the first\n", "    filter starting at `min_freq` and the last one stopping at `max_freq`.\n", "    Returns the filterbank as a matrix suitable for a dot product against\n", "    magnitude spectra created from samples at a sample rate of `sample_rate`\n", "    with a window length of `frame_len` samples.\n", "    \"\"\"\n", "    # prepare output matrix\n", "    input_bins = (frame_len // 2) + 1\n", "    filterbank = np.zeros((input_bins, num_bands))\n", "\n", "    # mel-spaced peak frequencies\n", "    min_mel = 1127 * np.log1p(min_freq / 700.0)\n", "    max_mel = 1127 * np.log1p(max_freq / 700.0)\n", "    spacing = (max_mel - min_mel) / (num_bands + 1)\n", "    peaks_mel = min_mel + np.arange(num_bands + 2) * spacing\n", "    peaks_hz = 700 * (np.exp(peaks_mel / 1127) - 1)\n", "    fft_freqs = np.linspace(0, sample_rate / 2.0, input_bins)\n", "    peaks_bin = np.searchsorted(fft_freqs, peaks_hz)\n", "\n", "    # fill output matrix with triangular filters\n", "    for b, filt in enumerate(filterbank.T):\n", "        # The triangle starts at the previous filter's peak (peaks_freq[b]),\n", "        # has its maximum at peaks_freq[b+1] and ends at peaks_freq[b+2].\n", "        left_hz, top_hz, right_hz = peaks_hz[b : b + 3]  # b, b+1, b+2\n", "        left_bin, top_bin, right_bin = peaks_bin[b : b + 3]\n", "        # Create triangular filter compatible to yaafe\n", "        filt[left_bin:top_bin] = (fft_freqs[left_bin:top_bin] - left_hz) / (\n", "            top_bin - left_bin\n", "        )\n", "        filt[top_bin:right_bin] = (right_hz - fft_freqs[top_bin:right_bin]) / (\n", "            right_bin - top_bin\n", "        )\n", "        filt[left_bin:right_bin] /= filt[left_bin:right_bin].sum()\n", "\n", "    return filterbank\n", "\n", "\n", "def spectrogram_plans(frame_len, batch=48, dtype=np.float32):\n", "    \"\"\"\n", "    Precompute plans for spectrogram(), for a given frame length, batch size\n", "    and dtype. Returns two plans (single spectrum and batch), and a window.\n", "    \"\"\"\n", "    input_array = empty_aligned((batch, frame_len), dtype=dtype)\n", "    win = np.hanning(frame_len).astype(dtype)\n", "    return (rfft_builder(input_array[0]), rfft_builder(input_array), win)\n", "\n", "\n", "filterbank = create_mel_filterbank(SAMPLE_RATE, 256, 80, 27.5, 10000)\n", "\n", "\n", "def audio_to_melspec(audio):\n", "    spec = spectrogram(audio, SAMPLE_RATE, 256, 128)\n", "    return (spec @ filterbank).T"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008372, "end_time": "2020-08-20T08:23:16.279577", "exception": false, "start_time": "2020-08-20T08:23:16.271205", "status": "completed"}, "tags": []}, "source": ["To train, we will use PyTorch. Let's put together a dataset we will be able to use to train our model."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.97058, "end_time": "2020-08-20T08:23:17.259063", "exception": false, "start_time": "2020-08-20T08:23:16.288483", "status": "completed"}, "tags": []}, "outputs": [], "source": ["BS = 100\n", "MAX_LR = 1e-3\n", "\n", "classes = [directory.name for directory in TRAIN_DIR.iterdir()]\n", "train_items = []\n", "\n", "for directory in TRAIN_DIR.iterdir():\n", "    ebird_code = directory.name\n", "    for recording in directory.iterdir():\n", "        train_items.append((ebird_code, recording))"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.028376, "end_time": "2020-08-20T08:23:17.297752", "exception": false, "start_time": "2020-08-20T08:23:17.269376", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class TrainDataset(torch.utils.data.Dataset):\n", "    def __getitem__(self, idx: int) -> tp.Tuple[np.ndarray, np.ndarray]:\n", "        \"\"\"\n", "        Get:\n", "        - three random segments of the audio file, stacked\n", "        - one-hot-encoded version of target label\n", "        \"\"\"\n", "        cls, path = train_items[idx]\n", "        example = self.get_spec(path)\n", "        return example, self.one_hot_encode(cls)\n", "\n", "    def get_spec(self, path: Path) -> np.ndarray:\n", "        \"\"\"\n", "        x is of shape (80, ?). Lets call this (width, height).\n", "\n", "        If x's height is more than 212, then we select a random\n", "        segment of length 212 from x.\n", "        If it's smaller, then we randomly pad it with zeros so\n", "        that it becomes of length 212.\n", "        We repeat this three times, so that we end up with three\n", "        (possibly overlapping) segments of shape (80, 212).\n", "        \"\"\"\n", "        frames_per_spec = 212\n", "        n_specs = 3\n", "        x = np.load(path)\n", "        width, height = x.shape\n", "        assert width == 80\n", "\n", "        specs = []\n", "        for _ in range(n_specs):\n", "            if x.shape[1] < frames_per_spec:\n", "                spec = np.zeros((80, frames_per_spec))\n", "                start_frame = np.random.randint(frames_per_spec - x.shape[1])\n", "                spec[:, start_frame : start_frame + x.shape[1]] = x\n", "            else:\n", "                start_frame = int(np.random.rand() * (x.shape[1] - frames_per_spec))\n", "                spec = x[:, start_frame : start_frame + frames_per_spec]\n", "            specs.append(spec)\n", "\n", "        stacked_specs = np.stack(specs)\n", "        assert stacked_specs.shape == (n_specs, 80, frames_per_spec)\n", "\n", "        return stacked_specs.astype(np.float32)\n", "\n", "    def show(self, idx: int):\n", "        x = self[idx][0]\n", "        return plt.imshow(x.transpose(1, 2, 0)[:, :, 0])\n", "\n", "    def one_hot_encode(self, cls: str) -> np.ndarray:\n", "        y = classes.index(cls)\n", "        one_hot = np.zeros((len(classes)))\n", "        one_hot[y] = 1\n", "        return one_hot\n", "\n", "    def __len__(self):\n", "        return len(train_items)\n", "\n", "\n", "train_ds = TrainDataset()\n", "\n", "train_dl = torch.utils.data.DataLoader(\n", "    train_ds,\n", "    batch_size=BS,\n", "    num_workers=multiprocessing.cpu_count(),\n", "    pin_memory=True,\n", "    shuffle=True,\n", ")"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.007995, "end_time": "2020-08-20T08:23:17.314730", "exception": false, "start_time": "2020-08-20T08:23:17.306735", "status": "completed"}, "tags": []}, "source": ["We are nearly ready to train! We still need a model and a training loop.\n", "\n", "For the model, let us use an architecture based on resnet34 with pretrained weights."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 6.277939, "end_time": "2020-08-20T08:23:23.600951", "exception": false, "start_time": "2020-08-20T08:23:17.323012", "status": "completed"}, "tags": []}, "outputs": [], "source": ["pretrained_res34 = torchvision.models.resnet34(False)\n", "pretrained_res34.load_state_dict(\n", "    torch.load(\"../input/pretrained-pytorch/resnet34-333f7ec4.pth\")\n", ")\n", "\n", "\n", "class Model(torch.nn.Module):\n", "    def __init__(self):\n", "        super().__init__()\n", "        self.bn = torch.nn.BatchNorm2d(3)\n", "        self.cnn = torch.nn.Sequential(\n", "            *list(pretrained_res34.children())[:-2], torch.nn.AdaptiveMaxPool2d(1)\n", "        )\n", "        self.classifier = torch.nn.Sequential(\n", "            *[\n", "                torch.nn.Linear(512, 512),\n", "                torch.nn.ReLU(),\n", "                torch.nn.Dropout(p=0.2),\n", "                torch.nn.BatchNorm1d(512),\n", "                torch.nn.Linear(512, 512),\n", "                torch.nn.ReLU(),\n", "                torch.nn.Dropout(p=0.2),\n", "                torch.nn.BatchNorm1d(512),\n", "                torch.nn.Linear(512, len(classes)),\n", "            ]\n", "        )\n", "\n", "    def forward(self, x: torch.Tensor) -> torch.Tensor:\n", "        \"\"\"\n", "        Forward pass through network.\n", "\n", "        Will:\n", "\n", "        - normalise (between 0 and 1) and reshape the input so\n", "          it can be fed into the cnn\n", "        - reshape and feed into the classifier\n", "        \"\"\"\n", "\n", "        batch_size, *shape = x.shape\n", "        assert batch_size <= BS\n", "        assert shape == [3, 80, 212]\n", "\n", "        x_reshaped = x.view(x.shape[0], -1)\n", "        assert x_reshaped.shape == (batch_size, reduce(lambda i, j: i * j, shape))\n", "\n", "        max_per_example, _ = x_reshaped.max(1)\n", "        assert max_per_example.shape == (batch_size,)\n", "\n", "        nonzero_mask = max_per_example != 0\n", "        x[nonzero_mask] /= max_per_example[nonzero_mask][:, None, None, None]\n", "\n", "        cnn_output = self.cnn(x)\n", "        assert cnn_output.shape == (batch_size, 512, 1, 1)\n", "\n", "        x = cnn_output.squeeze(3).squeeze(2)\n", "        assert x.shape == (batch_size, 512)\n", "\n", "        x = self.classifier(x)\n", "        assert x.shape == (batch_size, len(classes))\n", "        return x\n", "\n", "\n", "model = Model().cuda()"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008674, "end_time": "2020-08-20T08:23:23.618680", "exception": false, "start_time": "2020-08-20T08:23:23.610006", "status": "completed"}, "tags": []}, "source": ["With the model out of the way, time to implement the training loop and start training!\n", "\n", "To speed up the training and improve our results, let's first train just the new classifier we have created, keeping the convolutional part of our model frozen."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.017699, "end_time": "2020-08-20T08:23:23.644931", "exception": false, "start_time": "2020-08-20T08:23:23.627232", "status": "completed"}, "tags": []}, "outputs": [], "source": ["for param in model.cnn.parameters():\n", "    param.requires_grad = False"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008663, "end_time": "2020-08-20T08:23:23.662949", "exception": false, "start_time": "2020-08-20T08:23:23.654286", "status": "completed"}, "tags": []}, "source": ["Here is our training loop"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.021944, "end_time": "2020-08-20T08:23:23.693642", "exception": false, "start_time": "2020-08-20T08:23:23.671698", "status": "completed"}, "tags": []}, "outputs": [], "source": ["def train(num_epochs):\n", "    model.train()\n", "    criterion = torch.nn.BCEWithLogitsLoss()\n", "    optimizer = torch.optim.Adam(model.parameters(), 1e-3)\n", "    scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, num_epochs)\n", "    for epoch in range(num_epochs):\n", "        for data in train_dl:\n", "            inputs, labels = data[0].cuda(), data[1].cuda()\n", "            optimizer.zero_grad()\n", "\n", "            outputs = model(inputs)\n", "            loss = criterion(outputs, labels)\n", "\n", "            if np.isnan(loss.item()):\n", "                raise Exception(f\"!!! nan encountered in loss !!! epoch: {epoch}\\n\")\n", "            loss.backward()\n", "            optimizer.step()\n", "            scheduler.step()"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008367, "end_time": "2020-08-20T08:23:23.710870", "exception": false, "start_time": "2020-08-20T08:23:23.702503", "status": "completed"}, "tags": []}, "source": ["And let's train!"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 1392.952902, "end_time": "2020-08-20T08:46:36.672719", "exception": false, "start_time": "2020-08-20T08:23:23.719817", "status": "completed"}, "tags": []}, "outputs": [], "source": ["%%time\n", "\n", "train(30)"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008428, "end_time": "2020-08-20T08:46:36.689812", "exception": false, "start_time": "2020-08-20T08:46:36.681384", "status": "completed"}, "tags": []}, "source": ["Our classifier head is now trained! This means that when we start training the entire model, gradients update will not initally mess lower layers of our architecture too much. This technique called progressive unfreezing is extremely valuable. You can read more about it in [Universal Language Model Fine-tuning for Text Classification](https://arxiv.org/abs/1801.06146) by Jeremy Howard and Sebastian Ruder.\n", "\n", "Let's now unfreeze our model and train the entire arch."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 3874.917553, "end_time": "2020-08-20T09:51:11.615915", "exception": false, "start_time": "2020-08-20T08:46:36.698362", "status": "completed"}, "tags": []}, "outputs": [], "source": ["%%time\n", "\n", "for param in model.cnn.parameters():\n", "    param.requires_grad = True\n", "\n", "train(60)"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008707, "end_time": "2020-08-20T09:51:11.633725", "exception": false, "start_time": "2020-08-20T09:51:11.625018", "status": "completed"}, "tags": []}, "source": ["With training out of the way, its time to predict on the test set. To help us structure our work, let's use the extremely helpful [custom check phase](https://www.kaggle.com/c/birdsong-recognition/discussion/159993) shared by [Alex Shonenkov](https://www.kaggle.com/shonenkov)."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.074781, "end_time": "2020-08-20T09:51:11.717541", "exception": false, "start_time": "2020-08-20T09:51:11.642760", "status": "completed"}, "tags": []}, "outputs": [], "source": ["TEST_PATH = (\n", "    Path(\"../input/birdsong-recognition\")\n", "    if os.path.exists(\"../input/birdsong-recognition/test_audio\")\n", "    else Path(\"../input/birdcall-check\")\n", ")\n", "\n", "TEST_AUDIO_PATH = TEST_PATH / \"test_audio\"\n", "test_df = pd.read_csv(TEST_PATH / \"test.csv\")"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008832, "end_time": "2020-08-20T09:51:11.735486", "exception": false, "start_time": "2020-08-20T09:51:11.726654", "status": "completed"}, "tags": []}, "source": ["Our dataset that we will use for inference will need to be able to work with audio files."]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.025727, "end_time": "2020-08-20T09:51:11.770202", "exception": false, "start_time": "2020-08-20T09:51:11.744475", "status": "completed"}, "tags": []}, "outputs": [], "source": ["class AudioDataset(torch.utils.data.Dataset):\n", "    def __init__(self, items, classes, rec):\n", "        self.items = items\n", "        self.vocab = classes\n", "        self.rec = rec\n", "\n", "    def __getitem__(self, idx):\n", "        _, rec_fn, start = self.items[idx]\n", "        x = self.rec[start * SAMPLE_RATE : (start + 5) * SAMPLE_RATE]\n", "        example = self.get_specs(x)\n", "        return example.astype(np.float32)\n", "\n", "    def get_specs(self, x):\n", "        xs = []\n", "        for i in range(3):\n", "            start_frame = int(i * 1.66 * SAMPLE_RATE)\n", "            xs.append(x[start_frame : start_frame + int(1.66 * SAMPLE_RATE)])\n", "\n", "        specs = []\n", "        for x in xs:\n", "            specs.append(audio_to_melspec(x))\n", "        return np.stack(specs).reshape(3, 80, 212)\n", "\n", "    def show(self, idx):\n", "        x = self[idx][0]\n", "        return plt.imshow(x.transpose(1, 2, 0)[:, :, 0])\n", "\n", "    def __len__(self):\n", "        return len(self.items)"]}, {"cell_type": "markdown", "metadata": {"papermill": {"duration": 0.008542, "end_time": "2020-08-20T09:51:11.787948", "exception": false, "start_time": "2020-08-20T09:51:11.779406", "status": "completed"}, "tags": []}, "source": ["Let's predict on the test set and output predictions!"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 11.04722, "end_time": "2020-08-20T09:51:22.844252", "exception": false, "start_time": "2020-08-20T09:51:11.797032", "status": "completed"}, "tags": []}, "outputs": [], "source": ["%%time\n", "\n", "warnings.filterwarnings(\"ignore\", category=UserWarning)\n", "\n", "row_ids = []\n", "all_preds = []\n", "\n", "model.eval()\n", "for audio_id in test_df[test_df.site.isin([\"site_1\", \"site_2\"])].audio_id.unique():\n", "    items = [\n", "        (row.row_id, row.audio_id, int(row.seconds) - 5)\n", "        for idx, row in test_df[test_df.audio_id == audio_id].iterrows()\n", "    ]\n", "    rec = librosa.load(\n", "        TEST_AUDIO_PATH / f\"{audio_id}.mp3\", sr=SAMPLE_RATE, res_type=\"kaiser_fast\"\n", "    )[0]\n", "    test_ds = AudioDataset(items, classes, rec)\n", "    dl = torch.utils.data.DataLoader(test_ds, batch_size=64)\n", "    for batch in dl:\n", "        with torch.no_grad():\n", "            preds = model(batch.cuda()).sigmoid().cpu().detach()\n", "            all_preds.append(preds)\n", "    row_ids += [item[0] for item in items]"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 6.585007, "end_time": "2020-08-20T09:51:29.440419", "exception": false, "start_time": "2020-08-20T09:51:22.855412", "status": "completed"}, "tags": []}, "outputs": [], "source": ["%%time\n", "\n", "for audio_id in test_df[test_df.site == \"site_3\"].audio_id.unique():\n", "    rec = librosa.load(\n", "        TEST_AUDIO_PATH / f\"{audio_id}.mp3\", sr=SAMPLE_RATE, res_type=\"kaiser_fast\"\n", "    )[0]\n", "    # assume only one row per recording for site_3\n", "    current_row = test_df[test_df.audio_id == audio_id].iloc[0]\n", "    duration = rec.shape[0] // SAMPLE_RATE\n", "    items = [\n", "        (current_row.row_id, current_row.audio_id, start_sec)\n", "        for start_sec in [0 + i * 5 for i in range(duration // 5)]\n", "    ]\n", "    test_ds = AudioDataset(items, classes, rec)\n", "    dl = torch.utils.data.DataLoader(test_ds, batch_size=64)\n", "\n", "    preds_for_site = []\n", "    for batch in dl:\n", "        with torch.no_grad():\n", "            preds = model(batch.cuda()).sigmoid().cpu().detach()\n", "            preds_for_site.append(preds)\n", "\n", "    row_ids.append(current_row.row_id)\n", "    preds_for_site_tensor = torch.cat(preds_for_site)\n", "    assert preds_for_site_tensor.shape == (len(items), len(classes))\n", "    current_pred = preds_for_site_tensor.max(0)[0].unsqueeze(0)\n", "    assert current_pred.shape == (1, len(classes))\n", "    all_preds.append(current_pred)"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.018258, "end_time": "2020-08-20T09:51:29.468946", "exception": false, "start_time": "2020-08-20T09:51:29.450688", "status": "completed"}, "tags": []}, "outputs": [], "source": ["all_preds_tensor = torch.cat(all_preds)\n", "assert all_preds_tensor.shape == (len(test_df), len(classes))"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.048942, "end_time": "2020-08-20T09:51:29.527467", "exception": false, "start_time": "2020-08-20T09:51:29.478525", "status": "completed"}, "tags": []}, "outputs": [], "source": ["%%time\n", "thresh = 1.0\n", "minimum_prediction_rate = 0.04\n", "\n", "# Make sure that we are making a prediction for at least\n", "# `minimum_prediction_rate` of the rows.\n", "while (all_preds_tensor > thresh).any(1).float().mean() < minimum_prediction_rate:\n", "    thresh -= 0.001"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.021895, "end_time": "2020-08-20T09:51:29.559221", "exception": false, "start_time": "2020-08-20T09:51:29.537326", "status": "completed"}, "tags": []}, "outputs": [], "source": ["results = []\n", "\n", "for row in all_preds_tensor:\n", "    birds = []\n", "    for idx in np.where(row > thresh)[0]:\n", "        birds.append(classes[idx])\n", "    if not birds:\n", "        birds = [\"nocall\"]\n", "    results.append(\" \".join(birds))"]}, {"cell_type": "code", "execution_count": null, "metadata": {"papermill": {"duration": 0.433348, "end_time": "2020-08-20T09:51:30.002529", "exception": false, "start_time": "2020-08-20T09:51:29.569181", "status": "completed"}, "tags": []}, "outputs": [], "source": ["predicted = pd.DataFrame(data={\"row_id\": row_ids, \"birds\": results})\n", "predicted.to_csv(\"submission.csv\", index=False)"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": []}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6"}, "papermill": {"duration": 5334.832682, "end_time": "2020-08-20T09:51:31.382771", "environment_variables": {}, "exception": null, "input_path": "__notebook__.ipynb", "output_path": "__notebook__.ipynb", "parameters": {}, "start_time": "2020-08-20T08:22:36.550089", "version": "2.1.0"}}, "nbformat": 4, "nbformat_minor": 4}