{"cells":[{"metadata":{},"cell_type":"markdown","source":"\"drawing\"","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Understand Business Problem \n\nIn the challenge, you are identify which birds are calling in long recordings, given training data generated in meaningfully different contexts. This is the exact problem facing scientists trying to automate the remote monitoring of bird populations.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Data Overview \n\n\n`train_audio` : The train data consists of short recordings of individual bird calls generously uploaded by users of xenocanto.org.\n\n`test_audio` : The hidden test_audio directory contains approximately 150 recordings in mp3 format, each roughly 10 minutes long. They will not all fit in a notebook's memory at the same time. The recordings were taken at three separate remote locations. Sites 1 and 2 were labeled in 5 second increments and need matching predictions, but due to the time consuming nature of the labeling process the site 3 files are only labeled at the file level. Accordingly, site 3 has relatively few rows in the test set and needs lower time resolution predictions.\n\nTwo example soundscapes from another data source are also provided to illustrate how the soundscapes are labeled and the hidden dataset folder structure. The two example audio files are `BLKFR-10-CPL_20190611_093000.pt540.mp3` and `ORANGE-7-CAP_20190606_093000.pt623.mp3`. These soundscapes were kindly provided by Jack Dumbacher of the California Academy of Science's Department of Ornithology and Mammology.\n\n`test.csv` Only the first three rows are available for download; the full `test.csv` is in the hidden test set.\n\n- `site`: Site ID.\n\n- `row_id`: ID code for the row.\n\n- `seconds`: the second ending the time window, if any. Site 3 time windows cover the entire audio file and have null entries for seconds.\n\n- `audio_id`: ID code for the audio file.\n\n- `example_test_audio_metadata.csv` Complete metadata for the example test audio. These labels have higher time precision than is used for the hidden test set.\n\n`example_test_audio_summary.csv` Metadata for the example test audio, converted to the same format as used in the hidden test set.\n\n- `filename_seconds`: a row identifier.\n\n- `birds`: all ebird codes present in the time window.\n\n- `filename`: audio file names\n\n- `seconds`: the second ending the time window.\n\n`train.csv` A wide range of metadata is provided for the training data. The most directly relevant fields are:\n\n- `ebird_code`: a code for the bird species. You can review detailed information about the bird codes by appending the code to https://ebird.org/species/, such as https://ebird.org/species/amecro for the American Crow.\n\n- `recodist`: the user who provided the recording.\n\n- `location`: where the recording was taken. Some bird species may have local call 'dialects', so you may want to seek geographic diversity in your training data.\n\n- `date`: while some bird calls can be made year round, such as an alarm call, some are restricted to a specific season. You may want to seek temporal diversity in your training data.\n\n- `filename`: the name of the associated audio file.","execution_count":null},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!pip install pretrainedmodels\n!pip install pydub","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport librosa\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom tqdm import tqdm\nimport random\n\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.model_selection import KFold, StratifiedKFold\nimport math\nfrom collections import OrderedDict\n\nfrom PIL import Image\nimport albumentations\nfrom pydub import AudioSegment\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom torch.utils.data import Dataset, DataLoader\n\nimport pretrainedmodels\n\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Preprocessing ","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/birdsong-recognition/train.csv\")\ntest = pd.read_csv(\"../input/birdsong-recognition/test.csv\")\nsubmission = pd.read_csv(\"../input/birdsong-recognition/sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### e-bird code\n\na code for the bird species. we need to predict `ebird_code` using metadata and audio data \n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(\"Number of Unique birds : \", train.ebird_code.nunique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### top10 Birds\n\nwe are taking top10 birds to build stater model","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"top10_birds = list(train.ebird_code.value_counts().index[:10])\n\ntrain = train[train.ebird_code.isin(top10_birds)]\n\n# label encoding for target values\ntrain[\"ebird_label\"] = LabelEncoder().fit_transform(train.ebird_code.values)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### K-Fold","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train.loc[:, \"kfold\"] = -1\n\ntrain= train.sample(frac=1).reset_index(drop=True)\n\nX = train.filename.values\ny = train.ebird_code.values\n\nkfold = StratifiedKFold(n_splits=5)\n\nfor fold, (t_idx, v_idx) in enumerate(kfold.split(X, y)):\n train.loc[v_idx, \"kfold\"] = fold\n\nprint(train.kfold.value_counts())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Arguments","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class args:\n \n ROOT_PATH = \"../input/birdsong-recognition/train_audio\"\n \n num_classes = 10\n max_duration= 5 # seconds\n \n sample_rate = 32000\n \n img_height = 128\n img_width = 313\n std = (0.229, 0.224, 0.225)\n mean = (0.485, 0.456, 0.406)\n \n batch_size = 16\n num_workers = 4\n epochs = 5\n \n lr = 0.001\n wd = 1e-5\n momentum = 0.9\n eps = 1e-8\n betas = (0.9, 0.999)\n \n melspectrogram_parameters = {\n \"n_mels\": 128,\n \"fmin\": 20,\n \"fmax\": 16000\n }\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loading Audio Files","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_audio(path):\n try:\n sound = AudioSegment.from_mp3(path)\n sound = sound.set_frame_rate(args.sample_rate)\n sound = sound[:args.max_duration*1000]\n sound_array = np.array(sound.get_array_of_samples())\n except:\n sound_array = np.random.rand(args.sample_rate * args.max_duration)\n \n return sound_array, args.sample_rate","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Example\nsound, sample_rate = load_audio(\"../input/birdsong-recognition/train_audio/ameavo/XC139921.mp3\")\nplt.plot(sound)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Melspectrogram\n\n- extract Melspectrogram features from raw audio into using librosa","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def Melspectrogram(audio_path):\n \n y, sr = load_audio(audio_path)\n y = y.astype(float)\n\n melspec = librosa.feature.melspectrogram(y, sr=args.sample_rate, **args.melspectrogram_parameters)\n melspec = librosa.power_to_db(melspec).astype(np.float32)\n\n return melspec\n\n\n# Example\nspect = Melspectrogram(\"../input/birdsong-recognition/train_audio/ameavo/XC139921.mp3\")\nprint(\"shape \", spect.shape)\n\nplt.figure(figsize=(5 ,30))\nplt.imshow(spect)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pytorch DataLoader","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class BirdDataset:\n def __init__(self, df):\n \n self.filename = df.filename.values\n self.ebird_label = df.ebird_label.values\n self.ebird_code = df.ebird_code.values\n \n def __len__(self):\n return len(self.filename)\n \n def __getitem__(self, item):\n \n filename = self.filename[item]\n ebird_code = self.ebird_code[item]\n ebird_label = self.ebird_label[item]\n\n spect = Melspectrogram(f\"{args.ROOT_PATH}/{ebird_code}/{filename}\")\n \n spect = torch.FloatTensor(spect)\n target = ebird_label\n \n return spect, target\n \ndef _collate_fn(batch):\n def func(p):\n return p[0].size(1)\n \n batch = sorted(batch, key=lambda sample: sample[0].size(1), reverse=True)\n longest_sample = max(batch, key=func)[0]\n freq_size = longest_sample.size(0)\n minibatch_size = len(batch)\n max_seqlength = longest_sample.size(1)\n inputs = torch.zeros(minibatch_size, 1, freq_size, max_seqlength)\n input_percentages = torch.FloatTensor(minibatch_size)\n targets = []\n for x in range(minibatch_size):\n sample = batch[x]\n tensor = sample[0]\n target = sample[1]\n seq_length = tensor.size(1)\n inputs[x][0].narrow(1, 0, seq_length).copy_(tensor)\n input_percentages[x] = seq_length / float(max_seqlength)\n targets.append(target)\n\n targets = torch.tensor(targets, dtype=torch.long)\n return inputs, targets, input_percentages\n\n\nclass AudioDataLoader(DataLoader):\n \n def __init__(self, *args, **kwargs):\n \n \"\"\"\n Creates a data loader for AudioDatasets\n \"\"\"\n super(AudioDataLoader, self).__init__(*args, **kwargs)\n self.collate_fn = _collate_fn\n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Example \ndataset = BirdDataset(train)\nd = dataset.__getitem__(10)\n\nd[0].shape, d[1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"loader = AudioDataLoader(dataset, batch_size=2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i, d in enumerate(loader):\n print(d[0].shape)\n print(d[1].shape)\n if i == 0:\n break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Deepspeech2 Model\n\nReference : https://github.com/SeanNaren/deepspeech.pytorch/blob/master/model.py","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import math\nfrom collections import OrderedDict\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nsupported_rnns = {\n 'lstm': nn.LSTM,\n 'rnn': nn.RNN,\n 'gru': nn.GRU\n}\nsupported_rnns_inv = dict((v, k) for k, v in supported_rnns.items())\n\n\nclass SequenceWise(nn.Module):\n def __init__(self, module):\n \"\"\"\n Collapses input of dim T*N*H to (T*N)*H, and applies to a module.\n Allows handling of variable sequence lengths and minibatch sizes.\n :param module: Module to apply input to.\n \"\"\"\n super(SequenceWise, self).__init__()\n self.module = module\n\n def forward(self, x):\n t, n = x.size(0), x.size(1)\n x = x.view(t * n, -1)\n x = self.module(x)\n x = x.view(t, n, -1)\n return x\n\n def __repr__(self):\n tmpstr = self.__class__.__name__ + ' (\\n'\n tmpstr += self.module.__repr__()\n tmpstr += ')'\n return tmpstr\n\n\nclass MaskConv(nn.Module):\n def __init__(self, seq_module):\n \"\"\"\n Adds padding to the output of the module based on the given lengths. This is to ensure that the\n results of the model do not change when batch sizes change during inference.\n Input needs to be in the shape of (BxCxDxT)\n :param seq_module: The sequential module containing the conv stack.\n \"\"\"\n super(MaskConv, self).__init__()\n self.seq_module = seq_module\n\n def forward(self, x, lengths):\n \"\"\"\n :param x: The input of size BxCxDxT\n :param lengths: The actual length of each sequence in the batch\n :return: Masked output from the module\n \"\"\"\n for module in self.seq_module:\n x = module(x)\n mask = torch.BoolTensor(x.size()).fill_(0)\n if x.is_cuda:\n mask = mask.cuda()\n for i, length in enumerate(lengths):\n length = length.item()\n if (mask[i].size(2) - length) > 0:\n mask[i].narrow(2, length, mask[i].size(2) - length).fill_(1)\n x = x.masked_fill(mask, 0)\n return x, lengths\n\n\nclass InferenceBatchSoftmax(nn.Module):\n def forward(self, input_):\n if not self.training:\n return F.softmax(input_, dim=-1)\n else:\n return input_\n\n\nclass BatchRNN(nn.Module):\n def __init__(self, input_size, hidden_size, rnn_type=nn.LSTM, bidirectional=False, batch_norm=True):\n super(BatchRNN, self).__init__()\n self.input_size = input_size\n self.hidden_size = hidden_size\n self.bidirectional = bidirectional\n self.batch_norm = SequenceWise(nn.BatchNorm1d(input_size)) if batch_norm else None\n self.rnn = rnn_type(input_size=input_size, hidden_size=hidden_size,\n bidirectional=bidirectional, bias=True)\n self.num_directions = 2 if bidirectional else 1\n\n def flatten_parameters(self):\n self.rnn.flatten_parameters()\n\n def forward(self, x, output_lengths):\n if self.batch_norm is not None:\n x = self.batch_norm(x)\n #print(x.shape)\n x = nn.utils.rnn.pack_padded_sequence(x, output_lengths)\n x, h = self.rnn(x)\n x, _ = nn.utils.rnn.pad_packed_sequence(x)\n if self.bidirectional:\n x = x.view(x.size(0), x.size(1), 2, -1).sum(2).view(x.size(0), x.size(1), -1) # (TxNxH*2) -> (TxNxH) by sum\n return x\n\n\nclass Lookahead(nn.Module):\n # Wang et al 2016 - Lookahead Convolution Layer for Unidirectional Recurrent Neural Networks\n # input shape - sequence, batch, feature - TxNxH\n # output shape - same as input\n def __init__(self, n_features, context):\n super(Lookahead, self).__init__()\n assert context > 0\n self.context = context\n self.n_features = n_features\n self.pad = (0, self.context - 1)\n self.conv = nn.Conv1d(self.n_features, self.n_features, kernel_size=self.context, stride=1,\n groups=self.n_features, padding=0, bias=None)\n\n def forward(self, x):\n x = x.transpose(0, 1).transpose(1, 2)\n x = F.pad(x, pad=self.pad, value=0)\n x = self.conv(x)\n x = x.transpose(1, 2).transpose(0, 1).contiguous()\n return x\n\n def __repr__(self):\n return self.__class__.__name__ + '(' \\\n + 'n_features=' + str(self.n_features) \\\n + ', context=' + str(self.context) + ')'\n\n\nclass DeepSpeech(nn.Module):\n def __init__(self, rnn_type, rnn_hidden_size, nb_layers,\n bidirectional, context=20):\n super(DeepSpeech, self).__init__()\n\n self.hidden_size = rnn_hidden_size\n self.hidden_layers = nb_layers\n self.rnn_type = rnn_type\n self.bidirectional = bidirectional\n\n sample_rate = args.sample_rate\n num_classes = args.num_classes\n\n self.conv = MaskConv(nn.Sequential(\n nn.Conv2d(1, 32, kernel_size=(41, 11), stride=(2, 2), padding=(20, 5)),\n nn.BatchNorm2d(32),\n nn.Hardtanh(0, 20, inplace=True),\n nn.Conv2d(32, 32, kernel_size=(21, 11), stride=(2, 1), padding=(10, 5)),\n nn.BatchNorm2d(32),\n nn.Hardtanh(0, 20, inplace=True)\n ))\n \n rnn_input_size = 1024\n \n #print(rnn_input_size)\n\n rnns = []\n rnn = BatchRNN(input_size=rnn_input_size, hidden_size=rnn_hidden_size, rnn_type=rnn_type,\n bidirectional=bidirectional, batch_norm=False)\n rnns.append(('0', rnn))\n for x in range(nb_layers - 1):\n rnn = BatchRNN(input_size=rnn_hidden_size, hidden_size=rnn_hidden_size, rnn_type=rnn_type,\n bidirectional=bidirectional)\n rnns.append(('%d' % (x + 1), rnn))\n self.rnns = nn.Sequential(OrderedDict(rnns))\n self.lookahead = nn.Sequential(\n # consider adding batch norm?\n Lookahead(rnn_hidden_size, context=context),\n nn.Hardtanh(0, 20, inplace=True)\n ) if not bidirectional else None\n\n self.fc = nn.Sequential(\n nn.BatchNorm1d(rnn_hidden_size),\n nn.Linear(rnn_hidden_size, num_classes, bias=False)\n )\n \n\n def forward(self, x, lengths):\n lengths = lengths.cpu().int()\n output_lengths = self.get_seq_lens(lengths)\n x, _ = self.conv(x, output_lengths)\n\n sizes = x.size()\n x = x.view(sizes[0], sizes[1] * sizes[2], sizes[3]) # Collapse feature dimension\n x = x.transpose(1, 2).transpose(0, 1).contiguous() # TxNxH\n \n #print(x.shape)\n\n for rnn in self.rnns:\n x = rnn(x, output_lengths)\n\n if not self.bidirectional: # no need for lookahead layer in bidirectional\n x = self.lookahead(x)\n \n #print(x.shape)\n\n x = self.fc(x[-1])\n #x = x.transpose(0, 1)\n # identity in training mode, softmax in eval mode\n #x = self.inference_softmax(x)\n return x#, output_lengths\n\n def get_seq_lens(self, input_length):\n \"\"\"\n Given a 1D Tensor or Variable containing integer sequence lengths, return a 1D tensor or variable\n containing the size sequences that will be output by the network.\n :param input_length: 1D Tensor\n :return: 1D Tensor scaled by model\n \"\"\"\n seq_len = input_length\n for m in self.conv.modules():\n if type(m) == nn.modules.conv.Conv2d:\n seq_len = ((seq_len + 2 * m.padding[1] - m.dilation[1] * (m.kernel_size[1] - 1) - 1) / m.stride[1] + 1)\n return seq_len.int()\n\n @classmethod\n def load_model(cls, path):\n package = torch.load(path, map_location=lambda storage, loc: storage)\n model = DeepSpeech.load_model_package(package)\n return model\n\n @classmethod\n def load_model_package(cls, package):\n model = cls(rnn_hidden_size=package['hidden_size'],\n nb_layers=package['hidden_layers'],\n labels=package['labels'],\n audio_conf=package['audio_conf'],\n rnn_type=supported_rnns[package['rnn_type']],\n bidirectional=package.get('bidirectional', True))\n model.load_state_dict(package['state_dict'])\n return model\n\n def serialize_state(self):\n return {\n 'hidden_size': self.hidden_size,\n 'hidden_layers': self.hidden_layers,\n 'rnn_type': supported_rnns_inv.get(self.rnn_type, self.rnn_type.__name__.lower()),\n 'audio_conf': self.audio_conf,\n 'labels': self.labels,\n 'state_dict': self.state_dict(),\n 'bidirectional': self.bidirectional,\n }\n\n @staticmethod\n def get_param_size(model):\n params = 0\n for p in model.parameters():\n tmp = 1\n for x in p.size():\n tmp *= x\n params += tmp\n return params","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model = DeepSpeech(rnn_type=nn.LSTM, rnn_hidden_size=1024, nb_layers=2, bidirectional=False)\nmodel","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Utility functions","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def to_list(tensor):\n return tensor.detach().cpu().tolist()\n\nclass AverageMeter(object):\n \"\"\"Computes and stores the average and current values\"\"\"\n def __init__(self):\n self.reset()\n \n def __init__(self):\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count\n\ndef get_position_accuracy(logits, labels):\n predictions = np.argmax(F.softmax(logits, dim=1).cpu().data.numpy(), axis=1)\n labels = labels.cpu().data.numpy()\n total_num = 0\n sum_correct = 0\n for i in range(len(labels)):\n if labels[i] >= 0:\n total_num += 1\n if predictions[i] == labels[i]:\n sum_correct += 1\n if total_num == 0:\n total_num = 1e-7\n return np.float32(sum_correct) / total_num, total_num","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loss function","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def loss_fn(preds, labels):\n loss = nn.CrossEntropyLoss(ignore_index=-1)(preds, labels)\n return loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### train & validation functions","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_fn(train_loader, model, optimizer, epoch):\n total_loss = AverageMeter()\n accuracies = AverageMeter()\n \n model.train()\n\n t = tqdm(train_loader)\n for step, d in enumerate(t):\n \n inputs = d[0].to(args.device)\n targets = d[1].to(args.device)\n input_percentages = d[2].to(args.device)\n input_sizes = input_percentages.mul_(int(inputs.size(3))).int()\n \n outputs = model(inputs, input_sizes)\n\n loss = loss_fn(outputs, targets)\n\n acc, n_position = get_position_accuracy(outputs, targets)\n \n\n total_loss.update(loss.item(), n_position)\n accuracies.update(acc, n_position)\n\n optimizer.zero_grad()\n \n loss.backward()\n optimizer.step()\n \n t.set_description(f\"Train E:{epoch+1} - Loss:{total_loss.avg:0.4f} - Acc:{accuracies.avg:0.4f}\")\n \n return total_loss.avg\n\ndef valid_fn(valid_loader, model, epoch):\n total_loss = AverageMeter()\n accuracies = AverageMeter()\n \n model.eval()\n\n t = tqdm(valid_loader)\n for step, d in enumerate(t):\n \n with torch.no_grad():\n \n inputs = d[0].to(args.device)\n targets = d[1].to(args.device)\n input_percentages = d[2].to(args.device)\n input_sizes = input_percentages.mul_(int(inputs.size(3))).int()\n\n outputs = model(inputs, input_sizes)\n\n loss = loss_fn(outputs, targets)\n\n acc, n_position = get_position_accuracy(outputs, targets)\n\n\n total_loss.update(loss.item(), n_position)\n accuracies.update(acc, n_position)\n \n t.set_description(f\"Eval E:{epoch+1} - Loss:{total_loss.avg:0.4f} - Acc:{accuracies.avg:0.4f}\")\n\n return total_loss.avg, accuracies.avg","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def main(fold_index):\n \n model = DeepSpeech(rnn_type=nn.LSTM, rnn_hidden_size=1024, nb_layers=2, bidirectional=False)\n \n args.device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n \n \n # Setting seed\n seed = 42\n random.seed(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.backends.cudnn.deterministic = True\n \n model.to(args.device)\n \n optimizer = torch.optim.AdamW(model.parameters(),\n lr=args.lr,\n betas=args.betas,\n eps=args.eps,\n weight_decay=args.wd)\n \n train_df = train[~train.kfold.isin([fold_index])]\n \n train_dataset = BirdDataset(df=train_df)\n \n train_loader = AudioDataLoader(\n dataset = train_dataset,\n batch_size = args.batch_size,\n shuffle = True,\n num_workers = args.num_workers,\n pin_memory = True,\n drop_last = False\n )\n \n \n valid_df = train[train.kfold.isin([fold_index])]\n \n valid_dataset = BirdDataset(df=valid_df)\n \n valid_loader = AudioDataLoader(\n dataset = valid_dataset,\n batch_size = args.batch_size,\n shuffle = False,\n num_workers = args.num_workers,\n pin_memory = True,\n drop_last = False\n )\n \n best_acc = 0\n \n for epoch in range(args.epochs):\n train_loss = train_fn(train_loader, model, optimizer, epoch)\n valid_loss, valid_acc = valid_fn(valid_loader, model, epoch)\n \n print(f\"**** Epoch {epoch+1} **==>** Accuracy = {valid_acc}\")\n \n if valid_acc > best_acc:\n print(\"**** Model Improved !!!! Saving Model\")\n torch.save(model.state_dict(), f\"fold_{fold_index}.bin\")\n best_acc = valid_acc ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5 Folds","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold0\nmain(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold1\n#main(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold2\n#main(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold3\n#main(3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold4\n#main(4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Please upvote if you like it. It motivates me. Thank you ☺️ .

","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}