{"cells":[{"metadata":{},"cell_type":"markdown","source":"\"drawing\"","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Understand Business Problem \n\nIn the challenge, you are identify which birds are calling in long recordings, given training data generated in meaningfully different contexts. This is the exact problem facing scientists trying to automate the remote monitoring of bird populations.\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Data Overview \n\n\n`train_audio` : The train data consists of short recordings of individual bird calls generously uploaded by users of xenocanto.org.\n\n`test_audio` : The hidden test_audio directory contains approximately 150 recordings in mp3 format, each roughly 10 minutes long. They will not all fit in a notebook's memory at the same time. The recordings were taken at three separate remote locations. Sites 1 and 2 were labeled in 5 second increments and need matching predictions, but due to the time consuming nature of the labeling process the site 3 files are only labeled at the file level. Accordingly, site 3 has relatively few rows in the test set and needs lower time resolution predictions.\n\nTwo example soundscapes from another data source are also provided to illustrate how the soundscapes are labeled and the hidden dataset folder structure. The two example audio files are `BLKFR-10-CPL_20190611_093000.pt540.mp3` and `ORANGE-7-CAP_20190606_093000.pt623.mp3`. These soundscapes were kindly provided by Jack Dumbacher of the California Academy of Science's Department of Ornithology and Mammology.\n\n`test.csv` Only the first three rows are available for download; the full `test.csv` is in the hidden test set.\n\n- `site`: Site ID.\n\n- `row_id`: ID code for the row.\n\n- `seconds`: the second ending the time window, if any. Site 3 time windows cover the entire audio file and have null entries for seconds.\n\n- `audio_id`: ID code for the audio file.\n\n- `example_test_audio_metadata.csv` Complete metadata for the example test audio. These labels have higher time precision than is used for the hidden test set.\n\n`example_test_audio_summary.csv` Metadata for the example test audio, converted to the same format as used in the hidden test set.\n\n- `filename_seconds`: a row identifier.\n\n- `birds`: all ebird codes present in the time window.\n\n- `filename`: audio file names\n\n- `seconds`: the second ending the time window.\n\n`train.csv` A wide range of metadata is provided for the training data. The most directly relevant fields are:\n\n- `ebird_code`: a code for the bird species. You can review detailed information about the bird codes by appending the code to https://ebird.org/species/, such as https://ebird.org/species/amecro for the American Crow.\n\n- `recodist`: the user who provided the recording.\n\n- `location`: where the recording was taken. Some bird species may have local call 'dialects', so you may want to seek geographic diversity in your training data.\n\n- `date`: while some bird calls can be made year round, such as an alarm call, some are restricted to a specific season. You may want to seek temporal diversity in your training data.\n\n- `filename`: the name of the associated audio file.","execution_count":null},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py\n!python pytorch-xla-env-setup.py --version nightly --apt-packages libomp5 libopenblas-dev\n!pip install pretrainedmodels\n!pip install pydub","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport librosa\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom tqdm import tqdm\nimport random\n\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.model_selection import KFold, StratifiedKFold\nimport math\nfrom collections import OrderedDict\n\nfrom PIL import Image\nimport albumentations\nfrom pydub import AudioSegment\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom torch.utils.data import Dataset, DataLoader\n\nimport pretrainedmodels\n\nimport torch_xla.core.xla_model as xm\nimport torch_xla.distributed.parallel_loader as pl\nimport torch_xla.distributed.xla_multiprocessing as xmp\n\nimport warnings\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Preprocessing ","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/birdsong-recognition/train.csv\")\ntest = pd.read_csv(\"../input/birdsong-recognition/test.csv\")\nsubmission = pd.read_csv(\"../input/birdsong-recognition/sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### e-bird code\n\na code for the bird species. we need to predict `ebird_code` using metadata and audio data \n","execution_count":null},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(\"Number of Unique birds : \", train.ebird_code.nunique())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# label encoding for target values\ntrain[\"ebird_label\"] = LabelEncoder().fit_transform(train.ebird_code.values)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### K-Fold","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train.loc[:, \"kfold\"] = -1\n\ntrain= train.sample(frac=1).reset_index(drop=True)\n\nX = train.filename.values\ny = train.ebird_code.values\n\nkfold = StratifiedKFold(n_splits=5)\n\nfor fold, (t_idx, v_idx) in enumerate(kfold.split(X, y)):\n train.loc[v_idx, \"kfold\"] = fold\n\nprint(train.kfold.value_counts())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Arguments","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class args:\n \n ROOT_PATH = \"../input/birdsong-recognition/train_audio\"\n \n num_classes = 264\n max_duration= 5 # seconds\n \n sample_rate = 32000\n \n img_height = 128\n img_width = 313\n std = (0.229, 0.224, 0.225)\n mean = (0.485, 0.456, 0.406)\n \n batch_size = 16\n num_workers = 4\n epochs = 2\n \n lr = 0.001\n wd = 1e-5\n momentum = 0.9\n eps = 1e-8\n betas = (0.9, 0.999)\n \n melspectrogram_parameters = {\n \"n_mels\": 128,\n \"fmin\": 20,\n \"fmax\": 16000\n }\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loading Audio Files","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_audio(path):\n try:\n sound = AudioSegment.from_mp3(path)\n sound = sound.set_frame_rate(args.sample_rate)\n sound = sound[:args.max_duration*1000]\n sound_array = np.array(sound.get_array_of_samples())\n except:\n sound_array = np.random.rand(args.sample_rate * args.max_duration)\n \n return sound_array, args.sample_rate","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Example\nsound, sample_rate = load_audio(\"../input/birdsong-recognition/train_audio/ameavo/XC139921.mp3\")\nplt.plot(sound)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Melspectrogram\n\n- extract Melspectrogram features from raw audio into using librosa","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def Melspectrogram(audio_path):\n \n y, sr = load_audio(audio_path)\n y = y.astype(float)\n\n melspec = librosa.feature.melspectrogram(y, sr=args.sample_rate, **args.melspectrogram_parameters)\n melspec = librosa.power_to_db(melspec).astype(np.float32)\n\n return melspec\n\n\n# Example\nspect = Melspectrogram(\"../input/birdsong-recognition/train_audio/ameavo/XC139921.mp3\")\nprint(\"shape \", spect.shape)\n\nplt.figure(figsize=(5 ,30))\nplt.imshow(spect)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pytorch DataLoader","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class BirdDataset:\n def __init__(self, df):\n \n self.filename = df.filename.values\n self.ebird_label = df.ebird_label.values\n self.ebird_code = df.ebird_code.values\n \n self.aug = albumentations.Compose([\n albumentations.Resize(args.img_height, args.img_width, always_apply=True),\n albumentations.Normalize(args.mean, args.std, always_apply=True)\n ])\n \n def __len__(self):\n return len(self.filename)\n \n def __getitem__(self, item):\n \n filename = self.filename[item]\n ebird_code = self.ebird_code[item]\n ebird_label = self.ebird_label[item]\n\n spect = Melspectrogram(f\"{args.ROOT_PATH}/{ebird_code}/{filename}\")\n spect = Image.fromarray(spect).convert(\"RGB\") # converted to RGB\n spect = self.aug(image=np.array(spect))[\"image\"] # apply augmentation\n spect = np.transpose(spect, (2, 0, 1)).astype(np.float32)\n \n target = ebird_label\n \n return {\n \"spect\" : torch.tensor(spect, dtype=torch.float), \n \"target\" : torch.tensor(target, dtype=torch.long)\n }\n \n ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Example \ndataset = BirdDataset(train)\nd = dataset.__getitem__(10)\n\nd[\"spect\"].shape, d[\"target\"]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### ResNet50 Model","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class ResNet50(nn.Module):\n def __init__(self, pretrained):\n super(ResNet50, self).__init__()\n if pretrained is True:\n self.model = pretrainedmodels.__dict__[\"resnet50\"](pretrained=\"imagenet\")\n else:\n self.model = pretrainedmodels.__dict__[\"resnet50\"](pretrained=None)\n \n self.l0 = nn.Linear(2048, args.num_classes)\n \n def forward(self, x):\n bs, _, _, _ = x.shape\n x = self.model.features(x)\n x = F.adaptive_avg_pool2d(x, 1).reshape(bs, -1)\n x = self.l0(x)\n \n return x\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Utility functions","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def to_list(tensor):\n return tensor.detach().cpu().tolist()\n\ndef reduce_fn(vals):\n return sum(vals) / len(vals)\n\nclass AverageMeter(object):\n \"\"\"Computes and stores the average and current values\"\"\"\n def __init__(self):\n self.reset()\n \n def __init__(self):\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count\n\ndef get_position_accuracy(logits, labels):\n predictions = np.argmax(F.softmax(logits, dim=1).cpu().data.numpy(), axis=1)\n labels = labels.cpu().data.numpy()\n total_num = 0\n sum_correct = 0\n for i in range(len(labels)):\n if labels[i] >= 0:\n total_num += 1\n if predictions[i] == labels[i]:\n sum_correct += 1\n if total_num == 0:\n total_num = 1e-7\n return np.float32(sum_correct) / total_num, total_num","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loss function","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def loss_fn(preds, labels):\n loss = nn.CrossEntropyLoss(ignore_index=-1)(preds, labels)\n return loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### train & validation functions","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_fn(train_loader, model, optimizer, epoch):\n total_loss = AverageMeter()\n accuracies = AverageMeter()\n \n model.train()\n\n t = tqdm(train_loader, disable=not xm.is_master_ordinal())\n for step, d in enumerate(t):\n \n spect = d[\"spect\"].to(args.device)\n targets = d[\"target\"].to(args.device)\n \n outputs = model(spect)\n\n loss = loss_fn(outputs, targets)\n\n acc, n_position = get_position_accuracy(outputs, targets)\n \n\n total_loss.update(loss.item(), n_position)\n accuracies.update(acc, n_position)\n\n optimizer.zero_grad()\n \n loss.backward()\n xm.optimizer_step(optimizer)\n \n t.set_description(f\"Train E:{epoch+1} - Loss:{total_loss.avg:0.4f} - Acc:{accuracies.avg:0.4f}\")\n \n return total_loss.avg\n\ndef valid_fn(valid_loader, model, epoch):\n total_loss = AverageMeter()\n accuracies = AverageMeter()\n \n model.eval()\n\n t = tqdm(valid_loader, disable=not xm.is_master_ordinal())\n for step, d in enumerate(t):\n \n with torch.no_grad():\n \n spect = d[\"spect\"].to(args.device)\n targets = d[\"target\"].to(args.device)\n\n outputs = model(spect)\n\n loss = loss_fn(outputs, targets)\n\n acc, n_position = get_position_accuracy(outputs, targets)\n\n\n total_loss.update(loss.item(), n_position)\n accuracies.update(acc, n_position)\n \n t.set_description(f\"Eval E:{epoch+1} - Loss:{total_loss.avg:0.4f} - Acc:{accuracies.avg:0.4f}\")\n\n return total_loss.avg, accuracies.avg","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def main(fold_index):\n \n MX = ResNet50(pretrained=False)\n \n #args.device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n \n \n # Setting seed\n seed = 42\n random.seed(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.backends.cudnn.deterministic = True\n \n train_df = train[~train.kfold.isin([fold_index])]\n train_dataset = BirdDataset(df=train_df)\n \n \n valid_df = train[train.kfold.isin([fold_index])]\n valid_dataset = BirdDataset(df=valid_df)\n \n def run():\n \n args.device = xm.xla_device()\n model = MX.to(args.device)\n\n \n \n \n train_sampler = torch.utils.data.distributed.DistributedSampler(\n train_dataset,\n num_replicas=xm.xrt_world_size(),\n rank=xm.get_ordinal(),\n shuffle=True\n )\n\n train_loader = DataLoader(\n dataset = train_dataset,\n batch_size = args.batch_size,\n sampler = train_sampler,\n num_workers = args.num_workers,\n pin_memory = True,\n drop_last = False\n )\n\n\n \n valid_sampler = torch.utils.data.distributed.DistributedSampler(\n valid_dataset,\n num_replicas=xm.xrt_world_size(),\n rank=xm.get_ordinal(),\n shuffle=True\n )\n\n valid_loader = DataLoader(\n dataset = valid_dataset,\n batch_size = args.batch_size,\n sampler = valid_sampler,\n num_workers = args.num_workers,\n pin_memory = True,\n drop_last = False\n )\n \n optimizer = torch.optim.AdamW(model.parameters(),\n lr=args.lr * xm.xrt_world_size(),\n betas=args.betas,\n eps=args.eps,\n weight_decay=args.wd\n )\n \n xm.master_print(\"Training is Starting.........\")\n\n best_acc = 0\n\n for epoch in range(args.epochs):\n para_loader = pl.ParallelLoader(train_loader, [args.device])\n train_loss = train_fn(para_loader.per_device_loader(args.device), model, optimizer, epoch)\n \n para_loader = pl.ParallelLoader(valid_loader, [args.device])\n valid_loss, valid_acc = valid_fn(para_loader.per_device_loader(args.device), model, epoch)\n\n xm.master_print(f\"**** Epoch {epoch+1} **==>** Accuracy = {valid_acc}\")\n \n acc = xm.mesh_reduce(\"auc_reduce\", valid_acc, reduce_fn)\n\n if acc > best_acc:\n xm.master_print(\"**** Model Improved !!!! Saving Model\")\n xm.save(model.state_dict(), f\"fold_{fold_index}.bin\")\n best_acc = acc\n \n def _mp_fn(rank, flags):\n torch.set_default_tensor_type('torch.FloatTensor')\n a = run()\n \n FLAGS={}\n xmp.spawn(_mp_fn, args=(FLAGS,), nprocs=8, start_method='fork')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5 Folds","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold0\nmain(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold1\n#main(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold2\n#main(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold3\n#main(3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# fold4\n#main(4)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"

Please upvote if you like it. It motivates me. Thank you ☺️ .

","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}