![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1483555%2F4d0acfb9be5bb112c566bc8d55627aa2%2F2020-09-15%2018.45.50.png?generation=1600214606636136&alt=media\")
Congrats to all winners! And thanks to organizers!
\nMy score is not good, but sharing my gotten knowledge.
My base model is DenseNet201 trained ImageNet.
\nMost people used ResNet, but it not work for me.
I read this paper, and apply applied to my model.
\n
code is:
\nclass BirdcallNet(nn.Module):\n def __init__(self):\n super(BirdcallNet_densenet, self).__init__()\n densenet = densenet161(pretrained=config.PRETRAINED)\n self.features = densenet.features\n self.l8_a = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n self.l8_b = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n\n def forward(self, x, perm=None, gamma=None):\n # input: (batch, channel, Hz, time)\n frames_num = x.shape[3]\n x = x.transpose(3, 2) # (batch, channel, time, Hz)\n h = self.features(x) # (batch, unit, time, Hz)\n\n h = F.relu(h, inplace=True)\n h = torch.mean(h, dim=3) # (batch, unit, time)\n\n xa = self.l8_a(h) # (batch, n_class, time)\n xb = self.l8_b(h) # (batch, n_class, time)\n xb = torch.softmax(xb, dim=2)\n\n pseudo_label = (xa.sigmoid() >= 0.5).float()\n clipwise_preds = torch.sum(xa * xb, dim=2)\n attention_preds = xb\n\n return clipwise_preds, attention_preds, pseudo_label\nhttps://github.com/trtd56/Birdcall
\nhttps://www.ai-shift.co.jp/techblog/1271
\nThank you!!
", "messages": [ { "id": "1012127", "postDate": "09/16/2020 00:04:06", "content": "Congrats to all winners! And thanks to organizers!
\nMy score is not good, but sharing my gotten knowledge.
My base model is DenseNet201 trained ImageNet.
\nMost people used ResNet, but it not work for me.
I read this paper, and apply applied to my model.
\n
code is:
\nclass BirdcallNet(nn.Module):\n def __init__(self):\n super(BirdcallNet_densenet, self).__init__()\n densenet = densenet161(pretrained=config.PRETRAINED)\n self.features = densenet.features\n self.l8_a = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n self.l8_b = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n\n def forward(self, x, perm=None, gamma=None):\n # input: (batch, channel, Hz, time)\n frames_num = x.shape[3]\n x = x.transpose(3, 2) # (batch, channel, time, Hz)\n h = self.features(x) # (batch, unit, time, Hz)\n\n h = F.relu(h, inplace=True)\n h = torch.mean(h, dim=3) # (batch, unit, time)\n\n xa = self.l8_a(h) # (batch, n_class, time)\n xb = self.l8_b(h) # (batch, n_class, time)\n xb = torch.softmax(xb, dim=2)\n\n pseudo_label = (xa.sigmoid() >= 0.5).float()\n clipwise_preds = torch.sum(xa * xb, dim=2)\n attention_preds = xb\n\n return clipwise_preds, attention_preds, pseudo_label\nhttps://github.com/trtd56/Birdcall
\nhttps://www.ai-shift.co.jp/techblog/1271
\nThank you!!
", "rawMarkdown": "Congrats to all winners! And thanks to organizers!\nMy score is not good, but sharing my gotten knowledge.\n\n### modeling\nMy base model is DenseNet201 trained ImageNet.\nMost people used ResNet, but it not work for me.\n\nI read [this paper](https://arxiv.org/pdf/1908.02876.pdf), and apply applied to my model.\n\n\n\ncode is:\n```python\nclass BirdcallNet(nn.Module):\n def __init__(self):\n super(BirdcallNet_densenet, self).__init__()\n densenet = densenet161(pretrained=config.PRETRAINED)\n self.features = densenet.features\n self.l8_a = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n self.l8_b = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n\n def forward(self, x, perm=None, gamma=None):\n # input: (batch, channel, Hz, time)\n frames_num = x.shape[3]\n x = x.transpose(3, 2) # (batch, channel, time, Hz)\n h = self.features(x) # (batch, unit, time, Hz)\n\n h = F.relu(h, inplace=True)\n h = torch.mean(h, dim=3) # (batch, unit, time)\n\n xa = self.l8_a(h) # (batch, n_class, time)\n xb = self.l8_b(h) # (batch, n_class, time)\n xb = torch.softmax(xb, dim=2)\n\n pseudo_label = (xa.sigmoid() >= 0.5).float()\n clipwise_preds = torch.sum(xa * xb, dim=2)\n attention_preds = xb\n\n return clipwise_preds, attention_preds, pseudo_label\n```\n\n#### Parameters\n- BCE loss\n- 5-fold CV\n- Adam\n - learning_rate=1e-3\n - 55 epoch\n - batch size=64\n- CosineAnnealingWarmRestarts\n - T=10\n\n### Data Augumentation\n- Adjust Gamma\n- Spec Augmentation Freq\n- MixUp\n\n### Other Technique\n- SWA\n\n### Not Work for me\n- Denoise(I tried my best, but it didn't work ...)\n- \"nocall\" prediction by PANNs trained model\n- Transformer\n- WaveNet\n- use secondary_labels\n- optimize threshold\n- Focal Loss\n- CutMix\n- Label Smoothing\n- Spec Augmentation Time\n- DIfferent learning rate for each layer\n- Multi Sample Dropout\n\n\n### My Code\nhttps://github.com/trtd56/Birdcall\n\n### My Blog(Japanese)\nhttps://www.ai-shift.co.jp/techblog/1271\n\nThank you!!", "votes": null }, { "id": "1012131", "postDate": "09/16/2020 00:08:54", "content": "Thank you for participating in the challenge and for posting your code!
", "rawMarkdown": "Thank you for participating in the challenge and for posting your code!", "votes": null } ], "comments": [ { "id": 1012131, "author_name": "holgerklinck", "author_url": "", "post_date": "09/16/2020 00:08:54", "content": "Thank you for participating in the challenge and for posting your code!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1012127": "Congrats to all winners! And thanks to organizers!\nMy score is not good, but sharing my gotten knowledge.\n\n### modeling\nMy base model is DenseNet201 trained ImageNet.\nMost people used ResNet, but it not work for me.\n\nI read [this paper](https://arxiv.org/pdf/1908.02876.pdf), and apply applied to my model.\n\n\n\ncode is:\n```python\nclass BirdcallNet(nn.Module):\n def __init__(self):\n super(BirdcallNet_densenet, self).__init__()\n densenet = densenet161(pretrained=config.PRETRAINED)\n self.features = densenet.features\n self.l8_a = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n self.l8_b = nn.Conv1d(2208, config.N_LABEL, 1, bias=False)\n\n def forward(self, x, perm=None, gamma=None):\n # input: (batch, channel, Hz, time)\n frames_num = x.shape[3]\n x = x.transpose(3, 2) # (batch, channel, time, Hz)\n h = self.features(x) # (batch, unit, time, Hz)\n\n h = F.relu(h, inplace=True)\n h = torch.mean(h, dim=3) # (batch, unit, time)\n\n xa = self.l8_a(h) # (batch, n_class, time)\n xb = self.l8_b(h) # (batch, n_class, time)\n xb = torch.softmax(xb, dim=2)\n\n pseudo_label = (xa.sigmoid() >= 0.5).float()\n clipwise_preds = torch.sum(xa * xb, dim=2)\n attention_preds = xb\n\n return clipwise_preds, attention_preds, pseudo_label\n```\n\n#### Parameters\n- BCE loss\n- 5-fold CV\n- Adam\n - learning_rate=1e-3\n - 55 epoch\n - batch size=64\n- CosineAnnealingWarmRestarts\n - T=10\n\n### Data Augumentation\n- Adjust Gamma\n- Spec Augmentation Freq\n- MixUp\n\n### Other Technique\n- SWA\n\n### Not Work for me\n- Denoise(I tried my best, but it didn't work ...)\n- \"nocall\" prediction by PANNs trained model\n- Transformer\n- WaveNet\n- use secondary_labels\n- optimize threshold\n- Focal Loss\n- CutMix\n- Label Smoothing\n- Spec Augmentation Time\n- DIfferent learning rate for each layer\n- Multi Sample Dropout\n\n\n### My Code\nhttps://github.com/trtd56/Birdcall\n\n### My Blog(Japanese)\nhttps://www.ai-shift.co.jp/techblog/1271\n\nThank you!!", "1012131": "Thank you for participating in the challenge and for posting your code!" }, "source": "meta" }