{
  "id": 198132,
  "title": "Best Single Model CV-LB",
  "url": "/competitions/rfcx-species-audio-detection/discussion/198132",
  "author_name": "",
  "post_date": "2020-11-19T22:26:41.029795Z",
  "votes": 46,
  "comment_count": 31,
  "views": 0,
  "content": "<p>Starting common topic =)</p>\n<pre><code>model: resnet34\naugmentation: No\nduration: trainied with 60 sec clip\nchannel: 1\ndata: only TP\nepoch: 10 (best: 6)\nsplit: 80/20\nfold: 1\n\nCV - 0.576\nLB - 0.546\n</code></pre>",
  "messages": [
    {
      "id": "1084287",
      "postDate": "11/19/2020 22:26:41",
      "content": "<p>Starting common topic =)</p>\n<pre><code>model: resnet34\naugmentation: No\nduration: trainied with 60 sec clip\nchannel: 1\ndata: only TP\nepoch: 10 (best: 6)\nsplit: 80/20\nfold: 1\n\nCV - 0.576\nLB - 0.546\n</code></pre>",
      "rawMarkdown": "Starting common topic =)\n\n```\nmodel: resnet34\naugmentation: No\nduration: trainied with 60 sec clip\nchannel: 1\ndata: only TP\nepoch: 10 (best: 6)\nsplit: 80/20\nfold: 1\n\nCV - 0.576\nLB - 0.546\n```",
      "votes": null
    },
    {
      "id": "1085520",
      "postDate": "11/21/2020 00:19:48",
      "content": "<pre><code>model: resnet34\naugmentation: MixUP, TimeDrop\nduration: 60 sec \nchannel: 3\ndata: only TP\nepoch: 15\nsplit: 80/20\nfold: 0\n\nCV - 0.64\nLB - 0.617\n</code></pre>",
      "rawMarkdown": "```\nmodel: resnet34\naugmentation: MixUP, TimeDrop\nduration: 60 sec \nchannel: 3\ndata: only TP\nepoch: 15\nsplit: 80/20\nfold: 0\n\nCV - 0.64\nLB - 0.617\n```",
      "votes": null
    },
    {
      "id": "1088421",
      "postDate": "11/23/2020 15:43:42",
      "content": "<p>Finally mange to submit individual folds just to see how well CV-LB is correlated</p>\n<pre><code>FOLD 1 - CV: 0.74601 : LB: 0.68\nFOLD 2- CV: 0.74068 : LB: 0.69\nFOLD 3- CV: 0.74906 : LB: 0.70\nFOLD 3- CV: 0.72723 : LB: 0.71\nFOLD 4- CV: 0.72787 : LB: 0.67\n\nSimple Average: of all folds :  LB: 0.716\n</code></pre>\n<p>does anybody see similar trends ?</p>",
      "rawMarkdown": "Finally mange to submit individual folds just to see how well CV-LB is correlated\n```\nFOLD 1 - CV: 0.74601 : LB: 0.68\nFOLD 2- CV: 0.74068 : LB: 0.69\nFOLD 3- CV: 0.74906 : LB: 0.70\nFOLD 3- CV: 0.72723 : LB: 0.71\nFOLD 4- CV: 0.72787 : LB: 0.67\n\nSimple Average: of all folds :  LB: 0.716\n```\n\ndoes anybody see similar trends ?",
      "votes": null
    },
    {
      "id": "1089464",
      "postDate": "11/24/2020 14:26:01",
      "content": "<pre><code>FOLD 0 - CV: 0.63 : LB: 0.625\nFOLD 1 - CV: 0.70: LB: 0.69\nSimple Average: of all folds :  LB: 0.695\n</code></pre>",
      "rawMarkdown": "```\nFOLD 0 - CV: 0.63 : LB: 0.625\nFOLD 1 - CV: 0.70: LB: 0.69\nSimple Average: of all folds :  LB: 0.695\n```",
      "votes": null
    },
    {
      "id": "1090760",
      "postDate": "11/25/2020 15:11:27",
      "content": "<p>model: PANNs (Cnn14_DecisionLevelAtt)<br>\ndata: only TP<br>\nepoch: 30<br>\nsplit: 80/20<br>\nfold: 0</p>\n<p>CV: 0.82771<br>\nLB: 0.710</p>",
      "rawMarkdown": "model: PANNs (Cnn14_DecisionLevelAtt)\ndata: only TP\nepoch: 30\nsplit: 80/20\nfold: 0\n\nCV: 0.82771\nLB: 0.710",
      "votes": null
    },
    {
      "id": "1091278",
      "postDate": "11/25/2020 22:55:36",
      "content": "<p>Changed the loss<br>\nCV: 0.8243<br>\nLB: 0.771<br>\n[Update] I forgot to note that I also changed the pretrained weight.</p>",
      "rawMarkdown": "Changed the loss\nCV: 0.8243\nLB: 0.771\n[Update] I forgot to note that I also changed the pretrained weight.",
      "votes": null
    },
    {
      "id": "1091435",
      "postDate": "11/26/2020 02:50:38",
      "content": "<p>model: ResNest50<br>\naugmentation: Waveform augs from audiomentations + SpecAugment<br>\ndata: only TP<br>\nepochs: 30, scheduler: warmup cosine<br>\n5-fold cv: 0.8108<br>\nlb (5 folds averaging): 0.798</p>",
      "rawMarkdown": "model: ResNest50\naugmentation: Waveform augs from audiomentations + SpecAugment\ndata: only TP\nepochs: 30, scheduler: warmup cosine\n5-fold cv: 0.8108\nlb (5 folds averaging): 0.798",
      "votes": null
    },
    {
      "id": "1091437",
      "postDate": "11/26/2020 02:52:43",
      "content": "<p>Did you compute loss for the segment-wise output ? Or just the clip-wise output ?</p>",
      "rawMarkdown": "Did you compute loss for the segment-wise output ? Or just the clip-wise output ?",
      "votes": null
    },
    {
      "id": "1091458",
      "postDate": "11/26/2020 03:24:27",
      "content": "<p>👍 Merge team try hard thôi bạn ơi 🔥</p>",
      "rawMarkdown": "👍 Merge team try hard thôi bạn ơi 🔥",
      "votes": null
    },
    {
      "id": "1091463",
      "postDate": "11/26/2020 03:31:49",
      "content": "<p>Both. I did the same in Birdcall.</p>",
      "rawMarkdown": "Both. I did the same in Birdcall.",
      "votes": null
    },
    {
      "id": "1092486",
      "postDate": "11/26/2020 21:49:47",
      "content": "<p>ResNest50<br>\nCV: 0.857<br>\nLB: 0.831</p>\n<p>so far I think LB and CV is roughly correlated</p>",
      "rawMarkdown": "ResNest50\nCV: 0.857\nLB: 0.831\n\nso far I think LB and CV is roughly correlated",
      "votes": null
    },
    {
      "id": "1092672",
      "postDate": "11/27/2020 04:45:19",
      "content": "<p>PANNs 5fold <br>\nFold 0: 0.824<br>\nFold 1: 0.826<br>\nFold2: 0.844<br>\nFold3: 0.860<br>\nFold4: 0.852</p>\n<p>LB: 0.791</p>",
      "rawMarkdown": "PANNs 5fold \nFold 0: 0.824\nFold 1: 0.826\nFold2: 0.844\nFold3: 0.860\nFold4: 0.852\n\nLB: 0.791",
      "votes": null
    },
    {
      "id": "1094234",
      "postDate": "11/28/2020 12:34:26",
      "content": "<p>5fold ResNest50 + Loss modification<br>\nFold0: 0.857<br>\nFold1: 0.886<br>\nFold2: 0.881<br>\nFold3: 0.902<br>\nFold4: 0.872<br>\nLB: 0.873</p>",
      "rawMarkdown": "5fold ResNest50 + Loss modification\nFold0: 0.857\nFold1: 0.886\nFold2: 0.881\nFold3: 0.902\nFold4: 0.872\nLB: 0.873",
      "votes": null
    },
    {
      "id": "1094890",
      "postDate": "11/29/2020 04:11:36",
      "content": "<ul>\n<li>model: PANNs</li>\n<li>augmentation: Yes</li>\n<li>data; only TP</li>\n<li>split: 80/20</li>\n<li>fold: 5</li>\n<li>epoch: 15-25</li>\n</ul>\n<p>CV (5fold average): 0.8120<br>\nLB: 0.760<br>\n😃</p>",
      "rawMarkdown": "model: PANNs\n- augmentation: Yes\n- data; only TP\n- split: 80/20\n- fold: 5\n- epoch: 15-25\n\nCV (5fold average): 0.8120\nLB: 0.760\n😃",
      "votes": null
    },
    {
      "id": "1095227",
      "postDate": "11/29/2020 11:30:46",
      "content": "<p>how many seconds you use in dataset?</p>",
      "rawMarkdown": "how many seconds you use in dataset?",
      "votes": null
    },
    {
      "id": "1095233",
      "postDate": "11/29/2020 11:41:36",
      "content": "<p>10sec random crop :)</p>",
      "rawMarkdown": "10sec random crop :)",
      "votes": null
    },
    {
      "id": "1097713",
      "postDate": "12/01/2020 07:46:37",
      "content": "<p>model: ResNest50<br>\naugmentation: None<br>\ndata: only TP<br>\nsplit: 80/20<br>\nepochs: 30<br>\nSingle Fold0: 0.776<br>\nLB: 0.781</p>",
      "rawMarkdown": "model: ResNest50\naugmentation: None\ndata: only TP\nsplit: 80/20\nepochs: 30\nSingle Fold0: 0.776\nLB: 0.781",
      "votes": null
    },
    {
      "id": "1100833",
      "postDate": "12/03/2020 11:48:59",
      "content": "<p>Model: Resnest50<br>\nAugmentation: Noise<br>\nData: TP <br>\nEpochs: 30<br>\nCV 5 folds: [0.806, 0.819, 0.807, 0.841, 0.831]<br>\nLB: 0.828</p>",
      "rawMarkdown": "Model: Resnest50\nAugmentation: Noise\nData: TP \nEpochs: 30\nCV 5 folds: [0.806, 0.819, 0.807, 0.841, 0.831]\nLB: 0.828",
      "votes": null
    },
    {
      "id": "1100925",
      "postDate": "12/03/2020 13:24:16",
      "content": "<p>i use 10s random crop like this post <a href=\"https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200922\" target=\"_blank\">https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200922</a>. CV is 0.84 but on LB only 0.64 . Maybe some error here?</p>",
      "rawMarkdown": "i use 10s random crop like this post https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200922. CV is 0.84 but on LB only 0.64 . Maybe some error here?",
      "votes": null
    },
    {
      "id": "1100991",
      "postDate": "12/03/2020 14:34:56",
      "content": "<p>are you using spectrogram or raw audio as input?</p>",
      "rawMarkdown": "are you using spectrogram or raw audio as input?",
      "votes": null
    },
    {
      "id": "1100999",
      "postDate": "12/03/2020 14:41:07",
      "content": "<p>I use mel spectrogram </p>",
      "rawMarkdown": "I use mel spectrogram",
      "votes": null
    },
    {
      "id": "1101848",
      "postDate": "12/04/2020 10:21:44",
      "content": "<p>Seem almost people here are reported the <strong>Kfold Average Mini Batch</strong> CV rather than <strong>Kfold Overral Concat</strong> CV :D</p>",
      "rawMarkdown": "Seem almost people here are reported the **Kfold Average Mini Batch** CV rather than **Kfold Overral Concat** CV :D",
      "votes": null
    },
    {
      "id": "1102437",
      "postDate": "12/04/2020 23:32:20",
      "content": "<p>I don't see any error in the cropping but in other part there might be</p>",
      "rawMarkdown": "I don't see any error in the cropping but in other part there might be",
      "votes": null
    },
    {
      "id": "1105204",
      "postDate": "12/07/2020 16:08:34",
      "content": "<p>Thanks for the reply! I tried resnest50 with noise augmentation but can only get around 0.68 accuracy. What kind of noise are you using?</p>",
      "rawMarkdown": "Thanks for the reply! I tried resnest50 with noise augmentation but can only get around 0.68 accuracy. What kind of noise are you using?",
      "votes": null
    },
    {
      "id": "1105287",
      "postDate": "12/07/2020 18:17:46",
      "content": "<p>Model: ResNet50<br>\nAugmentation: Gaussian Noise<br>\nData: TP<br>\nEpochs: 20<br>\nCV 5 folds: [0.847, 0.801, 0.857, 0.84, 0.841]<br>\nLB: 0.798</p>",
      "rawMarkdown": "Model: ResNet50\nAugmentation: Gaussian Noise\nData: TP\nEpochs: 20\nCV 5 folds: [0.847, 0.801, 0.857, 0.84, 0.841]\nLB: 0.798",
      "votes": null
    },
    {
      "id": "1107375",
      "postDate": "12/09/2020 16:28:51",
      "content": "<p>Do you use the ResNet50 SED or not?</p>",
      "rawMarkdown": "Do you use the ResNet50 SED or not?",
      "votes": null
    },
    {
      "id": "1107631",
      "postDate": "12/09/2020 20:33:12",
      "content": "<blockquote>\n  <p>5fold ResNest50 + Loss modification<br>\n  Fold0: 0.857<br>\n  Fold1: 0.886<br>\n  Fold2: 0.881<br>\n  Fold3: 0.902<br>\n  Fold4: 0.872<br>\n  LB: 0.873</p>\n</blockquote>\n<p>Hey Hidehisa Arai, </p>\n<p>do you get these LWLRAP numbers on the 10 sec validation samples, or on the full clip validation samples? </p>\n<p>Thank you!</p>",
      "rawMarkdown": "> 5fold ResNest50 + Loss modification\nFold0: 0.857\nFold1: 0.886\nFold2: 0.881\nFold3: 0.902\nFold4: 0.872\nLB: 0.873\n\n\n\nHey Hidehisa Arai, \n\ndo you get these LWLRAP numbers on the 10 sec validation samples, or on the full clip validation samples? \n\nThank you!",
      "votes": null
    },
    {
      "id": "1107859",
      "postDate": "12/10/2020 02:44:00",
      "content": "<p><strong>Framework</strong>: PyTorch<br>\n<strong>Model</strong>: Custom with ResNet-18 backbone (-&gt; I have to use backbone w more capacity)<br>\n<strong>Augmentation</strong>: Mixup (-&gt; I have to try specaugment and others)<br>\n<strong>Data</strong>: only TP, log-mel spectrum<br>\n<strong>Epochs</strong>: 100<br>\n<strong>Data split</strong>: Stratified 5-Folds<br>\n<strong>Loss</strong>: Custom Loss<br>\n<strong>Optimizer</strong>: Adam<br>\n<strong>LR Scheduler</strong>: MultiStepLR<br>\n<strong>LB</strong>: 0.858</p>",
      "rawMarkdown": "**Framework**: PyTorch\n**Model**: Custom with ResNet-18 backbone (-> I have to use backbone w more capacity)\n**Augmentation**: Mixup (-> I have to try specaugment and others)\n**Data**: only TP, log-mel spectrum\n**Epochs**: 100\n**Data split**: Stratified 5-Folds\n**Loss**: Custom Loss\n**Optimizer**: Adam\n**LR Scheduler**: MultiStepLR\n**LB**: 0.858",
      "votes": null
    },
    {
      "id": "1107865",
      "postDate": "12/10/2020 02:54:16",
      "content": "<p>This is with 10 sec validation samples. I moved to the full clip validation now, but have not checked the validation score in the same setting.</p>",
      "rawMarkdown": "This is with 10 sec validation samples. I moved to the full clip validation now, but have not checked the validation score in the same setting.",
      "votes": null
    },
    {
      "id": "1126779",
      "postDate": "12/25/2020 23:34:28",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/guglielmocamporese\" target=\"_blank\">@guglielmocamporese</a> Any pointers on what we could look into to implement a custom loss?</p>",
      "rawMarkdown": "Hi @guglielmocamporese Any pointers on what we could look into to implement a custom loss?",
      "votes": null
    },
    {
      "id": "1131631",
      "postDate": "12/29/2020 21:29:00",
      "content": "<p>Dear Arai - Can you give any direction/pointers on optional loss modifications? I'm using ResNet50, into globalavgpooling2d, into dense layer with bcewithlogits. For 5-fold this gives a cv ~0.85 with lb ~0.83. Are you using a model architecture (resnet + attention) and loss similar to birdcall?</p>",
      "rawMarkdown": "Dear Arai - Can you give any direction/pointers on optional loss modifications? I'm using ResNet50, into globalavgpooling2d, into dense layer with bcewithlogits. For 5-fold this gives a cv ~0.85 with lb ~0.83. Are you using a model architecture (resnet + attention) and loss similar to birdcall?",
      "votes": null
    },
    {
      "id": "1136090",
      "postDate": "01/02/2021 18:08:15",
      "content": "<p>Sorry, but I don't see any 'trend' between CV and LB above. Do you have any idea what is happening there?</p>",
      "rawMarkdown": "Sorry, but I don't see any 'trend' between CV and LB above. Do you have any idea what is happening there?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1085520,
      "author_name": "drhabib",
      "author_url": "",
      "post_date": "11/21/2020 00:19:48",
      "content": "<pre><code>model: resnet34\naugmentation: MixUP, TimeDrop\nduration: 60 sec \nchannel: 3\ndata: only TP\nepoch: 15\nsplit: 80/20\nfold: 0\n\nCV - 0.64\nLB - 0.617\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1088421,
      "author_name": "drhabib",
      "author_url": "",
      "post_date": "11/23/2020 15:43:42",
      "content": "<p>Finally mange to submit individual folds just to see how well CV-LB is correlated</p>\n<pre><code>FOLD 1 - CV: 0.74601 : LB: 0.68\nFOLD 2- CV: 0.74068 : LB: 0.69\nFOLD 3- CV: 0.74906 : LB: 0.70\nFOLD 3- CV: 0.72723 : LB: 0.71\nFOLD 4- CV: 0.72787 : LB: 0.67\n\nSimple Average: of all folds :  LB: 0.716\n</code></pre>\n<p>does anybody see similar trends ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1136090,
          "author_name": "prvnkmr",
          "author_url": "",
          "post_date": "01/02/2021 18:08:15",
          "content": "<p>Sorry, but I don't see any 'trend' between CV and LB above. Do you have any idea what is happening there?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1089464,
      "author_name": "doanquanvietnamca",
      "author_url": "",
      "post_date": "11/24/2020 14:26:01",
      "content": "<pre><code>FOLD 0 - CV: 0.63 : LB: 0.625\nFOLD 1 - CV: 0.70: LB: 0.69\nSimple Average: of all folds :  LB: 0.695\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1090760,
      "author_name": "hidehisaarai1213",
      "author_url": "",
      "post_date": "11/25/2020 15:11:27",
      "content": "<p>model: PANNs (Cnn14_DecisionLevelAtt)<br>\ndata: only TP<br>\nepoch: 30<br>\nsplit: 80/20<br>\nfold: 0</p>\n<p>CV: 0.82771<br>\nLB: 0.710</p>",
      "votes": null,
      "replies": [
        {
          "id": 1091278,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "11/25/2020 22:55:36",
          "content": "<p>Changed the loss<br>\nCV: 0.8243<br>\nLB: 0.771<br>\n[Update] I forgot to note that I also changed the pretrained weight.</p>",
          "votes": null,
          "replies": [
            {
              "id": 1092672,
              "author_name": "hidehisaarai1213",
              "author_url": "",
              "post_date": "11/27/2020 04:45:19",
              "content": "<p>PANNs 5fold <br>\nFold 0: 0.824<br>\nFold 1: 0.826<br>\nFold2: 0.844<br>\nFold3: 0.860<br>\nFold4: 0.852</p>\n<p>LB: 0.791</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 1091437,
          "author_name": "andy2709",
          "author_url": "",
          "post_date": "11/26/2020 02:52:43",
          "content": "<p>Did you compute loss for the segment-wise output ? Or just the clip-wise output ?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1091463,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "11/26/2020 03:31:49",
          "content": "<p>Both. I did the same in Birdcall.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1092486,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "11/26/2020 21:49:47",
          "content": "<p>ResNest50<br>\nCV: 0.857<br>\nLB: 0.831</p>\n<p>so far I think LB and CV is roughly correlated</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1094234,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "11/28/2020 12:34:26",
          "content": "<p>5fold ResNest50 + Loss modification<br>\nFold0: 0.857<br>\nFold1: 0.886<br>\nFold2: 0.881<br>\nFold3: 0.902<br>\nFold4: 0.872<br>\nLB: 0.873</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1095227,
          "author_name": "doanquanvietnamca",
          "author_url": "",
          "post_date": "11/29/2020 11:30:46",
          "content": "<p>how many seconds you use in dataset?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1095233,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "11/29/2020 11:41:36",
          "content": "<p>10sec random crop :)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1100925,
          "author_name": "doanquanvietnamca",
          "author_url": "",
          "post_date": "12/03/2020 13:24:16",
          "content": "<p>i use 10s random crop like this post <a href=\"https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200922\" target=\"_blank\">https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200922</a>. CV is 0.84 but on LB only 0.64 . Maybe some error here?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1102437,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "12/04/2020 23:32:20",
          "content": "<p>I don't see any error in the cropping but in other part there might be</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1107631,
          "author_name": "guglielmocamporese",
          "author_url": "",
          "post_date": "12/09/2020 20:33:12",
          "content": "<blockquote>\n  <p>5fold ResNest50 + Loss modification<br>\n  Fold0: 0.857<br>\n  Fold1: 0.886<br>\n  Fold2: 0.881<br>\n  Fold3: 0.902<br>\n  Fold4: 0.872<br>\n  LB: 0.873</p>\n</blockquote>\n<p>Hey Hidehisa Arai, </p>\n<p>do you get these LWLRAP numbers on the 10 sec validation samples, or on the full clip validation samples? </p>\n<p>Thank you!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1107865,
          "author_name": "hidehisaarai1213",
          "author_url": "",
          "post_date": "12/10/2020 02:54:16",
          "content": "<p>This is with 10 sec validation samples. I moved to the full clip validation now, but have not checked the validation score in the same setting.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1131631,
          "author_name": "tdunlap607",
          "author_url": "",
          "post_date": "12/29/2020 21:29:00",
          "content": "<p>Dear Arai - Can you give any direction/pointers on optional loss modifications? I'm using ResNet50, into globalavgpooling2d, into dense layer with bcewithlogits. For 5-fold this gives a cv ~0.85 with lb ~0.83. Are you using a model architecture (resnet + attention) and loss similar to birdcall?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1091435,
      "author_name": "andy2709",
      "author_url": "",
      "post_date": "11/26/2020 02:50:38",
      "content": "<p>model: ResNest50<br>\naugmentation: Waveform augs from audiomentations + SpecAugment<br>\ndata: only TP<br>\nepochs: 30, scheduler: warmup cosine<br>\n5-fold cv: 0.8108<br>\nlb (5 folds averaging): 0.798</p>",
      "votes": null,
      "replies": [
        {
          "id": 1091458,
          "author_name": "truonghoang",
          "author_url": "",
          "post_date": "11/26/2020 03:24:27",
          "content": "<p>👍 Merge team try hard thôi bạn ơi 🔥</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1094890,
      "author_name": "daigohirooka",
      "author_url": "",
      "post_date": "11/29/2020 04:11:36",
      "content": "<ul>\n<li>model: PANNs</li>\n<li>augmentation: Yes</li>\n<li>data; only TP</li>\n<li>split: 80/20</li>\n<li>fold: 5</li>\n<li>epoch: 15-25</li>\n</ul>\n<p>CV (5fold average): 0.8120<br>\nLB: 0.760<br>\n😃</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1097713,
      "author_name": "duykhanh99",
      "author_url": "",
      "post_date": "12/01/2020 07:46:37",
      "content": "<p>model: ResNest50<br>\naugmentation: None<br>\ndata: only TP<br>\nsplit: 80/20<br>\nepochs: 30<br>\nSingle Fold0: 0.776<br>\nLB: 0.781</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1100833,
      "author_name": "nvnnghia",
      "author_url": "",
      "post_date": "12/03/2020 11:48:59",
      "content": "<p>Model: Resnest50<br>\nAugmentation: Noise<br>\nData: TP <br>\nEpochs: 30<br>\nCV 5 folds: [0.806, 0.819, 0.807, 0.841, 0.831]<br>\nLB: 0.828</p>",
      "votes": null,
      "replies": [
        {
          "id": 1100991,
          "author_name": "",
          "author_url": "",
          "post_date": "12/03/2020 14:34:56",
          "content": "<p>are you using spectrogram or raw audio as input?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1100999,
          "author_name": "nvnnghia",
          "author_url": "",
          "post_date": "12/03/2020 14:41:07",
          "content": "<p>I use mel spectrogram </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1105204,
          "author_name": "",
          "author_url": "",
          "post_date": "12/07/2020 16:08:34",
          "content": "<p>Thanks for the reply! I tried resnest50 with noise augmentation but can only get around 0.68 accuracy. What kind of noise are you using?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1101848,
      "author_name": "dathudeptrai",
      "author_url": "",
      "post_date": "12/04/2020 10:21:44",
      "content": "<p>Seem almost people here are reported the <strong>Kfold Average Mini Batch</strong> CV rather than <strong>Kfold Overral Concat</strong> CV :D</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1105287,
      "author_name": "nikhiljohnk",
      "author_url": "",
      "post_date": "12/07/2020 18:17:46",
      "content": "<p>Model: ResNet50<br>\nAugmentation: Gaussian Noise<br>\nData: TP<br>\nEpochs: 20<br>\nCV 5 folds: [0.847, 0.801, 0.857, 0.84, 0.841]<br>\nLB: 0.798</p>",
      "votes": null,
      "replies": [
        {
          "id": 1107375,
          "author_name": "truonghoang",
          "author_url": "",
          "post_date": "12/09/2020 16:28:51",
          "content": "<p>Do you use the ResNet50 SED or not?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1107859,
      "author_name": "guglielmocamporese",
      "author_url": "",
      "post_date": "12/10/2020 02:44:00",
      "content": "<p><strong>Framework</strong>: PyTorch<br>\n<strong>Model</strong>: Custom with ResNet-18 backbone (-&gt; I have to use backbone w more capacity)<br>\n<strong>Augmentation</strong>: Mixup (-&gt; I have to try specaugment and others)<br>\n<strong>Data</strong>: only TP, log-mel spectrum<br>\n<strong>Epochs</strong>: 100<br>\n<strong>Data split</strong>: Stratified 5-Folds<br>\n<strong>Loss</strong>: Custom Loss<br>\n<strong>Optimizer</strong>: Adam<br>\n<strong>LR Scheduler</strong>: MultiStepLR<br>\n<strong>LB</strong>: 0.858</p>",
      "votes": null,
      "replies": [
        {
          "id": 1126779,
          "author_name": "saurabh7",
          "author_url": "",
          "post_date": "12/25/2020 23:34:28",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/guglielmocamporese\" target=\"_blank\">@guglielmocamporese</a> Any pointers on what we could look into to implement a custom loss?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1084287": "Starting common topic =)\n\n```\nmodel: resnet34\naugmentation: No\nduration: trainied with 60 sec clip\nchannel: 1\ndata: only TP\nepoch: 10 (best: 6)\nsplit: 80/20\nfold: 1\n\nCV - 0.576\nLB - 0.546\n```",
    "1085520": "```\nmodel: resnet34\naugmentation: MixUP, TimeDrop\nduration: 60 sec \nchannel: 3\ndata: only TP\nepoch: 15\nsplit: 80/20\nfold: 0\n\nCV - 0.64\nLB - 0.617\n```",
    "1088421": "Finally mange to submit individual folds just to see how well CV-LB is correlated\n```\nFOLD 1 - CV: 0.74601 : LB: 0.68\nFOLD 2- CV: 0.74068 : LB: 0.69\nFOLD 3- CV: 0.74906 : LB: 0.70\nFOLD 3- CV: 0.72723 : LB: 0.71\nFOLD 4- CV: 0.72787 : LB: 0.67\n\nSimple Average: of all folds :  LB: 0.716\n```\n\ndoes anybody see similar trends ?",
    "1089464": "```\nFOLD 0 - CV: 0.63 : LB: 0.625\nFOLD 1 - CV: 0.70: LB: 0.69\nSimple Average: of all folds :  LB: 0.695\n```",
    "1090760": "model: PANNs (Cnn14_DecisionLevelAtt)\ndata: only TP\nepoch: 30\nsplit: 80/20\nfold: 0\n\nCV: 0.82771\nLB: 0.710",
    "1091278": "Changed the loss\nCV: 0.8243\nLB: 0.771\n[Update] I forgot to note that I also changed the pretrained weight.",
    "1091435": "model: ResNest50\naugmentation: Waveform augs from audiomentations + SpecAugment\ndata: only TP\nepochs: 30, scheduler: warmup cosine\n5-fold cv: 0.8108\nlb (5 folds averaging): 0.798",
    "1091437": "Did you compute loss for the segment-wise output ? Or just the clip-wise output ?",
    "1091458": "👍 Merge team try hard thôi bạn ơi 🔥",
    "1091463": "Both. I did the same in Birdcall.",
    "1092486": "ResNest50\nCV: 0.857\nLB: 0.831\n\nso far I think LB and CV is roughly correlated",
    "1092672": "PANNs 5fold \nFold 0: 0.824\nFold 1: 0.826\nFold2: 0.844\nFold3: 0.860\nFold4: 0.852\n\nLB: 0.791",
    "1094234": "5fold ResNest50 + Loss modification\nFold0: 0.857\nFold1: 0.886\nFold2: 0.881\nFold3: 0.902\nFold4: 0.872\nLB: 0.873",
    "1094890": "model: PANNs\n- augmentation: Yes\n- data; only TP\n- split: 80/20\n- fold: 5\n- epoch: 15-25\n\nCV (5fold average): 0.8120\nLB: 0.760\n😃",
    "1095227": "how many seconds you use in dataset?",
    "1095233": "10sec random crop :)",
    "1097713": "model: ResNest50\naugmentation: None\ndata: only TP\nsplit: 80/20\nepochs: 30\nSingle Fold0: 0.776\nLB: 0.781",
    "1100833": "Model: Resnest50\nAugmentation: Noise\nData: TP \nEpochs: 30\nCV 5 folds: [0.806, 0.819, 0.807, 0.841, 0.831]\nLB: 0.828",
    "1100925": "i use 10s random crop like this post https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/200922. CV is 0.84 but on LB only 0.64 . Maybe some error here?",
    "1100991": "are you using spectrogram or raw audio as input?",
    "1100999": "I use mel spectrogram",
    "1101848": "Seem almost people here are reported the **Kfold Average Mini Batch** CV rather than **Kfold Overral Concat** CV :D",
    "1102437": "I don't see any error in the cropping but in other part there might be",
    "1105204": "Thanks for the reply! I tried resnest50 with noise augmentation but can only get around 0.68 accuracy. What kind of noise are you using?",
    "1105287": "Model: ResNet50\nAugmentation: Gaussian Noise\nData: TP\nEpochs: 20\nCV 5 folds: [0.847, 0.801, 0.857, 0.84, 0.841]\nLB: 0.798",
    "1107375": "Do you use the ResNet50 SED or not?",
    "1107631": "> 5fold ResNest50 + Loss modification\nFold0: 0.857\nFold1: 0.886\nFold2: 0.881\nFold3: 0.902\nFold4: 0.872\nLB: 0.873\n\n\n\nHey Hidehisa Arai, \n\ndo you get these LWLRAP numbers on the 10 sec validation samples, or on the full clip validation samples? \n\nThank you!",
    "1107859": "**Framework**: PyTorch\n**Model**: Custom with ResNet-18 backbone (-> I have to use backbone w more capacity)\n**Augmentation**: Mixup (-> I have to try specaugment and others)\n**Data**: only TP, log-mel spectrum\n**Epochs**: 100\n**Data split**: Stratified 5-Folds\n**Loss**: Custom Loss\n**Optimizer**: Adam\n**LR Scheduler**: MultiStepLR\n**LB**: 0.858",
    "1107865": "This is with 10 sec validation samples. I moved to the full clip validation now, but have not checked the validation score in the same setting.",
    "1126779": "Hi @guglielmocamporese Any pointers on what we could look into to implement a custom loss?",
    "1131631": "Dear Arai - Can you give any direction/pointers on optional loss modifications? I'm using ResNet50, into globalavgpooling2d, into dense layer with bcewithlogits. For 5-fold this gives a cv ~0.85 with lb ~0.83. Are you using a model architecture (resnet + attention) and loss similar to birdcall?",
    "1136090": "Sorry, but I don't see any 'trend' between CV and LB above. Do you have any idea what is happening there?"
  },
  "source": "meta"
}