{
  "id": 180622,
  "title": "Any success on validation on 2 files in example_test_audio ?",
  "url": "/competitions/birdsong-recognition/discussion/180622",
  "author_name": "",
  "post_date": "2020-09-05T19:15:42.089148300Z",
  "votes": 5,
  "comment_count": 2,
  "views": 0,
  "content": "<p>As noted in this good notebook <a href=\"https://www.kaggle.com/jpison/inference-resnest50-fast-with-example-test-audio\" target=\"_blank\">by Martinez</a> the CV score on the 2 examples in \"example test audio\" is 0.45 for a corresponding LB score of 0.568 with popular great notebook from <a href=\"https://www.kaggle.com/ttahara\" target=\"_blank\">@ttahara</a> </p>\n<p>However, If you look closely at the results in the Martinez notebook, all Resnest model gets right is some \"nocalls\" right and nothing else much really. I trained some effnets in addition to the resnest and got similiar results…but the LB scores are 0.56+</p>\n<p>Has anyone got some success on trying to inference on the \"exampletest audio\" ? <br>\nOf the two, the BLKFR-10-CPL_20190611_093000.pt540.mp3 is really challenging to predict on with very faint bird sound volume and a lot of noise.</p>\n<p>Am I going in the wrong direction in trying to validate on these 2 \"example test audio\" files. My results on these 2 files have really discouraged me and any advice is welcome.</p>",
  "messages": [
    {
      "id": "999591",
      "postDate": "09/05/2020 19:15:42",
      "content": "<p>As noted in this good notebook <a href=\"https://www.kaggle.com/jpison/inference-resnest50-fast-with-example-test-audio\" target=\"_blank\">by Martinez</a> the CV score on the 2 examples in \"example test audio\" is 0.45 for a corresponding LB score of 0.568 with popular great notebook from <a href=\"https://www.kaggle.com/ttahara\" target=\"_blank\">@ttahara</a> </p>\n<p>However, If you look closely at the results in the Martinez notebook, all Resnest model gets right is some \"nocalls\" right and nothing else much really. I trained some effnets in addition to the resnest and got similiar results…but the LB scores are 0.56+</p>\n<p>Has anyone got some success on trying to inference on the \"exampletest audio\" ? <br>\nOf the two, the BLKFR-10-CPL_20190611_093000.pt540.mp3 is really challenging to predict on with very faint bird sound volume and a lot of noise.</p>\n<p>Am I going in the wrong direction in trying to validate on these 2 \"example test audio\" files. My results on these 2 files have really discouraged me and any advice is welcome.</p>",
      "rawMarkdown": "As noted in this good notebook [by Martinez](https://www.kaggle.com/jpison/inference-resnest50-fast-with-example-test-audio) the CV score on the 2 examples in \"example test audio\" is 0.45 for a corresponding LB score of 0.568 with popular great notebook from @ttahara \n\nHowever, If you look closely at the results in the Martinez notebook, all Resnest model gets right is some \"nocalls\" right and nothing else much really. I trained some effnets in addition to the resnest and got similiar results...but the LB scores are 0.56+\n\nHas anyone got some success on trying to inference on the \"exampletest audio\" ? \nOf the two, the BLKFR-10-CPL_20190611_093000.pt540.mp3 is really challenging to predict on with very faint bird sound volume and a lot of noise.\n\nAm I going in the wrong direction in trying to validate on these 2 \"example test audio\" files. My results on these 2 files have really discouraged me and any advice is welcome.",
      "votes": null
    },
    {
      "id": "1000009",
      "postDate": "09/06/2020 07:40:17",
      "content": "<p>Have tried different threshold strategies for each site since in theory hidden test should be 3 different North American locations, possibly different dates/seasons. And BLKFR… is different to ORANGE… Also tried a 2 step threshold to look at most likely species to be there based on first predictions then use lower thresholds for them and/or raise the threshold for the rest.  Best F1 for example was 0.53159 and most that were not nocall were in ORANGE…</p>\n<p>That seems promising sort of. Still working on it…  It would be good to have more soundscape test data with more birds.<br>\nThe sample submission all nocall scores 0.544 so getting nocall right has some value anyway. </p>",
      "rawMarkdown": "Have tried different threshold strategies for each site since in theory hidden test should be 3 different North American locations, possibly different dates/seasons. And BLKFR... is different to ORANGE... Also tried a 2 step threshold to look at most likely species to be there based on first predictions then use lower thresholds for them and/or raise the threshold for the rest.  Best F1 for example was 0.53159 and most that were not nocall were in ORANGE...\n\nThat seems promising sort of. Still working on it…  It would be good to have more soundscape test data with more birds.\nThe sample submission all nocall scores 0.544 so getting nocall right has some value anyway.",
      "votes": null
    },
    {
      "id": "1000188",
      "postDate": "09/06/2020 11:15:14",
      "content": "<blockquote>\n  <p>And BLKFR… is different to ORANGE</p>\n</blockquote>\n<p>that is quite true, BLKFR is much more challenging.</p>",
      "rawMarkdown": "> And BLKFR… is different to ORANGE\n\nthat is quite true, BLKFR is much more challenging.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1000009,
      "author_name": "something4kag",
      "author_url": "",
      "post_date": "09/06/2020 07:40:17",
      "content": "<p>Have tried different threshold strategies for each site since in theory hidden test should be 3 different North American locations, possibly different dates/seasons. And BLKFR… is different to ORANGE… Also tried a 2 step threshold to look at most likely species to be there based on first predictions then use lower thresholds for them and/or raise the threshold for the rest.  Best F1 for example was 0.53159 and most that were not nocall were in ORANGE…</p>\n<p>That seems promising sort of. Still working on it…  It would be good to have more soundscape test data with more birds.<br>\nThe sample submission all nocall scores 0.544 so getting nocall right has some value anyway. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1000188,
          "author_name": "watzisname",
          "author_url": "",
          "post_date": "09/06/2020 11:15:14",
          "content": "<blockquote>\n  <p>And BLKFR… is different to ORANGE</p>\n</blockquote>\n<p>that is quite true, BLKFR is much more challenging.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "999591": "As noted in this good notebook [by Martinez](https://www.kaggle.com/jpison/inference-resnest50-fast-with-example-test-audio) the CV score on the 2 examples in \"example test audio\" is 0.45 for a corresponding LB score of 0.568 with popular great notebook from @ttahara \n\nHowever, If you look closely at the results in the Martinez notebook, all Resnest model gets right is some \"nocalls\" right and nothing else much really. I trained some effnets in addition to the resnest and got similiar results...but the LB scores are 0.56+\n\nHas anyone got some success on trying to inference on the \"exampletest audio\" ? \nOf the two, the BLKFR-10-CPL_20190611_093000.pt540.mp3 is really challenging to predict on with very faint bird sound volume and a lot of noise.\n\nAm I going in the wrong direction in trying to validate on these 2 \"example test audio\" files. My results on these 2 files have really discouraged me and any advice is welcome.",
    "1000009": "Have tried different threshold strategies for each site since in theory hidden test should be 3 different North American locations, possibly different dates/seasons. And BLKFR... is different to ORANGE... Also tried a 2 step threshold to look at most likely species to be there based on first predictions then use lower thresholds for them and/or raise the threshold for the rest.  Best F1 for example was 0.53159 and most that were not nocall were in ORANGE...\n\nThat seems promising sort of. Still working on it…  It would be good to have more soundscape test data with more birds.\nThe sample submission all nocall scores 0.544 so getting nocall right has some value anyway.",
    "1000188": "> And BLKFR… is different to ORANGE\n\nthat is quite true, BLKFR is much more challenging."
  },
  "source": "meta"
}