{
  "id": 197910,
  "title": "Shouldn't this be a \"Code Competition\" ?",
  "url": "/competitions/rfcx-species-audio-detection/discussion/197910",
  "author_name": "",
  "post_date": "2020-11-18T17:50:57.965525Z",
  "votes": 16,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Code Competitions require the inference to be made in the kaggle kernels, in a limited time.</p>\n<p>Previous audio competitions were code competitions, and I don't really see why this one should not be one as well. Code competitions force participants to build better models instead of bigger ensembles, which is usually a good thing, especially when the solution is going to be put in production later on.</p>\n<p>Any thoughts ? </p>",
  "messages": [
    {
      "id": "1083203",
      "postDate": "11/18/2020 17:50:57",
      "content": "<p>Code Competitions require the inference to be made in the kaggle kernels, in a limited time.</p>\n<p>Previous audio competitions were code competitions, and I don't really see why this one should not be one as well. Code competitions force participants to build better models instead of bigger ensembles, which is usually a good thing, especially when the solution is going to be put in production later on.</p>\n<p>Any thoughts ? </p>",
      "rawMarkdown": "Code Competitions require the inference to be made in the kaggle kernels, in a limited time.\n\nPrevious audio competitions were code competitions, and I don't really see why this one should not be one as well. Code competitions force participants to build better models instead of bigger ensembles, which is usually a good thing, especially when the solution is going to be put in production later on.\n\nAny thoughts ?",
      "votes": null
    },
    {
      "id": "1083217",
      "postDate": "11/18/2020 18:09:15",
      "content": "<p>I agree with the pros. However there are cons as well. </p>\n<p>The cornell comp was quite a tough nut to crack. Hidden and completely different test set was the main reason but the not so obvious inference submissions caused quite a lot of headaches as well.</p>\n<p>The kaggle kernel env also has stability issues. Its a living thing with version upgrades fixes during competiton. The error messages also more transparent for the standard csv upload.  </p>",
      "rawMarkdown": "I agree with the pros. However there are cons as well. \n\nThe cornell comp was quite a tough nut to crack. Hidden and completely different test set was the main reason but the not so obvious inference submissions caused quite a lot of headaches as well.\n\nThe kaggle kernel env also has stability issues. Its a living thing with version upgrades fixes during competiton. The error messages also more transparent for the standard csv upload.",
      "votes": null
    },
    {
      "id": "1083223",
      "postDate": "11/18/2020 18:15:27",
      "content": "<p>Most importantly, the test set is publicly available despite it's small size . Hence, one can just walk through it and listen to the records and manually label them accordingly. It may be a tough task though !</p>",
      "rawMarkdown": "Most importantly, the test set is publicly available despite it's small size . Hence, one can just walk through it and listen to the records and manually label them accordingly. It may be a tough task though !",
      "votes": null
    },
    {
      "id": "1083318",
      "postDate": "11/18/2020 20:56:56",
      "content": "<p>I responded with some thoughts here:</p>\n<p><a href=\"https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197923#1083317\" target=\"_blank\">https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197923#1083317</a></p>",
      "rawMarkdown": "I responded with some thoughts here:\n\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197923#1083317",
      "votes": null
    },
    {
      "id": "1083677",
      "postDate": "11/19/2020 08:30:10",
      "content": "<p>Thanks a lot ! </p>",
      "rawMarkdown": "Thanks a lot !",
      "votes": null
    },
    {
      "id": "1083678",
      "postDate": "11/19/2020 08:31:32",
      "content": "<p>Agreed, being able to do everything offline is much more confortable.</p>",
      "rawMarkdown": "Agreed, being able to do everything offline is much more confortable.",
      "votes": null
    },
    {
      "id": "1085216",
      "postDate": "11/20/2020 18:57:53",
      "content": "<p>When I've seen this competition a few days ago, I was super happy and I was ready to join because it was similar to birdscall competition which was great! Then, I've noticed that test set was public and decided to not join. Public test set availability has more drawbacks than advantages in my opinion, it could lead to hundred of blends to just have a winning submission. Such blends are not really useful in real life. Also, as the test set is small, it could be labelled manually (even if not allowed) and final submissions coud be selected to match best of these labels (no way for Kaggle to check for cheating because manual labels are not involved in training directly). Anyway, good luck to everyone, have fun with this nice audio competition.</p>",
      "rawMarkdown": "When I've seen this competition a few days ago, I was super happy and I was ready to join because it was similar to birdscall competition which was great! Then, I've noticed that test set was public and decided to not join. Public test set availability has more drawbacks than advantages in my opinion, it could lead to hundred of blends to just have a winning submission. Such blends are not really useful in real life. Also, as the test set is small, it could be labelled manually (even if not allowed) and final submissions coud be selected to match best of these labels (no way for Kaggle to check for cheating because manual labels are not involved in training directly). Anyway, good luck to everyone, have fun with this nice audio competition.",
      "votes": null
    },
    {
      "id": "1086585",
      "postDate": "11/21/2020 19:55:22",
      "content": "<p>Having the public test available is not a bad idea. If we remember how it was complicated at the beggining of the birdcall competition. But having the entire test set will create as you said tons of blends of blends unfortunately </p>",
      "rawMarkdown": "Having the public test available is not a bad idea. If we remember how it was complicated at the beggining of the birdcall competition. But having the entire test set will create as you said tons of blends of blends unfortunately",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1083217,
      "author_name": "gaborfodor",
      "author_url": "",
      "post_date": "11/18/2020 18:09:15",
      "content": "<p>I agree with the pros. However there are cons as well. </p>\n<p>The cornell comp was quite a tough nut to crack. Hidden and completely different test set was the main reason but the not so obvious inference submissions caused quite a lot of headaches as well.</p>\n<p>The kaggle kernel env also has stability issues. Its a living thing with version upgrades fixes during competiton. The error messages also more transparent for the standard csv upload.  </p>",
      "votes": null,
      "replies": [
        {
          "id": 1083678,
          "author_name": "theoviel",
          "author_url": "",
          "post_date": "11/19/2020 08:31:32",
          "content": "<p>Agreed, being able to do everything offline is much more confortable.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1083223,
      "author_name": "kneroma",
      "author_url": "",
      "post_date": "11/18/2020 18:15:27",
      "content": "<p>Most importantly, the test set is publicly available despite it's small size . Hence, one can just walk through it and listen to the records and manually label them accordingly. It may be a tough task though !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1083318,
      "author_name": "inversion",
      "author_url": "",
      "post_date": "11/18/2020 20:56:56",
      "content": "<p>I responded with some thoughts here:</p>\n<p><a href=\"https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197923#1083317\" target=\"_blank\">https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197923#1083317</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1083677,
          "author_name": "theoviel",
          "author_url": "",
          "post_date": "11/19/2020 08:30:10",
          "content": "<p>Thanks a lot ! </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1085216,
      "author_name": "mpware",
      "author_url": "",
      "post_date": "11/20/2020 18:57:53",
      "content": "<p>When I've seen this competition a few days ago, I was super happy and I was ready to join because it was similar to birdscall competition which was great! Then, I've noticed that test set was public and decided to not join. Public test set availability has more drawbacks than advantages in my opinion, it could lead to hundred of blends to just have a winning submission. Such blends are not really useful in real life. Also, as the test set is small, it could be labelled manually (even if not allowed) and final submissions coud be selected to match best of these labels (no way for Kaggle to check for cheating because manual labels are not involved in training directly). Anyway, good luck to everyone, have fun with this nice audio competition.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1086585,
          "author_name": "ludovick",
          "author_url": "",
          "post_date": "11/21/2020 19:55:22",
          "content": "<p>Having the public test available is not a bad idea. If we remember how it was complicated at the beggining of the birdcall competition. But having the entire test set will create as you said tons of blends of blends unfortunately </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1083203": "Code Competitions require the inference to be made in the kaggle kernels, in a limited time.\n\nPrevious audio competitions were code competitions, and I don't really see why this one should not be one as well. Code competitions force participants to build better models instead of bigger ensembles, which is usually a good thing, especially when the solution is going to be put in production later on.\n\nAny thoughts ?",
    "1083217": "I agree with the pros. However there are cons as well. \n\nThe cornell comp was quite a tough nut to crack. Hidden and completely different test set was the main reason but the not so obvious inference submissions caused quite a lot of headaches as well.\n\nThe kaggle kernel env also has stability issues. Its a living thing with version upgrades fixes during competiton. The error messages also more transparent for the standard csv upload.",
    "1083223": "Most importantly, the test set is publicly available despite it's small size . Hence, one can just walk through it and listen to the records and manually label them accordingly. It may be a tough task though !",
    "1083318": "I responded with some thoughts here:\n\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/197923#1083317",
    "1083677": "Thanks a lot !",
    "1083678": "Agreed, being able to do everything offline is much more confortable.",
    "1085216": "When I've seen this competition a few days ago, I was super happy and I was ready to join because it was similar to birdscall competition which was great! Then, I've noticed that test set was public and decided to not join. Public test set availability has more drawbacks than advantages in my opinion, it could lead to hundred of blends to just have a winning submission. Such blends are not really useful in real life. Also, as the test set is small, it could be labelled manually (even if not allowed) and final submissions coud be selected to match best of these labels (no way for Kaggle to check for cheating because manual labels are not involved in training directly). Anyway, good luck to everyone, have fun with this nice audio competition.",
    "1086585": "Having the public test available is not a bad idea. If we remember how it was complicated at the beggining of the birdcall competition. But having the entire test set will create as you said tons of blends of blends unfortunately"
  },
  "source": "meta"
}