{
  "id": 30150,
  "title": "This competition is also vulnerable to LB-based truth mining",
  "url": "/competitions/intel-mobileodt-cervical-cancer-screening/discussion/30150",
  "author_name": "",
  "post_date": "2017-03-15T21:16:49.732212100Z",
  "votes": 9,
  "comment_count": 7,
  "views": 0,
  "content": "<p>I'm going to bet there's going to be a 0 LB score in a couple weeks, anyone wants to bet against me? XD</p>",
  "messages": [
    {
      "id": "167944",
      "postDate": "03/15/2017 21:16:49",
      "content": "<p>I'm going to bet there's going to be a 0 LB score in a couple weeks, anyone wants to bet against me? XD</p>",
      "rawMarkdown": "I'm going to bet there's going to be a 0 LB score in a couple weeks, anyone wants to bet against me? XD",
      "votes": null
    },
    {
      "id": "167952",
      "postDate": "03/15/2017 22:01:28",
      "content": "<p>But just like with other competitions it will achieve nothing and be against the spirit of the competition.</p>",
      "rawMarkdown": "But just like with other competitions it will achieve nothing and be against the spirit of the competition.",
      "votes": null
    },
    {
      "id": "167953",
      "postDate": "03/15/2017 22:06:46",
      "content": "<p>Yes, I am sure it is possible but that will not foster creativity and as always It will really amount to nothing. </p>",
      "rawMarkdown": "Yes, I am sure it is possible but that will not foster creativity and as always It will really amount to nothing.",
      "votes": null
    },
    {
      "id": "167958",
      "postDate": "03/15/2017 22:29:41",
      "content": "<p>The \"nothing\" it achieves is giving you 512 extra images to use as a validation set 2 weeks into the competition,</p>",
      "rawMarkdown": "The \"nothing\" it achieves is giving you 512 extra images to use as a validation set 2 weeks into the competition,",
      "votes": null
    },
    {
      "id": "168152",
      "postDate": "03/16/2017 12:36:34",
      "content": "<p>Actually it fosters creative thought by showing people possibilities relating to information theory that they wouldn't have otherwise understood.  I for one had no idea of the concepts behind these techniques until seeing Oleg's submission in DSB.  I don't think that Oleg was trying to win first place, but more show that it could be accomplished.  And like Ramon mentioned, the extra data is definitely worth having.</p>",
      "rawMarkdown": "Actually it fosters creative thought by showing people possibilities relating to information theory that they wouldn't have otherwise understood.  I for one had no idea of the concepts behind these techniques until seeing Oleg's submission in DSB.  I don't think that Oleg was trying to win first place, but more show that it could be accomplished.  And like Ramon mentioned, the extra data is definitely worth having.",
      "votes": null
    },
    {
      "id": "172109",
      "postDate": "04/02/2017 00:31:42",
      "content": "<p>... And done. Took a bit more than 2 weeks, but I've finished \"mining\" all the truth values of the test set.</p>\n\n<p>To the site admins and other people \"mining\": Can we keep the LB clear of the 0's here like we're (mostly) in the DSB? If so, I'll keep the data to myself and refrain from posting any silly scores.</p>\n\n<p>And for the curious: With this version of Oleg's technique I can get the Type 1/2/3 classification of 6 samples per submission. 7 may be possible, but I think some overlap on the answers isn't avoidable. So for now, it takes 86 submissions to get all the values.</p>",
      "rawMarkdown": "... And done. Took a bit more than 2 weeks, but I've finished \"mining\" all the truth values of the test set.\n\nTo the site admins and other people \"mining\": Can we keep the LB clear of the 0's here like we're (mostly) in the DSB? If so, I'll keep the data to myself and refrain from posting any silly scores.\n\nAnd for the curious: With this version of Oleg's technique I can get the Type 1/2/3 classification of 6 samples per submission. 7 may be possible, but I think some overlap on the answers isn't avoidable. So for now, it takes 86 submissions to get all the values.",
      "votes": null
    },
    {
      "id": "172884",
      "postDate": "04/05/2017 05:13:28",
      "content": "<p>@r4m0n Do you think it will be against the spirit of the competition if rather than submitting the \"0 score\" submission, you would rather make the truth values of the 512 test images public (even if, say, 1 week or so before the second stage begins)?  <strong>with the intervention of Kaggle admins, of course</strong> Extra validation data is always good to have :)</p>\n\n<p>Extra validation data aside, I believe it would be better than cluttering the leaderboard with submissions and \"top scorers\" that are guaranteed to do poorly in the second stage.\nToo many submissions with extraneously high (and absurd) leaderboard scores are actually bad for the competition in my opinion since for the uninitiated and the ones who are new and who are interested in joining it might seem like \"well, crap. this competition is washed off already. I'd rather not participate now\".</p>\n\n<p>I know this wouldn't really stop anyone from doing the truth value mining and submitting the predictions, but it would still be indefinitely better than a publically available script that does so. </p>\n\n<p>But this is just my opinion.</p>",
      "rawMarkdown": "r4m0n Do you think it will be against the spirit of the competition if rather than submitting the \"0 score\" submission, you would rather make the truth values of the 512 test images public (even if, say, 1 week or so before the second stage begins)?  **with the intervention of Kaggle admins, of course** Extra validation data is always good to have :)\n\nExtra validation data aside, I believe it would be better than cluttering the leaderboard with submissions and \"top scorers\" that are guaranteed to do poorly in the second stage.\nToo many submissions with extraneously high (and absurd) leaderboard scores are actually bad for the competition in my opinion since for the uninitiated and the ones who are new and who are interested in joining it might seem like \"well, crap. this competition is washed off already. I'd rather not participate now\".\n\nI know this wouldn't really stop anyone from doing the truth value mining and submitting the predictions, but it would still be indefinitely better than a publically available script that does so. \n\nBut this is just my opinion.",
      "votes": null
    },
    {
      "id": "172956",
      "postDate": "04/05/2017 11:13:35",
      "content": "<p>I expect the admins to release the stage 1 truth with the second stage data like they are doing in the DSB, but if they don't, I think it's fair that I do that, yeah.</p>",
      "rawMarkdown": "I expect the admins to release the stage 1 truth with the second stage data like they are doing in the DSB, but if they don't, I think it's fair that I do that, yeah.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 167952,
      "author_name": "jamessargeant",
      "author_url": "",
      "post_date": "03/15/2017 22:01:28",
      "content": "<p>But just like with other competitions it will achieve nothing and be against the spirit of the competition.</p>",
      "votes": null,
      "replies": [
        {
          "id": 167958,
          "author_name": "mumech",
          "author_url": "",
          "post_date": "03/15/2017 22:29:41",
          "content": "<p>The \"nothing\" it achieves is giving you 512 extra images to use as a validation set 2 weeks into the competition,</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 167953,
      "author_name": "stansilas",
      "author_url": "",
      "post_date": "03/15/2017 22:06:46",
      "content": "<p>Yes, I am sure it is possible but that will not foster creativity and as always It will really amount to nothing. </p>",
      "votes": null,
      "replies": [
        {
          "id": 168152,
          "author_name": "mcleodx",
          "author_url": "",
          "post_date": "03/16/2017 12:36:34",
          "content": "<p>Actually it fosters creative thought by showing people possibilities relating to information theory that they wouldn't have otherwise understood.  I for one had no idea of the concepts behind these techniques until seeing Oleg's submission in DSB.  I don't think that Oleg was trying to win first place, but more show that it could be accomplished.  And like Ramon mentioned, the extra data is definitely worth having.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 172109,
      "author_name": "mumech",
      "author_url": "",
      "post_date": "04/02/2017 00:31:42",
      "content": "<p>... And done. Took a bit more than 2 weeks, but I've finished \"mining\" all the truth values of the test set.</p>\n\n<p>To the site admins and other people \"mining\": Can we keep the LB clear of the 0's here like we're (mostly) in the DSB? If so, I'll keep the data to myself and refrain from posting any silly scores.</p>\n\n<p>And for the curious: With this version of Oleg's technique I can get the Type 1/2/3 classification of 6 samples per submission. 7 may be possible, but I think some overlap on the answers isn't avoidable. So for now, it takes 86 submissions to get all the values.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 172884,
      "author_name": "yadavsarthak",
      "author_url": "",
      "post_date": "04/05/2017 05:13:28",
      "content": "<p>@r4m0n Do you think it will be against the spirit of the competition if rather than submitting the \"0 score\" submission, you would rather make the truth values of the 512 test images public (even if, say, 1 week or so before the second stage begins)?  <strong>with the intervention of Kaggle admins, of course</strong> Extra validation data is always good to have :)</p>\n\n<p>Extra validation data aside, I believe it would be better than cluttering the leaderboard with submissions and \"top scorers\" that are guaranteed to do poorly in the second stage.\nToo many submissions with extraneously high (and absurd) leaderboard scores are actually bad for the competition in my opinion since for the uninitiated and the ones who are new and who are interested in joining it might seem like \"well, crap. this competition is washed off already. I'd rather not participate now\".</p>\n\n<p>I know this wouldn't really stop anyone from doing the truth value mining and submitting the predictions, but it would still be indefinitely better than a publically available script that does so. </p>\n\n<p>But this is just my opinion.</p>",
      "votes": null,
      "replies": [
        {
          "id": 172956,
          "author_name": "mumech",
          "author_url": "",
          "post_date": "04/05/2017 11:13:35",
          "content": "<p>I expect the admins to release the stage 1 truth with the second stage data like they are doing in the DSB, but if they don't, I think it's fair that I do that, yeah.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "167944": "I'm going to bet there's going to be a 0 LB score in a couple weeks, anyone wants to bet against me? XD",
    "167952": "But just like with other competitions it will achieve nothing and be against the spirit of the competition.",
    "167953": "Yes, I am sure it is possible but that will not foster creativity and as always It will really amount to nothing.",
    "167958": "The \"nothing\" it achieves is giving you 512 extra images to use as a validation set 2 weeks into the competition,",
    "168152": "Actually it fosters creative thought by showing people possibilities relating to information theory that they wouldn't have otherwise understood.  I for one had no idea of the concepts behind these techniques until seeing Oleg's submission in DSB.  I don't think that Oleg was trying to win first place, but more show that it could be accomplished.  And like Ramon mentioned, the extra data is definitely worth having.",
    "172109": "... And done. Took a bit more than 2 weeks, but I've finished \"mining\" all the truth values of the test set.\n\nTo the site admins and other people \"mining\": Can we keep the LB clear of the 0's here like we're (mostly) in the DSB? If so, I'll keep the data to myself and refrain from posting any silly scores.\n\nAnd for the curious: With this version of Oleg's technique I can get the Type 1/2/3 classification of 6 samples per submission. 7 may be possible, but I think some overlap on the answers isn't avoidable. So for now, it takes 86 submissions to get all the values.",
    "172884": "r4m0n Do you think it will be against the spirit of the competition if rather than submitting the \"0 score\" submission, you would rather make the truth values of the 512 test images public (even if, say, 1 week or so before the second stage begins)?  **with the intervention of Kaggle admins, of course** Extra validation data is always good to have :)\n\nExtra validation data aside, I believe it would be better than cluttering the leaderboard with submissions and \"top scorers\" that are guaranteed to do poorly in the second stage.\nToo many submissions with extraneously high (and absurd) leaderboard scores are actually bad for the competition in my opinion since for the uninitiated and the ones who are new and who are interested in joining it might seem like \"well, crap. this competition is washed off already. I'd rather not participate now\".\n\nI know this wouldn't really stop anyone from doing the truth value mining and submitting the predictions, but it would still be indefinitely better than a publically available script that does so. \n\nBut this is just my opinion.",
    "172956": "I expect the admins to release the stage 1 truth with the second stage data like they are doing in the DSB, but if they don't, I think it's fair that I do that, yeah."
  },
  "source": "meta"
}