{
  "id": 34740,
  "title": "Is the stage2 test set the same as stage1?",
  "url": "/competitions/intel-mobileodt-cervical-cancer-screening/discussion/34740",
  "author_name": "",
  "post_date": "2017-06-15T03:46:21.757477700Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I'm using stage1 test set as my local validation set, but the computed scores, based on published stage1 test set labels, don't match stage2 public scores.  So I'm thinking the stage2 public score is based on a random 13% of the entire (stage1+2) test set.</p>\n\n<p>This doesn't make sense, because that means part of the private test set will be based on published stage1 solution.  So the correct solution will be to merge published solution with stage2 test set predictions.  I didn't expect this, didn't write code for it, probably few of us did.</p>",
  "messages": [
    {
      "id": "192897",
      "postDate": "06/15/2017 03:46:21",
      "content": "<p>I'm using stage1 test set as my local validation set, but the computed scores, based on published stage1 test set labels, don't match stage2 public scores.  So I'm thinking the stage2 public score is based on a random 13% of the entire (stage1+2) test set.</p>\n\n<p>This doesn't make sense, because that means part of the private test set will be based on published stage1 solution.  So the correct solution will be to merge published solution with stage2 test set predictions.  I didn't expect this, didn't write code for it, probably few of us did.</p>",
      "rawMarkdown": "I'm using stage1 test set as my local validation set, but the computed scores, based on published stage1 test set labels, don't match stage2 public scores.  So I'm thinking the stage2 public score is based on a random 13% of the entire (stage1+2) test set.\n\nThis doesn't make sense, because that means part of the private test set will be based on published stage1 solution.  So the correct solution will be to merge published solution with stage2 test set predictions.  I didn't expect this, didn't write code for it, probably few of us did.",
      "votes": null
    },
    {
      "id": "192913",
      "postDate": "06/15/2017 05:02:12",
      "content": "<p>As I believe, 13% of the entire test data is stage1 test set, so stage2 public score is based on stage1 test data, and stage2 private score is based on stage2 test data.</p>",
      "rawMarkdown": "As I believe, 13% of the entire test data is stage1 test set, so stage2 public score is based on stage1 test data, and stage2 private score is based on stage2 test data.",
      "votes": null
    },
    {
      "id": "192926",
      "postDate": "06/15/2017 05:38:39",
      "content": "<p>I hope this is the case...</p>",
      "rawMarkdown": "I hope this is the case...",
      "votes": null
    },
    {
      "id": "193048",
      "postDate": "06/15/2017 13:27:33",
      "content": "<p>I think they are the same. As observed on the public LB, many guys got 0 score. So I think the public test set in stage2 is the same as stage 1. \nAlso, I've made a concatenation of my stage 1 submission and a random submission of stage 2 test set, the score remains unchanged.</p>",
      "rawMarkdown": "I think they are the same. As observed on the public LB, many guys got 0 score. So I think the public test set in stage2 is the same as stage 1. \nAlso, I've made a concatenation of my stage 1 submission and a random submission of stage 2 test set, the score remains unchanged.",
      "votes": null
    },
    {
      "id": "193165",
      "postDate": "06/15/2017 18:30:28",
      "content": "<p>I don't know about yours. My stage 2 leaderboard score for my current submission is EXACTLY the same (like, same as in all the decimal digits same!) as my stage1 leaderboard score for the same submission.\nSince I simply appended the predictions of the stage2 submissions to those of stage1, I can say with certainty, that the leaderboard is being calculated on the stage1 lb subset.</p>\n\n<p>Now with threads coming up that say that there is a leak in the Stage2 submissions as well <a href=\"https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/34788\">Stage 2 data is not purely unseen data ?</a>, I don't even know what I am supposed to do.</p>",
      "rawMarkdown": "I don't know about yours. My stage 2 leaderboard score for my current submission is EXACTLY the same (like, same as in all the decimal digits same!) as my stage1 leaderboard score for the same submission.\nSince I simply appended the predictions of the stage2 submissions to those of stage1, I can say with certainty, that the leaderboard is being calculated on the stage1 lb subset.\n\nNow with threads coming up that say that there is a leak in the Stage2 submissions as well [Stage 2 data is not purely unseen data ?][1], I don't even know what I am supposed to do.\n\n\n  [1]: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/34788",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 192913,
      "author_name": "uhfband",
      "author_url": "",
      "post_date": "06/15/2017 05:02:12",
      "content": "<p>As I believe, 13% of the entire test data is stage1 test set, so stage2 public score is based on stage1 test data, and stage2 private score is based on stage2 test data.</p>",
      "votes": null,
      "replies": [
        {
          "id": 192926,
          "author_name": "kubilai",
          "author_url": "",
          "post_date": "06/15/2017 05:38:39",
          "content": "<p>I hope this is the case...</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 193048,
      "author_name": "wangg12",
      "author_url": "",
      "post_date": "06/15/2017 13:27:33",
      "content": "<p>I think they are the same. As observed on the public LB, many guys got 0 score. So I think the public test set in stage2 is the same as stage 1. \nAlso, I've made a concatenation of my stage 1 submission and a random submission of stage 2 test set, the score remains unchanged.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 193165,
      "author_name": "yadavsarthak",
      "author_url": "",
      "post_date": "06/15/2017 18:30:28",
      "content": "<p>I don't know about yours. My stage 2 leaderboard score for my current submission is EXACTLY the same (like, same as in all the decimal digits same!) as my stage1 leaderboard score for the same submission.\nSince I simply appended the predictions of the stage2 submissions to those of stage1, I can say with certainty, that the leaderboard is being calculated on the stage1 lb subset.</p>\n\n<p>Now with threads coming up that say that there is a leak in the Stage2 submissions as well <a href=\"https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/34788\">Stage 2 data is not purely unseen data ?</a>, I don't even know what I am supposed to do.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "192897": "I'm using stage1 test set as my local validation set, but the computed scores, based on published stage1 test set labels, don't match stage2 public scores.  So I'm thinking the stage2 public score is based on a random 13% of the entire (stage1+2) test set.\n\nThis doesn't make sense, because that means part of the private test set will be based on published stage1 solution.  So the correct solution will be to merge published solution with stage2 test set predictions.  I didn't expect this, didn't write code for it, probably few of us did.",
    "192913": "As I believe, 13% of the entire test data is stage1 test set, so stage2 public score is based on stage1 test data, and stage2 private score is based on stage2 test data.",
    "192926": "I hope this is the case...",
    "193048": "I think they are the same. As observed on the public LB, many guys got 0 score. So I think the public test set in stage2 is the same as stage 1. \nAlso, I've made a concatenation of my stage 1 submission and a random submission of stage 2 test set, the score remains unchanged.",
    "193165": "I don't know about yours. My stage 2 leaderboard score for my current submission is EXACTLY the same (like, same as in all the decimal digits same!) as my stage1 leaderboard score for the same submission.\nSince I simply appended the predictions of the stage2 submissions to those of stage1, I can say with certainty, that the leaderboard is being calculated on the stage1 lb subset.\n\nNow with threads coming up that say that there is a leak in the Stage2 submissions as well [Stage 2 data is not purely unseen data ?][1], I don't even know what I am supposed to do.\n\n\n  [1]: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/34788"
  },
  "source": "meta"
}