{
  "id": 62013,
  "title": "The score diff between validateset and testset?",
  "url": "/competitions/google-ai-open-images-object-detection-track/discussion/62013",
  "author_name": "",
  "post_date": "2018-07-26T10:13:30.871944Z",
  "votes": null,
  "comment_count": 6,
  "views": 0,
  "content": "<p>So sad that I debug the code for the whole day without figure out why the score evaluted in valset and teset is so big.</p>\n\n<p>I evaluated my valset result by code in tensorflow, and got score with 23. While after the same infer steps, I submit the testset result to service, and got score with 13. </p>\n\n<p>Does anyone has the same problem？</p>",
  "messages": [
    {
      "id": "362420",
      "postDate": "07/26/2018 10:13:30",
      "content": "<p>So sad that I debug the code for the whole day without figure out why the score evaluted in valset and teset is so big.</p>\n\n<p>I evaluated my valset result by code in tensorflow, and got score with 23. While after the same infer steps, I submit the testset result to service, and got score with 13. </p>\n\n<p>Does anyone has the same problem？</p>",
      "rawMarkdown": "So sad that I debug the code for the whole day without figure out why the score evaluted in valset and teset is so big.\n\nI evaluated my valset result by code in tensorflow, and got score with 23. While after the same infer steps, I submit the testset result to service, and got score with 13. \n\nDoes anyone has the same problem？",
      "votes": null
    },
    {
      "id": "363078",
      "postDate": "07/27/2018 19:59:21",
      "content": "<p>Try, as a debugging, to use the ground truth as prediction for one or few entries. This should give you perfect score i think</p>",
      "rawMarkdown": "Try, as a debugging, to use the ground truth as prediction for one or few entries. This should give you perfect score i think",
      "votes": null
    },
    {
      "id": "363203",
      "postDate": "07/28/2018 07:09:11",
      "content": "<p>thanks, Moshel.  which version of evaluation code you used to evaluate validateset in native?  C# or Python.</p>",
      "rawMarkdown": "thanks, Moshel.  which version of evaluation code you used to evaluate validateset in native?  C# or Python.",
      "votes": null
    },
    {
      "id": "363208",
      "postDate": "07/28/2018 07:19:21",
      "content": "<p>See <a href=\"https://www.kaggle.com/c/google-ai-open-images-object-detection-track/discussion/61086\">https://www.kaggle.com/c/google-ai-open-images-object-detection-track/discussion/61086</a></p>\n\n<p>There is a link there to a colab notebook</p>\n\n<p>I am not sure i applied the patch to this notebook so make sure it is applied. Let me know if you can't figure it out. I have just spent the morning playing with it to better understand the way its calculated. It seems reasonably consistent. </p>",
      "rawMarkdown": "See https://www.kaggle.com/c/google-ai-open-images-object-detection-track/discussion/61086\n\nThere is a link there to a colab notebook\n\nI am not sure i applied the patch to this notebook so make sure it is applied. Let me know if you can't figure it out. I have just spent the morning playing with it to better understand the way its calculated. It seems reasonably consistent.",
      "votes": null
    },
    {
      "id": "363410",
      "postDate": "07/28/2018 22:42:44",
      "content": "<p>Please double check the following:</p>\n\n<ul>\n<li>you subtracted the recommended validation subset of images from the train set of your model.</li>\n<li>you ran the label expansion algorithm on the validation data.</li>\n</ul>\n\n<p>In general it is entirely possible to see different results on the dashboards in comparison to the results on validation set. </p>",
      "rawMarkdown": "Please double check the following:\n\n- you subtracted the recommended validation subset of images from the train set of your model.\n- you ran the label expansion algorithm on the validation data.\n\nIn general it is entirely possible to see different results on the dashboards in comparison to the results on validation set.",
      "votes": null
    },
    {
      "id": "366703",
      "postDate": "08/06/2018 08:59:27",
      "content": "<p>Hi,fromsea.\nI get the same problem,have you solved it?\nlooking for your reply.</p>",
      "rawMarkdown": "Hi,fromsea.\nI get the same problem,have you solved it?\nlooking for your reply.",
      "votes": null
    },
    {
      "id": "367062",
      "postDate": "08/07/2018 02:16:02",
      "content": "<p>NO,thats not make sense~\nplease check your program,many people get the same problem~</p>",
      "rawMarkdown": "NO,thats not make sense~\nplease check your program,many people get the same problem~",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 363078,
      "author_name": "moshel",
      "author_url": "",
      "post_date": "07/27/2018 19:59:21",
      "content": "<p>Try, as a debugging, to use the ground truth as prediction for one or few entries. This should give you perfect score i think</p>",
      "votes": null,
      "replies": [
        {
          "id": 363203,
          "author_name": "xmubingo",
          "author_url": "",
          "post_date": "07/28/2018 07:09:11",
          "content": "<p>thanks, Moshel.  which version of evaluation code you used to evaluate validateset in native?  C# or Python.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 363208,
          "author_name": "moshel",
          "author_url": "",
          "post_date": "07/28/2018 07:19:21",
          "content": "<p>See <a href=\"https://www.kaggle.com/c/google-ai-open-images-object-detection-track/discussion/61086\">https://www.kaggle.com/c/google-ai-open-images-object-detection-track/discussion/61086</a></p>\n\n<p>There is a link there to a colab notebook</p>\n\n<p>I am not sure i applied the patch to this notebook so make sure it is applied. Let me know if you can't figure it out. I have just spent the morning playing with it to better understand the way its calculated. It seems reasonably consistent. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 363410,
      "author_name": "akuznetsa",
      "author_url": "",
      "post_date": "07/28/2018 22:42:44",
      "content": "<p>Please double check the following:</p>\n\n<ul>\n<li>you subtracted the recommended validation subset of images from the train set of your model.</li>\n<li>you ran the label expansion algorithm on the validation data.</li>\n</ul>\n\n<p>In general it is entirely possible to see different results on the dashboards in comparison to the results on validation set. </p>",
      "votes": null,
      "replies": [
        {
          "id": 367062,
          "author_name": "mayufeng",
          "author_url": "",
          "post_date": "08/07/2018 02:16:02",
          "content": "<p>NO,thats not make sense~\nplease check your program,many people get the same problem~</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 366703,
      "author_name": "mayufeng",
      "author_url": "",
      "post_date": "08/06/2018 08:59:27",
      "content": "<p>Hi,fromsea.\nI get the same problem,have you solved it?\nlooking for your reply.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "362420": "So sad that I debug the code for the whole day without figure out why the score evaluted in valset and teset is so big.\n\nI evaluated my valset result by code in tensorflow, and got score with 23. While after the same infer steps, I submit the testset result to service, and got score with 13. \n\nDoes anyone has the same problem？",
    "363078": "Try, as a debugging, to use the ground truth as prediction for one or few entries. This should give you perfect score i think",
    "363203": "thanks, Moshel.  which version of evaluation code you used to evaluate validateset in native?  C# or Python.",
    "363208": "See https://www.kaggle.com/c/google-ai-open-images-object-detection-track/discussion/61086\n\nThere is a link there to a colab notebook\n\nI am not sure i applied the patch to this notebook so make sure it is applied. Let me know if you can't figure it out. I have just spent the morning playing with it to better understand the way its calculated. It seems reasonably consistent.",
    "363410": "Please double check the following:\n\n- you subtracted the recommended validation subset of images from the train set of your model.\n- you ran the label expansion algorithm on the validation data.\n\nIn general it is entirely possible to see different results on the dashboards in comparison to the results on validation set.",
    "366703": "Hi,fromsea.\nI get the same problem,have you solved it?\nlooking for your reply.",
    "367062": "NO,thats not make sense~\nplease check your program,many people get the same problem~"
  },
  "source": "meta"
}