{
  "id": 477056,
  "title": "clarification of the test data",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/477056",
  "author_name": "",
  "post_date": "2024-02-14T14:37:46.520068300Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi <a href=\"https://www.kaggle.com/jetakow\" target=\"_blank\">@jetakow</a>, <a href=\"https://www.kaggle.com/tomasjeline2\" target=\"_blank\">@tomasjeline2</a> </p>\n<p>I have a few questions regarding the test data. I would appreciate the clarification.</p>\n<ul>\n<li>How is the test data split into public and private? Is it divided based on time series or just random by <code>case_id</code>?</li>\n<li>Are both public and private test data included in test files during the public evaluation phase?<ul>\n<li>I'm curious if the test data only includes the public data or not.</li></ul></li>\n<li>How many weeks are there in the test data?</li>\n</ul>\n<p>Thank you in advance.</p>",
  "messages": [
    {
      "id": "2652052",
      "postDate": "02/14/2024 14:37:46",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/jetakow\" target=\"_blank\">@jetakow</a>, <a href=\"https://www.kaggle.com/tomasjeline2\" target=\"_blank\">@tomasjeline2</a> </p>\n<p>I have a few questions regarding the test data. I would appreciate the clarification.</p>\n<ul>\n<li>How is the test data split into public and private? Is it divided based on time series or just random by <code>case_id</code>?</li>\n<li>Are both public and private test data included in test files during the public evaluation phase?<ul>\n<li>I'm curious if the test data only includes the public data or not.</li></ul></li>\n<li>How many weeks are there in the test data?</li>\n</ul>\n<p>Thank you in advance.</p>",
      "rawMarkdown": "Hi @jetakow, @tomasjeline2 \n\nI have a few questions regarding the test data. I would appreciate the clarification.\n\n* How is the test data split into public and private? Is it divided based on time series or just random by `case_id`?\n* Are both public and private test data included in test files during the public evaluation phase?\n  * I'm curious if the test data only includes the public data or not.\n* How many weeks are there in the test data?\n\nThank you in advance.",
      "votes": null
    },
    {
      "id": "2652069",
      "postDate": "02/14/2024 14:45:47",
      "content": "<blockquote>\n  <p>How is the test data split into public and private? Is it divided based on time series or just random by case_id</p>\n</blockquote>\n<p>We will not disclose that. You should not think about how to maximize the model on private test set, but how to make the model as stable and well-performing at the same time :)<br>\nI get this question almost on daily basis. I think we will include it somewhere where everyone can see it.</p>\n<blockquote>\n  <p>Are both public and private test data included in test files during the public evaluation phase?</p>\n</blockquote>\n<p>Interesting question, we will discuss internally whether this can be disclosed. </p>\n<blockquote>\n  <p>How many weeks are there in the test data?</p>\n</blockquote>\n<p>We will not disclose that. That would lead only to further hacking the metric.</p>",
      "rawMarkdown": ">How is the test data split into public and private? Is it divided based on time series or just random by case_id\n\nWe will not disclose that. You should not think about how to maximize the model on private test set, but how to make the model as stable and well-performing at the same time :)\nI get this question almost on daily basis. I think we will include it somewhere where everyone can see it.\n\n>Are both public and private test data included in test files during the public evaluation phase?\n\nInteresting question, we will discuss internally whether this can be disclosed. \n\n>How many weeks are there in the test data?\n\nWe will not disclose that. That would lead only to further hacking the metric.",
      "votes": null
    },
    {
      "id": "2652086",
      "postDate": "02/14/2024 14:56:45",
      "content": "<blockquote>\n  <p>Are both public and private test data included in test files during the public evaluation phase?</p>\n</blockquote>\n<p>So, I am back with an answer. We will not disclose that as well. </p>",
      "rawMarkdown": ">Are both public and private test data included in test files during the public evaluation phase?\n\nSo, I am back with an answer. We will not disclose that as well.",
      "votes": null
    },
    {
      "id": "2652095",
      "postDate": "02/14/2024 15:03:43",
      "content": "<p>thanks for your prompt reply :)</p>",
      "rawMarkdown": "thanks for your prompt reply :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2652069,
      "author_name": "jetakow",
      "author_url": "",
      "post_date": "02/14/2024 14:45:47",
      "content": "<blockquote>\n  <p>How is the test data split into public and private? Is it divided based on time series or just random by case_id</p>\n</blockquote>\n<p>We will not disclose that. You should not think about how to maximize the model on private test set, but how to make the model as stable and well-performing at the same time :)<br>\nI get this question almost on daily basis. I think we will include it somewhere where everyone can see it.</p>\n<blockquote>\n  <p>Are both public and private test data included in test files during the public evaluation phase?</p>\n</blockquote>\n<p>Interesting question, we will discuss internally whether this can be disclosed. </p>\n<blockquote>\n  <p>How many weeks are there in the test data?</p>\n</blockquote>\n<p>We will not disclose that. That would lead only to further hacking the metric.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2652086,
          "author_name": "jetakow",
          "author_url": "",
          "post_date": "02/14/2024 14:56:45",
          "content": "<blockquote>\n  <p>Are both public and private test data included in test files during the public evaluation phase?</p>\n</blockquote>\n<p>So, I am back with an answer. We will not disclose that as well. </p>",
          "votes": null,
          "replies": [
            {
              "id": 2652095,
              "author_name": "keisho",
              "author_url": "",
              "post_date": "02/14/2024 15:03:43",
              "content": "<p>thanks for your prompt reply :)</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2652052": "Hi @jetakow, @tomasjeline2 \n\nI have a few questions regarding the test data. I would appreciate the clarification.\n\n* How is the test data split into public and private? Is it divided based on time series or just random by `case_id`?\n* Are both public and private test data included in test files during the public evaluation phase?\n  * I'm curious if the test data only includes the public data or not.\n* How many weeks are there in the test data?\n\nThank you in advance.",
    "2652069": ">How is the test data split into public and private? Is it divided based on time series or just random by case_id\n\nWe will not disclose that. You should not think about how to maximize the model on private test set, but how to make the model as stable and well-performing at the same time :)\nI get this question almost on daily basis. I think we will include it somewhere where everyone can see it.\n\n>Are both public and private test data included in test files during the public evaluation phase?\n\nInteresting question, we will discuss internally whether this can be disclosed. \n\n>How many weeks are there in the test data?\n\nWe will not disclose that. That would lead only to further hacking the metric.",
    "2652086": ">Are both public and private test data included in test files during the public evaluation phase?\n\nSo, I am back with an answer. We will not disclose that as well.",
    "2652095": "thanks for your prompt reply :)"
  },
  "source": "meta"
}