{
  "id": 479905,
  "title": "A very important detail regarding WEEK_NUM and the competition",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/479905",
  "author_name": "",
  "post_date": "2024-02-26T11:56:19.943689500Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I've posted it as a response to another message, but I believe that due to its importance it's worth opening a thread with this.<br>\nWithout WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).<br>\n<a href=\"https://www.kaggle.com/jetakow\" target=\"_blank\">@jetakow</a> has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.<br>\nWEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.</p>",
  "messages": [
    {
      "id": "2669632",
      "postDate": "02/26/2024 11:56:19",
      "content": "<p>I've posted it as a response to another message, but I believe that due to its importance it's worth opening a thread with this.<br>\nWithout WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).<br>\n<a href=\"https://www.kaggle.com/jetakow\" target=\"_blank\">@jetakow</a> has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.<br>\nWEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.</p>",
      "rawMarkdown": "I've posted it as a response to another message, but I believe that due to its importance it's worth opening a thread with this.\nWithout WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).\n@jetakow has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.\nWEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.",
      "votes": null
    },
    {
      "id": "2669657",
      "postDate": "02/26/2024 12:11:41",
      "content": "<p>Hi José,</p>\n<p>We will not change train data, all data will remain same including WEEK_NUM. Only test sample will be transformed.<br>\nWe are consistent regarding this topic in our communication, both in announcement and in FAQ we said we will change test sample:</p>\n<blockquote>\n  <p>Q: Questions regarding transformation of test data.<br>\n  A: The transformation was done to train and test before the start of the competition and now we will transform test data in different manner. We won't specify what is the transformation exactly. We will keep column names, dtypes and the change is not going to be drastic.</p>\n</blockquote>",
      "rawMarkdown": "Hi José,\n\nWe will not change train data, all data will remain same including WEEK_NUM. Only test sample will be transformed.\nWe are consistent regarding this topic in our communication, both in announcement and in FAQ we said we will change test sample:\n\n>Q: Questions regarding transformation of test data.\nA: The transformation was done to train and test before the start of the competition and now we will transform test data in different manner. We won't specify what is the transformation exactly. We will keep column names, dtypes and the change is not going to be drastic.",
      "votes": null
    },
    {
      "id": "2669676",
      "postDate": "02/26/2024 12:21:38",
      "content": "<p>Thank you for the clarification Tomas. I understood WEEK_NUM will be delete at all.</p>",
      "rawMarkdown": "Thank you for the clarification Tomas. I understood WEEK_NUM will be delete at all.",
      "votes": null
    },
    {
      "id": "2669864",
      "postDate": "02/26/2024 14:20:14",
      "content": "<blockquote>\n  <p>Without WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).<br>\n  <a href=\"https://www.kaggle.com/jetakow\" target=\"_blank\">@jetakow</a> has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.<br>\n  WEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.</p>\n</blockquote>\n<p>As Tomas answered it will be deleted only from test set. Sorry for the unclear answer, I thought it was obvious we were talking about test set. Removing WEEK_NUM is suboptimal for the aim of this competition but hacking LB left us no other choice. Changing metric is left to discuss. </p>",
      "rawMarkdown": ">Without WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).\n@jetakow has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.\nWEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.\n\nAs Tomas answered it will be deleted only from test set. Sorry for the unclear answer, I thought it was obvious we were talking about test set. Removing WEEK_NUM is suboptimal for the aim of this competition but hacking LB left us no other choice. Changing metric is left to discuss.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2669657,
      "author_name": "tomasjeline2",
      "author_url": "",
      "post_date": "02/26/2024 12:11:41",
      "content": "<p>Hi José,</p>\n<p>We will not change train data, all data will remain same including WEEK_NUM. Only test sample will be transformed.<br>\nWe are consistent regarding this topic in our communication, both in announcement and in FAQ we said we will change test sample:</p>\n<blockquote>\n  <p>Q: Questions regarding transformation of test data.<br>\n  A: The transformation was done to train and test before the start of the competition and now we will transform test data in different manner. We won't specify what is the transformation exactly. We will keep column names, dtypes and the change is not going to be drastic.</p>\n</blockquote>",
      "votes": null,
      "replies": [
        {
          "id": 2669676,
          "author_name": "blindape",
          "author_url": "",
          "post_date": "02/26/2024 12:21:38",
          "content": "<p>Thank you for the clarification Tomas. I understood WEEK_NUM will be delete at all.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2669864,
      "author_name": "jetakow",
      "author_url": "",
      "post_date": "02/26/2024 14:20:14",
      "content": "<blockquote>\n  <p>Without WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).<br>\n  <a href=\"https://www.kaggle.com/jetakow\" target=\"_blank\">@jetakow</a> has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.<br>\n  WEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.</p>\n</blockquote>\n<p>As Tomas answered it will be deleted only from test set. Sorry for the unclear answer, I thought it was obvious we were talking about test set. Removing WEEK_NUM is suboptimal for the aim of this competition but hacking LB left us no other choice. Changing metric is left to discuss. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2669632": "I've posted it as a response to another message, but I believe that due to its importance it's worth opening a thread with this.\nWithout WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).\n@jetakow has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.\nWEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.",
    "2669657": "Hi José,\n\nWe will not change train data, all data will remain same including WEEK_NUM. Only test sample will be transformed.\nWe are consistent regarding this topic in our communication, both in announcement and in FAQ we said we will change test sample:\n\n>Q: Questions regarding transformation of test data.\nA: The transformation was done to train and test before the start of the competition and now we will transform test data in different manner. We won't specify what is the transformation exactly. We will keep column names, dtypes and the change is not going to be drastic.",
    "2669676": "Thank you for the clarification Tomas. I understood WEEK_NUM will be delete at all.",
    "2669864": ">Without WEEK_NUM in the training dataset, neither can a temporal validation framework be established, nor can the metric be calculated (and therefore optimized).\n@jetakow has stated that WEEK_NUM will be completely removed or fixed to a constant value. This is totally incompatible with the objective of optimizing a stability-based metric.\nWEEK_NUM can be deleted from the test dataset, to ensure that no one uses it for hacking, but in the training dataset it is ESSENTIAL.\n\nAs Tomas answered it will be deleted only from test set. Sorry for the unclear answer, I thought it was obvious we were talking about test set. Removing WEEK_NUM is suboptimal for the aim of this competition but hacking LB left us no other choice. Changing metric is left to discuss."
  },
  "source": "meta"
}