{
  "id": 475574,
  "title": "Question on test data",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/475574",
  "author_name": "",
  "post_date": "2024-02-09T00:31:30.025442200Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Competition details state the following:</p>\n<p>'Your work in helping to assess potential clients' default risks will enable consumer finance providers to accept more loan applications. This may improve the lives of people who have historically been denied due to lack of credit history.'</p>\n<p>Train data has details about the vintage being underwritten. Needless to say, this information can't be used in models for new originations - which introduces a bit of a disconnect between the stated intent for use of the model and what the model(s) will actually be doing under the hood.</p>\n<p>All that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. Wondering if we can get some more details on the justification of including details about the specific vintage.</p>",
  "messages": [
    {
      "id": "2643610",
      "postDate": "02/09/2024 00:31:30",
      "content": "<p>Competition details state the following:</p>\n<p>'Your work in helping to assess potential clients' default risks will enable consumer finance providers to accept more loan applications. This may improve the lives of people who have historically been denied due to lack of credit history.'</p>\n<p>Train data has details about the vintage being underwritten. Needless to say, this information can't be used in models for new originations - which introduces a bit of a disconnect between the stated intent for use of the model and what the model(s) will actually be doing under the hood.</p>\n<p>All that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. Wondering if we can get some more details on the justification of including details about the specific vintage.</p>",
      "rawMarkdown": "Competition details state the following:\n\n'Your work in helping to assess potential clients' default risks will enable consumer finance providers to accept more loan applications. This may improve the lives of people who have historically been denied due to lack of credit history.'\n\nTrain data has details about the vintage being underwritten. Needless to say, this information can't be used in models for new originations - which introduces a bit of a disconnect between the stated intent for use of the model and what the model(s) will actually be doing under the hood.\n\nAll that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. Wondering if we can get some more details on the justification of including details about the specific vintage.",
      "votes": null
    },
    {
      "id": "2643930",
      "postDate": "02/09/2024 07:20:41",
      "content": "<p>I am not sure if I understand perfectly vintage in this context, I will try to answer anyway.</p>\n<blockquote>\n  <p>All that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. </p>\n</blockquote>\n<p>Test data may include similar or same data, but I can assure we didn't include train data into test. We don't want to and we won't be providing any better detail on how long is the test sample or which intervals are included. </p>",
      "rawMarkdown": "I am not sure if I understand perfectly vintage in this context, I will try to answer anyway.\n\n>All that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. \n\nTest data may include similar or same data, but I can assure we didn't include train data into test. We don't want to and we won't be providing any better detail on how long is the test sample or which intervals are included.",
      "votes": null
    },
    {
      "id": "2644457",
      "postDate": "02/09/2024 13:49:41",
      "content": "<p>Thanks for the response. Vintage being the year the mortgage was originated. It seems counterintuitive given the stated use of the model that we would be allowed to use vintage information in the model. </p>",
      "rawMarkdown": "Thanks for the response. Vintage being the year the mortgage was originated. It seems counterintuitive given the stated use of the model that we would be allowed to use vintage information in the model.",
      "votes": null
    },
    {
      "id": "2644518",
      "postDate": "02/09/2024 14:51:12",
      "content": "<p>I see your point. I think what can be done for sure is to see how the population changes over time and utilize that information for future prediction (in unspecified future period). Even if the data may be outdated, there can be still some information value that can be utilized. </p>",
      "rawMarkdown": "I see your point. I think what can be done for sure is to see how the population changes over time and utilize that information for future prediction (in unspecified future period). Even if the data may be outdated, there can be still some information value that can be utilized.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2643930,
      "author_name": "jetakow",
      "author_url": "",
      "post_date": "02/09/2024 07:20:41",
      "content": "<p>I am not sure if I understand perfectly vintage in this context, I will try to answer anyway.</p>\n<blockquote>\n  <p>All that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. </p>\n</blockquote>\n<p>Test data may include similar or same data, but I can assure we didn't include train data into test. We don't want to and we won't be providing any better detail on how long is the test sample or which intervals are included. </p>",
      "votes": null,
      "replies": [
        {
          "id": 2644457,
          "author_name": "seanmacmaghnusa",
          "author_url": "",
          "post_date": "02/09/2024 13:49:41",
          "content": "<p>Thanks for the response. Vintage being the year the mortgage was originated. It seems counterintuitive given the stated use of the model that we would be allowed to use vintage information in the model. </p>",
          "votes": null,
          "replies": [
            {
              "id": 2644518,
              "author_name": "jetakow",
              "author_url": "",
              "post_date": "02/09/2024 14:51:12",
              "content": "<p>I see your point. I think what can be done for sure is to see how the population changes over time and utilize that information for future prediction (in unspecified future period). Even if the data may be outdated, there can be still some information value that can be utilized. </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2643610": "Competition details state the following:\n\n'Your work in helping to assess potential clients' default risks will enable consumer finance providers to accept more loan applications. This may improve the lives of people who have historically been denied due to lack of credit history.'\n\nTrain data has details about the vintage being underwritten. Needless to say, this information can't be used in models for new originations - which introduces a bit of a disconnect between the stated intent for use of the model and what the model(s) will actually be doing under the hood.\n\nAll that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. Wondering if we can get some more details on the justification of including details about the specific vintage.",
    "2643930": "I am not sure if I understand perfectly vintage in this context, I will try to answer anyway.\n\n>All that being said, there's nothing suggesting the private leaderboard data will be any different from the train data. \n\nTest data may include similar or same data, but I can assure we didn't include train data into test. We don't want to and we won't be providing any better detail on how long is the test sample or which intervals are included.",
    "2644457": "Thanks for the response. Vintage being the year the mortgage was originated. It seems counterintuitive given the stated use of the model that we would be allowed to use vintage information in the model.",
    "2644518": "I see your point. I think what can be done for sure is to see how the population changes over time and utilize that information for future prediction (in unspecified future period). Even if the data may be outdated, there can be still some information value that can be utilized."
  },
  "source": "meta"
}