{
  "id": 476231,
  "title": "Question on Test Data Columns Availability",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/476231",
  "author_name": "",
  "post_date": "2024-02-11T16:25:32.043205500Z",
  "votes": 5,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hi (@jetakow),</p>\n<p>The data tab states \"It's worth noting that some external data providers might not be available for future (test) evaluations, which is anticipated\". What does this exactly mean? Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set? </p>\n<p>Thank you.</p>",
  "messages": [
    {
      "id": "2647564",
      "postDate": "02/11/2024 16:25:32",
      "content": "<p>Hi (@jetakow),</p>\n<p>The data tab states \"It's worth noting that some external data providers might not be available for future (test) evaluations, which is anticipated\". What does this exactly mean? Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set? </p>\n<p>Thank you.</p>",
      "rawMarkdown": "Hi (@jetakow),\n\nThe data tab states \"It's worth noting that some external data providers might not be available for future (test) evaluations, which is anticipated\". What does this exactly mean? Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set? \n\nThank you.",
      "votes": null
    },
    {
      "id": "2647753",
      "postDate": "02/11/2024 18:29:27",
      "content": "<p>Hi,</p>\n<blockquote>\n  <p>Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set?</p>\n</blockquote>\n<p>Exactly, or even the whole table can be missing after some date. </p>",
      "rawMarkdown": "Hi,\n\n> Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set?\n\nExactly, or even the whole table can be missing after some date.",
      "votes": null
    },
    {
      "id": "2647970",
      "postDate": "02/12/2024 00:10:36",
      "content": "<p>Thank you for the reply. Much appreciated. (@jetakow) </p>\n<p>If the table starts to be missing right before the date when the private leaderboard starts, does the file \"xxx.parquet\" used to contain that table still exist (with proper case_id and same columns but has null values)? Or the file just complete gone? </p>\n<p>I just try to understand whether the files count and column names/count are the same if some data is unavailable during the private leaderboard.  </p>\n<p>Thank you. </p>",
      "rawMarkdown": "Thank you for the reply. Much appreciated. (@jetakow) \n\nIf the table starts to be missing right before the date when the private leaderboard starts, does the file \"xxx.parquet\" used to contain that table still exist (with proper case_id and same columns but has null values)? Or the file just complete gone? \n\nI just try to understand whether the files count and column names/count are the same if some data is unavailable during the private leaderboard.  \n\nThank you.",
      "votes": null
    },
    {
      "id": "2648442",
      "postDate": "02/12/2024 09:03:50",
      "content": "<p>If all data are missing on the test sample, we would still keep the empty tables.</p>",
      "rawMarkdown": "If all data are missing on the test sample, we would still keep the empty tables.",
      "votes": null
    },
    {
      "id": "2648743",
      "postDate": "02/12/2024 11:41:39",
      "content": "<p>Thank you for your reply. It is very helpful to have the empty tables in place.</p>",
      "rawMarkdown": "Thank you for your reply. It is very helpful to have the empty tables in place.",
      "votes": null
    },
    {
      "id": "2648809",
      "postDate": "02/12/2024 12:50:28",
      "content": "<p>I was thinking it might be :)</p>",
      "rawMarkdown": "I was thinking it might be :)",
      "votes": null
    },
    {
      "id": "2649534",
      "postDate": "02/12/2024 23:15:40",
      "content": "<p>If some column is missing in the test data, does it mean the there the column isnt there any more or do you plan to fill the column with nan?</p>",
      "rawMarkdown": "If some column is missing in the test data, does it mean the there the column isnt there any more or do you plan to fill the column with nan?",
      "votes": null
    },
    {
      "id": "2650331",
      "postDate": "02/13/2024 11:51:49",
      "content": "<p>If a column is missing in the test set, it implies it is no longer available in future. The simplest solution on your side would be to will it with nans. </p>",
      "rawMarkdown": "If a column is missing in the test set, it implies it is no longer available in future. The simplest solution on your side would be to will it with nans.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2647753,
      "author_name": "jetakow",
      "author_url": "",
      "post_date": "02/11/2024 18:29:27",
      "content": "<p>Hi,</p>\n<blockquote>\n  <p>Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set?</p>\n</blockquote>\n<p>Exactly, or even the whole table can be missing after some date. </p>",
      "votes": null,
      "replies": [
        {
          "id": 2647970,
          "author_name": "fengpan23",
          "author_url": "",
          "post_date": "02/12/2024 00:10:36",
          "content": "<p>Thank you for the reply. Much appreciated. (@jetakow) </p>\n<p>If the table starts to be missing right before the date when the private leaderboard starts, does the file \"xxx.parquet\" used to contain that table still exist (with proper case_id and same columns but has null values)? Or the file just complete gone? </p>\n<p>I just try to understand whether the files count and column names/count are the same if some data is unavailable during the private leaderboard.  </p>\n<p>Thank you. </p>",
          "votes": null,
          "replies": [
            {
              "id": 2648442,
              "author_name": "jetakow",
              "author_url": "",
              "post_date": "02/12/2024 09:03:50",
              "content": "<p>If all data are missing on the test sample, we would still keep the empty tables.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2648743,
                  "author_name": "fengpan23",
                  "author_url": "",
                  "post_date": "02/12/2024 11:41:39",
                  "content": "<p>Thank you for your reply. It is very helpful to have the empty tables in place.</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2648809,
                      "author_name": "jetakow",
                      "author_url": "",
                      "post_date": "02/12/2024 12:50:28",
                      "content": "<p>I was thinking it might be :)</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                },
                {
                  "id": 2649534,
                  "author_name": "lucamtb",
                  "author_url": "",
                  "post_date": "02/12/2024 23:15:40",
                  "content": "<p>If some column is missing in the test data, does it mean the there the column isnt there any more or do you plan to fill the column with nan?</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2650331,
                      "author_name": "jetakow",
                      "author_url": "",
                      "post_date": "02/13/2024 11:51:49",
                      "content": "<p>If a column is missing in the test set, it implies it is no longer available in future. The simplest solution on your side would be to will it with nans. </p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2647564": "Hi (@jetakow),\n\nThe data tab states \"It's worth noting that some external data providers might not be available for future (test) evaluations, which is anticipated\". What does this exactly mean? Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set? \n\nThank you.",
    "2647753": "Hi,\n\n> Does it mean some columns available in the train dataset and public leaderboard test dataset can be missing for private leaderboard test dataset? For example, currently both train and test data are available for Credit bureau provider B (an external data provider) and does the statement suggest some data columns from this provider might be missing during the private leaderboard test set?\n\nExactly, or even the whole table can be missing after some date.",
    "2647970": "Thank you for the reply. Much appreciated. (@jetakow) \n\nIf the table starts to be missing right before the date when the private leaderboard starts, does the file \"xxx.parquet\" used to contain that table still exist (with proper case_id and same columns but has null values)? Or the file just complete gone? \n\nI just try to understand whether the files count and column names/count are the same if some data is unavailable during the private leaderboard.  \n\nThank you.",
    "2648442": "If all data are missing on the test sample, we would still keep the empty tables.",
    "2648743": "Thank you for your reply. It is very helpful to have the empty tables in place.",
    "2648809": "I was thinking it might be :)",
    "2649534": "If some column is missing in the test data, does it mean the there the column isnt there any more or do you plan to fill the column with nan?",
    "2650331": "If a column is missing in the test set, it implies it is no longer available in future. The simplest solution on your side would be to will it with nans."
  },
  "source": "meta"
}