{
  "id": 475922,
  "title": "A look into feature's dtypes",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/475922",
  "author_name": "Oleksiy Kononenko",
  "post_date": "2024-02-10T12:51:09.393000",
  "votes": 42,
  "comment_count": 4,
  "views": 0,
  "content": "<p>For this competition we are provided with <code>feature_definitions.csv</code> file, that contains features and their descriptions. While doing some feature engineering, I found it useful to add two more columns to that table:</p>\n<ul>\n<li><code>dtype</code>, type to which a feature should be resolved; </li>\n<li><code>tables</code>, a list of tables where a particular feature could be encountered.</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Fd7c302717ca90f526fd3017b4161bf0a%2FScreenshot%202024-02-10%20at%204.43.10%20AM.png?generation=1707569005681360&amp;alt=media\"></p>\n<p><code>dtype</code> is pretty straightforward for <code>P</code>, <code>A</code>, <code>M</code> and <code>D</code> features, however, for <code>T</code> and <code>L</code> it should be deduced from the actual data. In addition, there is a feature called <code>score_940</code> that has a description, but couldn't be found in the training dataset. </p>\n<p>Here is a bar chart that summarizes dtypes analysis, the \"Missing\" dtype is the <code>score_940</code> feature. </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Ff57f8e53dd8685e4e5afc6971a0b4873%2FScreenshot%202024-02-10%20at%205.17.50%20AM.png?generation=1707571091623992&amp;alt=media\"></p>\n<p>I've uploaded the enhanced feature definitions as a <a href=\"https://www.kaggle.com/datasets/kononenko/home-credit-enhanced-feature-definitions\" target=\"_blank\">Parquet file</a> to kaggle datasets, feel free to use it. Hope you will find it helpful. </p>",
  "messages": [
    {
      "id": 2645755,
      "postDate": "2024-02-10T12:51:09.393Z",
      "content": "<p>For this competition we are provided with <code>feature_definitions.csv</code> file, that contains features and their descriptions. While doing some feature engineering, I found it useful to add two more columns to that table:</p>\n<ul>\n<li><code>dtype</code>, type to which a feature should be resolved; </li>\n<li><code>tables</code>, a list of tables where a particular feature could be encountered.</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Fd7c302717ca90f526fd3017b4161bf0a%2FScreenshot%202024-02-10%20at%204.43.10%20AM.png?generation=1707569005681360&amp;alt=media\"></p>\n<p><code>dtype</code> is pretty straightforward for <code>P</code>, <code>A</code>, <code>M</code> and <code>D</code> features, however, for <code>T</code> and <code>L</code> it should be deduced from the actual data. In addition, there is a feature called <code>score_940</code> that has a description, but couldn't be found in the training dataset. </p>\n<p>Here is a bar chart that summarizes dtypes analysis, the \"Missing\" dtype is the <code>score_940</code> feature. </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Ff57f8e53dd8685e4e5afc6971a0b4873%2FScreenshot%202024-02-10%20at%205.17.50%20AM.png?generation=1707571091623992&amp;alt=media\"></p>\n<p>I've uploaded the enhanced feature definitions as a <a href=\"https://www.kaggle.com/datasets/kononenko/home-credit-enhanced-feature-definitions\" target=\"_blank\">Parquet file</a> to kaggle datasets, feel free to use it. Hope you will find it helpful. </p>",
      "rawMarkdown": "For this competition we are provided with `feature_definitions.csv` file, that contains features and their descriptions. While doing some feature engineering, I found it useful to add two more columns to that table:\n- `dtype`, type to which a feature should be resolved; \n- `tables`, a list of tables where a particular feature could be encountered.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Fd7c302717ca90f526fd3017b4161bf0a%2FScreenshot%202024-02-10%20at%204.43.10%20AM.png?generation=1707569005681360&alt=media)\n\n`dtype` is pretty straightforward for `P`, `A`, `M` and `D` features, however, for `T` and `L` it should be deduced from the actual data. In addition, there is a feature called `score_940` that has a description, but couldn't be found in the training dataset. \n\nHere is a bar chart that summarizes dtypes analysis, the \"Missing\" dtype is the `score_940` feature. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Ff57f8e53dd8685e4e5afc6971a0b4873%2FScreenshot%202024-02-10%20at%205.17.50%20AM.png?generation=1707571091623992&alt=media)\n\nI've uploaded the enhanced feature definitions as a [Parquet file](https://www.kaggle.com/datasets/kononenko/home-credit-enhanced-feature-definitions) to kaggle datasets, feel free to use it. Hope you will find it helpful. ",
      "votes": 42
    },
    {
      "id": 2690529,
      "postDate": "2024-03-10T16:12:26.157Z",
      "content": "<p>Thanks, very helpful! Based on this answer <a href=\"https://money.stackexchange.com/a/142747\" target=\"_blank\">https://money.stackexchange.com/a/142747</a>, my guess regarding <code>score_940</code> is that it is most likely a binary indicator showing if a person's credit rating is at least 940, so it makes sense it's not available.</p>",
      "rawMarkdown": "Thanks, very helpful! Based on this answer https://money.stackexchange.com/a/142747, my guess regarding `score_940` is that it is most likely a binary indicator showing if a person's credit rating is at least 940, so it makes sense it's not available.",
      "votes": 2,
      "replies": [
        {
          "id": 2690770,
          "postDate": "2024-03-10T19:16:15.790Z",
          "content": "<p>Interesting, I guess you are right. The host actually <a href=\"https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478491#2662440\" target=\"_blank\">said</a> they decided not to include it at some point, so this feature is an artifact.</p>",
          "rawMarkdown": "Interesting, I guess you are right. The host actually [said](https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478491#2662440) they decided not to include it at some point, so this feature is an artifact."
        }
      ]
    },
    {
      "id": 2725987,
      "postDate": "2024-03-31T21:36:53.073Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 2648315,
      "postDate": "2024-02-12T08:02:04.617Z",
      "content": "<p>Thank you very much it is helpfull</p>",
      "rawMarkdown": "Thank you very much it is helpfull",
      "votes": 2
    }
  ],
  "comments": [
    {
      "id": 2690529,
      "author_name": "Antonio Šajatović",
      "author_url": "",
      "post_date": "2024-03-10T16:12:26.157000",
      "content": "<p>Thanks, very helpful! Based on this answer <a href=\"https://money.stackexchange.com/a/142747\" target=\"_blank\">https://money.stackexchange.com/a/142747</a>, my guess regarding <code>score_940</code> is that it is most likely a binary indicator showing if a person's credit rating is at least 940, so it makes sense it's not available.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2690770,
          "author_name": "Oleksiy Kononenko",
          "author_url": "",
          "post_date": "2024-03-10T19:16:15.790000",
          "content": "<p>Interesting, I guess you are right. The host actually <a href=\"https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/478491#2662440\" target=\"_blank\">said</a> they decided not to include it at some point, so this feature is an artifact.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2725987,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-03-31T21:36:53.073000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2648315,
      "author_name": "LucaMTB",
      "author_url": "",
      "post_date": "2024-02-12T08:02:04.617000",
      "content": "<p>Thank you very much it is helpfull</p>",
      "votes": 2,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2645755": "For this competition we are provided with `feature_definitions.csv` file, that contains features and their descriptions. While doing some feature engineering, I found it useful to add two more columns to that table:\n- `dtype`, type to which a feature should be resolved; \n- `tables`, a list of tables where a particular feature could be encountered.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Fd7c302717ca90f526fd3017b4161bf0a%2FScreenshot%202024-02-10%20at%204.43.10%20AM.png?generation=1707569005681360&alt=media)\n\n`dtype` is pretty straightforward for `P`, `A`, `M` and `D` features, however, for `T` and `L` it should be deduced from the actual data. In addition, there is a feature called `score_940` that has a description, but couldn't be found in the training dataset. \n\nHere is a bar chart that summarizes dtypes analysis, the \"Missing\" dtype is the `score_940` feature. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2099265%2Ff57f8e53dd8685e4e5afc6971a0b4873%2FScreenshot%202024-02-10%20at%205.17.50%20AM.png?generation=1707571091623992&alt=media)\n\nI've uploaded the enhanced feature definitions as a [Parquet file](https://www.kaggle.com/datasets/kononenko/home-credit-enhanced-feature-definitions) to kaggle datasets, feel free to use it. Hope you will find it helpful. ",
    "2690529": "Thanks, very helpful! Based on this answer https://money.stackexchange.com/a/142747, my guess regarding `score_940` is that it is most likely a binary indicator showing if a person's credit rating is at least 940, so it makes sense it's not available.",
    "2725987": "",
    "2648315": "Thank you very much it is helpfull"
  }
}