{
  "id": 341171,
  "title": "The Default Rate Captured at 4%",
  "url": "/competitions/amex-default-prediction/discussion/341171",
  "author_name": "Yuga",
  "post_date": "2022-08-01T15:28:23.044000",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I have read few discussions relating to evaluation metric. I understand about first one (normalized AUC) but I am still confused about the second one, the default rate captured at 4%.</p>\n<p>I understand the definition of D.</p>\n<p>D = (#default captured by 4% threshold) / (#default)</p>\n<p>Given the fact that the metric has maximum value of 1, maximum value for D is 1.</p>\n<p>Even if we have test label, it is only achievable iff</p>\n<p>(#default) &gt; (#non_default * 20 + #default) * 0.04</p>\n<p>In training data, proportion of default is only 1.7% so it is impossible to achieve D=1 at 4%.</p>\n<p>Does this mean test data has at least more than 4% of default after weighting applied?</p>",
  "messages": [
    {
      "id": 1880281,
      "postDate": "2022-08-01T15:28:23.043Z",
      "content": "<p>I have read few discussions relating to evaluation metric. I understand about first one (normalized AUC) but I am still confused about the second one, the default rate captured at 4%.</p>\n<p>I understand the definition of D.</p>\n<p>D = (#default captured by 4% threshold) / (#default)</p>\n<p>Given the fact that the metric has maximum value of 1, maximum value for D is 1.</p>\n<p>Even if we have test label, it is only achievable iff</p>\n<p>(#default) &gt; (#non_default * 20 + #default) * 0.04</p>\n<p>In training data, proportion of default is only 1.7% so it is impossible to achieve D=1 at 4%.</p>\n<p>Does this mean test data has at least more than 4% of default after weighting applied?</p>",
      "rawMarkdown": "I have read few discussions relating to evaluation metric. I understand about first one (normalized AUC) but I am still confused about the second one, the default rate captured at 4%.\n\nI understand the definition of D.\n\nD = (#default captured by 4% threshold) / (#default)\n\nGiven the fact that the metric has maximum value of 1, maximum value for D is 1.\n\nEven if we have test label, it is only achievable iff\n\n(#default) > (#non_default * 20 + #default) * 0.04\n\nIn training data, proportion of default is only 1.7% so it is impossible to achieve D=1 at 4%.\n\nDoes this mean test data has at least more than 4% of default after weighting applied?\n\n",
      "votes": 1
    },
    {
      "id": 1880402,
      "postDate": "2022-08-01T17:29:45.487Z",
      "content": "<p>From the Data Description</p>\n<blockquote>\n  <p>Note that the negative class has been subsampled for this dataset at 5%, and thus receives a 20x weighting in the scoring metric.</p>\n</blockquote>",
      "rawMarkdown": "From the Data Description\n>  Note that the negative class has been subsampled for this dataset at 5%, and thus receives a 20x weighting in the scoring metric.",
      "replies": [
        {
          "id": 1880413,
          "postDate": "2022-08-01T17:38:58.020Z",
          "content": "<p>I think if you weight negative class in training data, default rate will be less than 4% hence it is impossible to achieve D=1 at 4%.</p>",
          "rawMarkdown": "I think if you weight negative class in training data, default rate will be less than 4% hence it is impossible to achieve D=1 at 4%."
        }
      ]
    },
    {
      "id": 1880409,
      "postDate": "2022-08-01T17:38:21.507Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1880402,
      "author_name": "CucaSF",
      "author_url": "",
      "post_date": "2022-08-01T17:29:45.487000",
      "content": "<p>From the Data Description</p>\n<blockquote>\n  <p>Note that the negative class has been subsampled for this dataset at 5%, and thus receives a 20x weighting in the scoring metric.</p>\n</blockquote>",
      "votes": 0,
      "replies": [
        {
          "id": 1880413,
          "author_name": "Yuga",
          "author_url": "",
          "post_date": "2022-08-01T17:38:58.020000",
          "content": "<p>I think if you weight negative class in training data, default rate will be less than 4% hence it is impossible to achieve D=1 at 4%.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1880409,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-08-01T17:38:21.507000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1880281": "I have read few discussions relating to evaluation metric. I understand about first one (normalized AUC) but I am still confused about the second one, the default rate captured at 4%.\n\nI understand the definition of D.\n\nD = (#default captured by 4% threshold) / (#default)\n\nGiven the fact that the metric has maximum value of 1, maximum value for D is 1.\n\nEven if we have test label, it is only achievable iff\n\n(#default) > (#non_default * 20 + #default) * 0.04\n\nIn training data, proportion of default is only 1.7% so it is impossible to achieve D=1 at 4%.\n\nDoes this mean test data has at least more than 4% of default after weighting applied?\n\n",
    "1880402": "From the Data Description\n>  Note that the negative class has been subsampled for this dataset at 5%, and thus receives a 20x weighting in the scoring metric.",
    "1880409": ""
  }
}