{
  "id": 332574,
  "title": "Towards data de-anonymization",
  "url": "/competitions/amex-default-prediction/discussion/332574",
  "author_name": "raddar",
  "post_date": "2022-06-22T11:37:17.923000",
  "votes": 71,
  "comment_count": 11,
  "views": 0,
  "content": "<p>I think many of you already consider that P_2 is some kind of internal AMEX credit rating, indicated by its very high AUC and normal score distribution.</p>\n<p>But what if I told you there can be more discovered? I focused on one specific feature - <code>D_39</code>. This has caught my attention for a while and I was able to finally figure it out! This is actually <code>target</code> related feature! More on that you can read in my exploratory notebook:</p>\n<p><a href=\"https://www.kaggle.com/code/raddar/deanonymized-days-overdue-feat-amex\" target=\"_blank\">https://www.kaggle.com/code/raddar/deanonymized-days-overdue-feat-amex</a></p>\n<p>Hope you like it :)</p>",
  "messages": [
    {
      "id": 1829166,
      "postDate": "2022-06-22T11:37:17.923Z",
      "content": "<p>I think many of you already consider that P_2 is some kind of internal AMEX credit rating, indicated by its very high AUC and normal score distribution.</p>\n<p>But what if I told you there can be more discovered? I focused on one specific feature - <code>D_39</code>. This has caught my attention for a while and I was able to finally figure it out! This is actually <code>target</code> related feature! More on that you can read in my exploratory notebook:</p>\n<p><a href=\"https://www.kaggle.com/code/raddar/deanonymized-days-overdue-feat-amex\" target=\"_blank\">https://www.kaggle.com/code/raddar/deanonymized-days-overdue-feat-amex</a></p>\n<p>Hope you like it :)</p>",
      "rawMarkdown": "I think many of you already consider that P_2 is some kind of internal AMEX credit rating, indicated by its very high AUC and normal score distribution.\n\nBut what if I told you there can be more discovered? I focused on one specific feature - `D_39`. This has caught my attention for a while and I was able to finally figure it out! This is actually `target` related feature! More on that you can read in my exploratory notebook:\n\nhttps://www.kaggle.com/code/raddar/deanonymized-days-overdue-feat-amex\n\nHope you like it :)",
      "votes": 71
    },
    {
      "id": 1829174,
      "postDate": "2022-06-22T11:49:04.040Z",
      "content": "<p>This also reminds me when I worked in credit risk industry and had to construct similar dataset. I used to put credit rating in first position and days overdue in the second position in the dataset - this must be a pattern among people who prepares such datasets :D</p>",
      "rawMarkdown": "This also reminds me when I worked in credit risk industry and had to construct similar dataset. I used to put credit rating in first position and days overdue in the second position in the dataset - this must be a pattern among people who prepares such datasets :D",
      "votes": 18,
      "replies": [
        {
          "id": 1829231,
          "postDate": "2022-06-22T13:11:54.310Z",
          "content": "<p>Very interesting insight!</p>",
          "rawMarkdown": "Very interesting insight!",
          "votes": 1
        },
        {
          "id": 1832497,
          "postDate": "2022-06-25T05:21:23.563Z",
          "content": "<p>I also work in the financial Industry but I wonder, we try to predict default prob, but the credit score is basically that. Do you think its the Initial rating?</p>",
          "rawMarkdown": "I also work in the financial Industry but I wonder, we try to predict default prob, but the credit score is basically that. Do you think its the Initial rating?"
        },
        {
          "id": 1832622,
          "postDate": "2022-06-25T08:08:37.147Z",
          "content": "<p>I think that organizer wants to improve its internal rating (P_2).</p>",
          "rawMarkdown": "I think that organizer wants to improve its internal rating (P_2).",
          "votes": 5
        }
      ]
    },
    {
      "id": 1829273,
      "postDate": "2022-06-22T13:39:53.783Z",
      "content": "<p>I got this from the first viz of <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>. D_105 looks like a 'days since incident' feature, B_20 like a proportion of distressed amount. </p>",
      "rawMarkdown": "I got this from the first viz of @cdeotte. D_105 looks like a 'days since incident' feature, B_20 like a proportion of distressed amount. ",
      "votes": 8
    },
    {
      "id": 1832643,
      "postDate": "2022-06-25T08:53:42.923Z",
      "content": "<p>Thank you for sharing <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a> ! Learned a lot</p>",
      "rawMarkdown": "Thank you for sharing @raddar ! Learned a lot",
      "votes": 1
    },
    {
      "id": 1829736,
      "postDate": "2022-06-22T21:57:41.010Z",
      "content": "<p>you may want to check this:<br>\n<a href=\"https://www.kaggle.com/datasets/pradip11/amexpert-codelab-2021\" target=\"_blank\">https://www.kaggle.com/datasets/pradip11/amexpert-codelab-2021</a></p>",
      "rawMarkdown": "you may want to check this:\nhttps://www.kaggle.com/datasets/pradip11/amexpert-codelab-2021",
      "votes": 2,
      "replies": [
        {
          "id": 1829738,
          "postDate": "2022-06-22T22:06:59.167Z",
          "content": "<p>Very disturbing to see this not anonymized dataset… I wonder if he got permission to publish it </p>",
          "rawMarkdown": "Very disturbing to see this not anonymized dataset... I wonder if he got permission to publish it ",
          "votes": 7
        }
      ]
    },
    {
      "id": 1878823,
      "postDate": "2022-07-31T16:48:55.967Z",
      "rawMarkdown": "",
      "votes": 1,
      "isDeleted": true,
      "replies": [
        {
          "id": 1879045,
          "postDate": "2022-07-31T20:23:03.960Z",
          "content": "<p>I have worked in credit scoring before. the shape of distribution and target rate indicates this. And it has the largest AUC score of all the features - which itself indicates that it is an aggregate of other features (aggregate=credit risk model in this case)</p>",
          "rawMarkdown": "I have worked in credit scoring before. the shape of distribution and target rate indicates this. And it has the largest AUC score of all the features - which itself indicates that it is an aggregate of other features (aggregate=credit risk model in this case)",
          "votes": 4
        },
        {
          "id": 1879298,
          "postDate": "2022-08-01T01:32:43.413Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1829174,
      "author_name": "raddar",
      "author_url": "",
      "post_date": "2022-06-22T11:49:04.040000",
      "content": "<p>This also reminds me when I worked in credit risk industry and had to construct similar dataset. I used to put credit rating in first position and days overdue in the second position in the dataset - this must be a pattern among people who prepares such datasets :D</p>",
      "votes": 18,
      "replies": [
        {
          "id": 1829231,
          "author_name": "Ali Abdin",
          "author_url": "",
          "post_date": "2022-06-22T13:11:54.310000",
          "content": "<p>Very interesting insight!</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1832497,
          "author_name": "Art Vandelay",
          "author_url": "",
          "post_date": "2022-06-25T05:21:23.563000",
          "content": "<p>I also work in the financial Industry but I wonder, we try to predict default prob, but the credit score is basically that. Do you think its the Initial rating?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1832622,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-06-25T08:08:37.147000",
          "content": "<p>I think that organizer wants to improve its internal rating (P_2).</p>",
          "votes": 5,
          "replies": []
        }
      ]
    },
    {
      "id": 1829273,
      "author_name": "Lucas Morin",
      "author_url": "",
      "post_date": "2022-06-22T13:39:53.783000",
      "content": "<p>I got this from the first viz of <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>. D_105 looks like a 'days since incident' feature, B_20 like a proportion of distressed amount. </p>",
      "votes": 8,
      "replies": []
    },
    {
      "id": 1832643,
      "author_name": "Making TARS",
      "author_url": "",
      "post_date": "2022-06-25T08:53:42.923000",
      "content": "<p>Thank you for sharing <a href=\"https://www.kaggle.com/raddar\" target=\"_blank\">@raddar</a> ! Learned a lot</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1829736,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2022-06-22T21:57:41.010000",
      "content": "<p>you may want to check this:<br>\n<a href=\"https://www.kaggle.com/datasets/pradip11/amexpert-codelab-2021\" target=\"_blank\">https://www.kaggle.com/datasets/pradip11/amexpert-codelab-2021</a></p>",
      "votes": 2,
      "replies": [
        {
          "id": 1829738,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-06-22T22:06:59.167000",
          "content": "<p>Very disturbing to see this not anonymized dataset… I wonder if he got permission to publish it </p>",
          "votes": 7,
          "replies": []
        }
      ]
    },
    {
      "id": 1878823,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-07-31T16:48:55.967000",
      "content": "",
      "votes": 1,
      "replies": [
        {
          "id": 1879045,
          "author_name": "raddar",
          "author_url": "",
          "post_date": "2022-07-31T20:23:03.960000",
          "content": "<p>I have worked in credit scoring before. the shape of distribution and target rate indicates this. And it has the largest AUC score of all the features - which itself indicates that it is an aggregate of other features (aggregate=credit risk model in this case)</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1879298,
          "author_name": "",
          "author_url": "",
          "post_date": "2022-08-01T01:32:43.413000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1829166": "I think many of you already consider that P_2 is some kind of internal AMEX credit rating, indicated by its very high AUC and normal score distribution.\n\nBut what if I told you there can be more discovered? I focused on one specific feature - `D_39`. This has caught my attention for a while and I was able to finally figure it out! This is actually `target` related feature! More on that you can read in my exploratory notebook:\n\nhttps://www.kaggle.com/code/raddar/deanonymized-days-overdue-feat-amex\n\nHope you like it :)",
    "1829174": "This also reminds me when I worked in credit risk industry and had to construct similar dataset. I used to put credit rating in first position and days overdue in the second position in the dataset - this must be a pattern among people who prepares such datasets :D",
    "1829273": "I got this from the first viz of @cdeotte. D_105 looks like a 'days since incident' feature, B_20 like a proportion of distressed amount. ",
    "1832643": "Thank you for sharing @raddar ! Learned a lot",
    "1829736": "you may want to check this:\nhttps://www.kaggle.com/datasets/pradip11/amexpert-codelab-2021",
    "1878823": ""
  }
}