{
  "id": 347141,
  "title": "How is Amex Metric Score Working?",
  "url": "/competitions/amex-default-prediction/discussion/347141",
  "author_name": "",
  "post_date": "2022-08-23T03:46:54.752861Z",
  "votes": 1,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Sorry if it sounds silly but I tried experimenting with the dataset, EDA and ML models and below are the outcomes of some of them -</p>\n<ol>\n<li>Amex Metric Score - 0.54, Kaggle Public Leaderboard Score - 0.73</li>\n<li>Amex Metric Score - 0.61, Kaggle Public Leaderboard Score - 0.74</li>\n<li>Amex Metric Score - 0.74, Kaggle LB Score - 0.774</li>\n<li>Amex Metric Score - 0.82, Kaggle LB Score - 0.65</li>\n<li>Amex Metric Score - 0.87, Kaggle LB Score - 0.58</li>\n</ol>\n<p>Why with increasing AmEx metric score, leaderboard score is falling down?</p>",
  "messages": [
    {
      "id": "1909924",
      "postDate": "08/23/2022 03:46:54",
      "content": "<p>Sorry if it sounds silly but I tried experimenting with the dataset, EDA and ML models and below are the outcomes of some of them -</p>\n<ol>\n<li>Amex Metric Score - 0.54, Kaggle Public Leaderboard Score - 0.73</li>\n<li>Amex Metric Score - 0.61, Kaggle Public Leaderboard Score - 0.74</li>\n<li>Amex Metric Score - 0.74, Kaggle LB Score - 0.774</li>\n<li>Amex Metric Score - 0.82, Kaggle LB Score - 0.65</li>\n<li>Amex Metric Score - 0.87, Kaggle LB Score - 0.58</li>\n</ol>\n<p>Why with increasing AmEx metric score, leaderboard score is falling down?</p>",
      "rawMarkdown": "Sorry if it sounds silly but I tried experimenting with the dataset, EDA and ML models and below are the outcomes of some of them -\n\n1. Amex Metric Score - 0.54, Kaggle Public Leaderboard Score - 0.73\n2. Amex Metric Score - 0.61, Kaggle Public Leaderboard Score - 0.74\n2. Amex Metric Score - 0.74, Kaggle LB Score - 0.774\n3. Amex Metric Score - 0.82, Kaggle LB Score - 0.65\n4. Amex Metric Score - 0.87, Kaggle LB Score - 0.58\n\nWhy with increasing AmEx metric score, leaderboard score is falling down?",
      "votes": null
    },
    {
      "id": "1909992",
      "postDate": "08/23/2022 05:32:48",
      "content": "<p>You are dealing with overfitting/underfitting </p>",
      "rawMarkdown": "You are dealing with overfitting/underfitting",
      "votes": null
    },
    {
      "id": "1909998",
      "postDate": "08/23/2022 05:39:26",
      "content": "<p>No, all these models are neither underfitting nor overfitting.<br>\nChecked with train and test accuracy.</p>",
      "rawMarkdown": "No, all these models are neither underfitting nor overfitting.\nChecked with train and test accuracy.",
      "votes": null
    },
    {
      "id": "1910102",
      "postDate": "08/23/2022 07:14:06",
      "content": "<p>Guess you know best since you’re in 4000th place lol</p>\n<p>This is the 2nd post you’ve made where people tell you you’re overfitting and you still think you’re not..</p>\n<p>Incase you haven’t noticed, the highest score on the leaderboard is 0.802, so when you see 0.870 is your alarm not ringing?</p>",
      "rawMarkdown": "Guess you know best since you’re in 4000th place lol\n\nThis is the 2nd post you’ve made where people tell you you’re overfitting and you still think you’re not..\n\nIncase you haven’t noticed, the highest score on the leaderboard is 0.802, so when you see 0.870 is your alarm not ringing?",
      "votes": null
    },
    {
      "id": "1910134",
      "postDate": "08/23/2022 07:51:41",
      "content": "<p>Thank you for the response, I am unable to figure out that's why I am asking the experts here.</p>\n<p>Can you please guide me on how to check overfitting apart from checking the difference between train and test scores?</p>\n<p>Amex Metric Score - 0.82 (Test) 0.86 (Train) , Kaggle LB Score - 0.65<br>\n'target' is not present in training data.</p>\n<p>Thanks in advance. <a href=\"https://www.kaggle.com/julianmukaj\" target=\"_blank\">@julianmukaj</a> </p>",
      "rawMarkdown": "Thank you for the response, I am unable to figure out that's why I am asking the experts here.\n\nCan you please guide me on how to check overfitting apart from checking the difference between train and test scores?\n\nAmex Metric Score - 0.82 (Test) 0.86 (Train) , Kaggle LB Score - 0.65\n'target' is not present in training data.\n\nThanks in advance. @julianmukaj",
      "votes": null
    },
    {
      "id": "1910264",
      "postDate": "08/23/2022 09:50:08",
      "content": "<p>You need to show us some code to properly investigate, can be many reasons causing it</p>\n<p>Did you do grouby on customer_ID or are you using the full data for starters? Try doing CV instead of test_train_split also</p>\n<blockquote>\n  <p>Can you please guide me on how to check overfitting apart from checking the difference between train and test scores?</p>\n</blockquote>\n<p>The most obvious tell should be, you get 0.87 locally yet 0.5 on the leaderboard..</p>\n<p>Maybe just continue discussion on your old topic: <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/346617\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/346617</a></p>",
      "rawMarkdown": "You need to show us some code to properly investigate, can be many reasons causing it\n\nDid you do grouby on customer_ID or are you using the full data for starters? Try doing CV instead of test_train_split also\n\n>Can you please guide me on how to check overfitting apart from checking the difference between train and test scores?\n\nThe most obvious tell should be, you get 0.87 locally yet 0.5 on the leaderboard..\n\nMaybe just continue discussion on your old topic: https://www.kaggle.com/competitions/amex-default-prediction/discussion/346617",
      "votes": null
    },
    {
      "id": "1910375",
      "postDate": "08/23/2022 11:32:05",
      "content": "<p>Yes, I did group by on customer id and took the last row of each customer.<br>\nSure, will try CV instead of test_train_split.</p>",
      "rawMarkdown": "Yes, I did group by on customer id and took the last row of each customer.\nSure, will try CV instead of test_train_split.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1909992,
      "author_name": "abysswalker1994",
      "author_url": "",
      "post_date": "08/23/2022 05:32:48",
      "content": "<p>You are dealing with overfitting/underfitting </p>",
      "votes": null,
      "replies": [
        {
          "id": 1909998,
          "author_name": "neha845",
          "author_url": "",
          "post_date": "08/23/2022 05:39:26",
          "content": "<p>No, all these models are neither underfitting nor overfitting.<br>\nChecked with train and test accuracy.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1910102,
          "author_name": "julianmukaj",
          "author_url": "",
          "post_date": "08/23/2022 07:14:06",
          "content": "<p>Guess you know best since you’re in 4000th place lol</p>\n<p>This is the 2nd post you’ve made where people tell you you’re overfitting and you still think you’re not..</p>\n<p>Incase you haven’t noticed, the highest score on the leaderboard is 0.802, so when you see 0.870 is your alarm not ringing?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1910134,
          "author_name": "neha845",
          "author_url": "",
          "post_date": "08/23/2022 07:51:41",
          "content": "<p>Thank you for the response, I am unable to figure out that's why I am asking the experts here.</p>\n<p>Can you please guide me on how to check overfitting apart from checking the difference between train and test scores?</p>\n<p>Amex Metric Score - 0.82 (Test) 0.86 (Train) , Kaggle LB Score - 0.65<br>\n'target' is not present in training data.</p>\n<p>Thanks in advance. <a href=\"https://www.kaggle.com/julianmukaj\" target=\"_blank\">@julianmukaj</a> </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1910264,
          "author_name": "julianmukaj",
          "author_url": "",
          "post_date": "08/23/2022 09:50:08",
          "content": "<p>You need to show us some code to properly investigate, can be many reasons causing it</p>\n<p>Did you do grouby on customer_ID or are you using the full data for starters? Try doing CV instead of test_train_split also</p>\n<blockquote>\n  <p>Can you please guide me on how to check overfitting apart from checking the difference between train and test scores?</p>\n</blockquote>\n<p>The most obvious tell should be, you get 0.87 locally yet 0.5 on the leaderboard..</p>\n<p>Maybe just continue discussion on your old topic: <a href=\"https://www.kaggle.com/competitions/amex-default-prediction/discussion/346617\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/discussion/346617</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1910375,
          "author_name": "neha845",
          "author_url": "",
          "post_date": "08/23/2022 11:32:05",
          "content": "<p>Yes, I did group by on customer id and took the last row of each customer.<br>\nSure, will try CV instead of test_train_split.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1909924": "Sorry if it sounds silly but I tried experimenting with the dataset, EDA and ML models and below are the outcomes of some of them -\n\n1. Amex Metric Score - 0.54, Kaggle Public Leaderboard Score - 0.73\n2. Amex Metric Score - 0.61, Kaggle Public Leaderboard Score - 0.74\n2. Amex Metric Score - 0.74, Kaggle LB Score - 0.774\n3. Amex Metric Score - 0.82, Kaggle LB Score - 0.65\n4. Amex Metric Score - 0.87, Kaggle LB Score - 0.58\n\nWhy with increasing AmEx metric score, leaderboard score is falling down?",
    "1909992": "You are dealing with overfitting/underfitting",
    "1909998": "No, all these models are neither underfitting nor overfitting.\nChecked with train and test accuracy.",
    "1910102": "Guess you know best since you’re in 4000th place lol\n\nThis is the 2nd post you’ve made where people tell you you’re overfitting and you still think you’re not..\n\nIncase you haven’t noticed, the highest score on the leaderboard is 0.802, so when you see 0.870 is your alarm not ringing?",
    "1910134": "Thank you for the response, I am unable to figure out that's why I am asking the experts here.\n\nCan you please guide me on how to check overfitting apart from checking the difference between train and test scores?\n\nAmex Metric Score - 0.82 (Test) 0.86 (Train) , Kaggle LB Score - 0.65\n'target' is not present in training data.\n\nThanks in advance. @julianmukaj",
    "1910264": "You need to show us some code to properly investigate, can be many reasons causing it\n\nDid you do grouby on customer_ID or are you using the full data for starters? Try doing CV instead of test_train_split also\n\n>Can you please guide me on how to check overfitting apart from checking the difference between train and test scores?\n\nThe most obvious tell should be, you get 0.87 locally yet 0.5 on the leaderboard..\n\nMaybe just continue discussion on your old topic: https://www.kaggle.com/competitions/amex-default-prediction/discussion/346617",
    "1910375": "Yes, I did group by on customer id and took the last row of each customer.\nSure, will try CV instead of test_train_split."
  },
  "source": "meta"
}