{
  "id": 475669,
  "title": "Is covid really impact that much?",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/475669",
  "author_name": "",
  "post_date": "2024-02-09T09:57:11.039510400Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I do not believe that COVID will significantly impact the model's AUC (Area Under the Curve) performance. Based on the evidence, <strong>training the model exclusively on data from pre-COVID times does not seem to deteriorate its performance on test sets from during the COVID era.</strong></p>\n<p>Must be something I missed.🙃</p>",
  "messages": [
    {
      "id": "2644152",
      "postDate": "02/09/2024 09:57:11",
      "content": "<p>I do not believe that COVID will significantly impact the model's AUC (Area Under the Curve) performance. Based on the evidence, <strong>training the model exclusively on data from pre-COVID times does not seem to deteriorate its performance on test sets from during the COVID era.</strong></p>\n<p>Must be something I missed.🙃</p>",
      "rawMarkdown": "I do not believe that COVID will significantly impact the model's AUC (Area Under the Curve) performance. Based on the evidence, **training the model exclusively on data from pre-COVID times does not seem to deteriorate its performance on test sets from during the COVID era.**\n\nMust be something I missed.🙃",
      "votes": null
    },
    {
      "id": "2666550",
      "postDate": "02/24/2024 12:53:08",
      "content": "<p>I guess COVID outbreak significantly affects performance, especially from April 2020 and beyond. Can you, please, provide evidence on both training and test sets AUC? </p>",
      "rawMarkdown": "I guess COVID outbreak significantly affects performance, especially from April 2020 and beyond. Can you, please, provide evidence on both training and test sets AUC?",
      "votes": null
    },
    {
      "id": "2672202",
      "postDate": "02/28/2024 01:04:16",
      "content": "<p>covid period is expected to have higher ratio of default  because people were stood down or lost their jobs… so it is expected that their ability to serve a loan approved before covid has decreased.  The question is whether the difference is significant from model gain / feature importance point of view in order to justify the effort of covid related FE (for example hospitality sector and travel were hit badly but telecom sector flourished due to remote work) So even with obfuscated categorial  employment  data, some sectors would have higher default ratios .</p>\n<p>I had some FE work in this area but I think I will discard most of it given the recent announcement here:<br>\n<a href=\"https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477074#2653249\" target=\"_blank\">https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477074#2653249</a></p>",
      "rawMarkdown": "covid period is expected to have higher ratio of default  because people were stood down or lost their jobs... so it is expected that their ability to serve a loan approved before covid has decreased.  The question is whether the difference is significant from model gain / feature importance point of view in order to justify the effort of covid related FE (for example hospitality sector and travel were hit badly but telecom sector flourished due to remote work) So even with obfuscated categorial  employment  data, some sectors would have higher default ratios .\n\nI had some FE work in this area but I think I will discard most of it given the recent announcement here:\nhttps://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477074#2653249",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2666550,
      "author_name": "ablaydosmaganbetov",
      "author_url": "",
      "post_date": "02/24/2024 12:53:08",
      "content": "<p>I guess COVID outbreak significantly affects performance, especially from April 2020 and beyond. Can you, please, provide evidence on both training and test sets AUC? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2672202,
      "author_name": "fadylabib",
      "author_url": "",
      "post_date": "02/28/2024 01:04:16",
      "content": "<p>covid period is expected to have higher ratio of default  because people were stood down or lost their jobs… so it is expected that their ability to serve a loan approved before covid has decreased.  The question is whether the difference is significant from model gain / feature importance point of view in order to justify the effort of covid related FE (for example hospitality sector and travel were hit badly but telecom sector flourished due to remote work) So even with obfuscated categorial  employment  data, some sectors would have higher default ratios .</p>\n<p>I had some FE work in this area but I think I will discard most of it given the recent announcement here:<br>\n<a href=\"https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477074#2653249\" target=\"_blank\">https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477074#2653249</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2644152": "I do not believe that COVID will significantly impact the model's AUC (Area Under the Curve) performance. Based on the evidence, **training the model exclusively on data from pre-COVID times does not seem to deteriorate its performance on test sets from during the COVID era.**\n\nMust be something I missed.🙃",
    "2666550": "I guess COVID outbreak significantly affects performance, especially from April 2020 and beyond. Can you, please, provide evidence on both training and test sets AUC?",
    "2672202": "covid period is expected to have higher ratio of default  because people were stood down or lost their jobs... so it is expected that their ability to serve a loan approved before covid has decreased.  The question is whether the difference is significant from model gain / feature importance point of view in order to justify the effort of covid related FE (for example hospitality sector and travel were hit badly but telecom sector flourished due to remote work) So even with obfuscated categorial  employment  data, some sectors would have higher default ratios .\n\nI had some FE work in this area but I think I will discard most of it given the recent announcement here:\nhttps://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/discussion/477074#2653249"
  },
  "source": "meta"
}