{
  "id": 501256,
  "title": "Has anyone tried  some dl models this competition?",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/501256",
  "author_name": "",
  "post_date": "2024-05-08T16:06:25.150675400Z",
  "votes": 3,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hello everyone,<br>\nIt seems that tree-based models still outperform deep learning on tabular data, but we might also want to try some new models,  such as TabR - A simple feed-forward network with k-Nearest-Neighbors. <a href=\"url\" target=\"_blank\">https://github.com/yandex-research/tabular-dl-tabr?tab=readme-ov-file</a><br>\nUnfortunately, I tried it but failed. It's just so new and niche that it caused me to not find codes about it in the Kaggle community. Has anyone else tried this or any other deep learning model ? I'd like to learn more with the use of deep learning in this game,  Thanks so much for the discussion and advice!</p>",
  "messages": [
    {
      "id": "2801424",
      "postDate": "05/08/2024 16:06:25",
      "content": "<p>Hello everyone,<br>\nIt seems that tree-based models still outperform deep learning on tabular data, but we might also want to try some new models,  such as TabR - A simple feed-forward network with k-Nearest-Neighbors. <a href=\"url\" target=\"_blank\">https://github.com/yandex-research/tabular-dl-tabr?tab=readme-ov-file</a><br>\nUnfortunately, I tried it but failed. It's just so new and niche that it caused me to not find codes about it in the Kaggle community. Has anyone else tried this or any other deep learning model ? I'd like to learn more with the use of deep learning in this game,  Thanks so much for the discussion and advice!</p>",
      "rawMarkdown": "Hello everyone,\nIt seems that tree-based models still outperform deep learning on tabular data, but we might also want to try some new models,  such as TabR - A simple feed-forward network with k-Nearest-Neighbors. [https://github.com/yandex-research/tabular-dl-tabr?tab=readme-ov-file](url)\nUnfortunately, I tried it but failed. It's just so new and niche that it caused me to not find codes about it in the Kaggle community. Has anyone else tried this or any other deep learning model ? I'd like to learn more with the use of deep learning in this game,  Thanks so much for the discussion and advice!",
      "votes": null
    },
    {
      "id": "2802897",
      "postDate": "05/09/2024 08:11:29",
      "content": "<p>Logistic regression was giving 0.5-0.6 roc auc score , sklearn nural network with 100, 50 2 hidden layers was giving even less score … </p>",
      "rawMarkdown": "Logistic regression was giving 0.5-0.6 roc auc score , sklearn nural network with 100, 50 2 hidden layers was giving even less score ...",
      "votes": null
    },
    {
      "id": "2802907",
      "postDate": "05/09/2024 08:20:05",
      "content": "<p>You can get much higher AUC than that if you use logistic regression from sklearn with penalty and transformation/standardization of variables. I tested with quantile transformer. Though the stability on LB was pretty bad.</p>",
      "rawMarkdown": "You can get much higher AUC than that if you use logistic regression from sklearn with penalty and transformation/standardization of variables. I tested with quantile transformer. Though the stability on LB was pretty bad.",
      "votes": null
    },
    {
      "id": "2805241",
      "postDate": "05/10/2024 12:50:09",
      "content": "<p>I did, the highest LB score for my DL model is .545 , I have tested with MLP, GRU, LSTM and ResNet. <br>\nThe LB increase to 0.56-0.57 when I blended it with a tree-based model like CAT and LGBM.<br>\nHowever, till now, the combination models of purely tree-based is still better than having a DL</p>",
      "rawMarkdown": "I did, the highest LB score for my DL model is .545 , I have tested with MLP, GRU, LSTM and ResNet. \nThe LB increase to 0.56-0.57 when I blended it with a tree-based model like CAT and LGBM.\nHowever, till now, the combination models of purely tree-based is still better than having a DL",
      "votes": null
    },
    {
      "id": "2805391",
      "postDate": "05/10/2024 14:24:09",
      "content": "<p>How does your DL perform on CV compared to tree models? Mine didn't add much to an esnemble even on CV.</p>",
      "rawMarkdown": "How does your DL perform on CV compared to tree models? Mine didn't add much to an esnemble even on CV.",
      "votes": null
    },
    {
      "id": "2805634",
      "postDate": "05/10/2024 16:52:12",
      "content": "<p>Nice to hear from you, yeah mine neither. <br>\nInterested! it seems like doing CV with DL doesn't work</p>",
      "rawMarkdown": "Nice to hear from you, yeah mine neither. \nInterested! it seems like doing CV with DL doesn't work",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2802897,
      "author_name": "kunduruanil",
      "author_url": "",
      "post_date": "05/09/2024 08:11:29",
      "content": "<p>Logistic regression was giving 0.5-0.6 roc auc score , sklearn nural network with 100, 50 2 hidden layers was giving even less score … </p>",
      "votes": null,
      "replies": [
        {
          "id": 2802907,
          "author_name": "ern711",
          "author_url": "",
          "post_date": "05/09/2024 08:20:05",
          "content": "<p>You can get much higher AUC than that if you use logistic regression from sklearn with penalty and transformation/standardization of variables. I tested with quantile transformer. Though the stability on LB was pretty bad.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2805241,
      "author_name": "vklinhhh",
      "author_url": "",
      "post_date": "05/10/2024 12:50:09",
      "content": "<p>I did, the highest LB score for my DL model is .545 , I have tested with MLP, GRU, LSTM and ResNet. <br>\nThe LB increase to 0.56-0.57 when I blended it with a tree-based model like CAT and LGBM.<br>\nHowever, till now, the combination models of purely tree-based is still better than having a DL</p>",
      "votes": null,
      "replies": [
        {
          "id": 2805391,
          "author_name": "eivolkova",
          "author_url": "",
          "post_date": "05/10/2024 14:24:09",
          "content": "<p>How does your DL perform on CV compared to tree models? Mine didn't add much to an esnemble even on CV.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2805634,
              "author_name": "vklinhhh",
              "author_url": "",
              "post_date": "05/10/2024 16:52:12",
              "content": "<p>Nice to hear from you, yeah mine neither. <br>\nInterested! it seems like doing CV with DL doesn't work</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2801424": "Hello everyone,\nIt seems that tree-based models still outperform deep learning on tabular data, but we might also want to try some new models,  such as TabR - A simple feed-forward network with k-Nearest-Neighbors. [https://github.com/yandex-research/tabular-dl-tabr?tab=readme-ov-file](url)\nUnfortunately, I tried it but failed. It's just so new and niche that it caused me to not find codes about it in the Kaggle community. Has anyone else tried this or any other deep learning model ? I'd like to learn more with the use of deep learning in this game,  Thanks so much for the discussion and advice!",
    "2802897": "Logistic regression was giving 0.5-0.6 roc auc score , sklearn nural network with 100, 50 2 hidden layers was giving even less score ...",
    "2802907": "You can get much higher AUC than that if you use logistic regression from sklearn with penalty and transformation/standardization of variables. I tested with quantile transformer. Though the stability on LB was pretty bad.",
    "2805241": "I did, the highest LB score for my DL model is .545 , I have tested with MLP, GRU, LSTM and ResNet. \nThe LB increase to 0.56-0.57 when I blended it with a tree-based model like CAT and LGBM.\nHowever, till now, the combination models of purely tree-based is still better than having a DL",
    "2805391": "How does your DL perform on CV compared to tree models? Mine didn't add much to an esnemble even on CV.",
    "2805634": "Nice to hear from you, yeah mine neither. \nInterested! it seems like doing CV with DL doesn't work"
  },
  "source": "meta"
}