{
  "id": 199199,
  "title": "How to do EDA and Auto ML on huge datasets ?",
  "url": "/competitions/riiid-test-answer-prediction/discussion/199199",
  "author_name": "",
  "post_date": "2020-11-24T20:10:11.432184700Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hello, I am quite new to this machine learning world and I have never seen dataset as huge as this. Please help me out on how to do data analysis on this much huge dataset, also how to get the best model for this type of dataset, on searching google I ended up on library called PyCaret which is an autoML library helps to find out best models.</p>\n<p>Please, someone, help me out on this that how to do data analytics  on this much huge dataset</p>",
  "messages": [
    {
      "id": "1089819",
      "postDate": "11/24/2020 20:10:11",
      "content": "<p>Hello, I am quite new to this machine learning world and I have never seen dataset as huge as this. Please help me out on how to do data analysis on this much huge dataset, also how to get the best model for this type of dataset, on searching google I ended up on library called PyCaret which is an autoML library helps to find out best models.</p>\n<p>Please, someone, help me out on this that how to do data analytics  on this much huge dataset</p>",
      "rawMarkdown": "Hello, I am quite new to this machine learning world and I have never seen dataset as huge as this. Please help me out on how to do data analysis on this much huge dataset, also how to get the best model for this type of dataset, on searching google I ended up on library called PyCaret which is an autoML library helps to find out best models.\n\nPlease, someone, help me out on this that how to do data analytics  on this much huge dataset",
      "votes": null
    },
    {
      "id": "1091071",
      "postDate": "11/25/2020 18:51:51",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/omniking1999\" target=\"_blank\">@omniking1999</a></p>\n<p>I think that this notebook can help you 🙂</p>\n<p><a href=\"https://www.kaggle.com/rohanrao/tutorial-on-reading-large-datasets\" target=\"_blank\">\"Tutorial on reading large datasets\"</a></p>\n<p>For the model, I see a lot of people using tree-based models if that helps you 🙂 There are a lot of great notebooks public here where we can see how others are dealing with this dataset</p>",
      "rawMarkdown": "Hi @omniking1999\n\nI think that this notebook can help you 🙂\n\n[\"Tutorial on reading large datasets\"](https://www.kaggle.com/rohanrao/tutorial-on-reading-large-datasets)\n\nFor the model, I see a lot of people using tree-based models if that helps you 🙂 There are a lot of great notebooks public here where we can see how others are dealing with this dataset",
      "votes": null
    },
    {
      "id": "1091157",
      "postDate": "11/25/2020 19:59:08",
      "content": "<p><a href=\"https://www.kaggle.com/carlosdg\" target=\"_blank\">@carlosdg</a> Thanks for your Support </p>",
      "rawMarkdown": "carlosdg Thanks for your Support",
      "votes": null
    },
    {
      "id": "1093682",
      "postDate": "11/27/2020 23:21:34",
      "content": "<p>if using R, I have had some luck using ffbase &amp; ff packages and biglm package for regression analysis, although traditional regression analysis for this competition probably won't cut it, its been a good learning experience for me.</p>",
      "rawMarkdown": "if using R, I have had some luck using ffbase & ff packages and biglm package for regression analysis, although traditional regression analysis for this competition probably won't cut it, its been a good learning experience for me.",
      "votes": null
    },
    {
      "id": "1094781",
      "postDate": "11/28/2020 23:06:33",
      "content": "<p><a href=\"https://www.kaggle.com/michaelj8\" target=\"_blank\">@michaelj8</a> can you recommend me some ff packages  biglm package for classification analysis.<br>\nalso thanks for your reply </p>",
      "rawMarkdown": "michaelj8 can you recommend me some ff packages  biglm package for classification analysis.\nalso thanks for your reply",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1091071,
      "author_name": "carlosdg",
      "author_url": "",
      "post_date": "11/25/2020 18:51:51",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/omniking1999\" target=\"_blank\">@omniking1999</a></p>\n<p>I think that this notebook can help you 🙂</p>\n<p><a href=\"https://www.kaggle.com/rohanrao/tutorial-on-reading-large-datasets\" target=\"_blank\">\"Tutorial on reading large datasets\"</a></p>\n<p>For the model, I see a lot of people using tree-based models if that helps you 🙂 There are a lot of great notebooks public here where we can see how others are dealing with this dataset</p>",
      "votes": null,
      "replies": [
        {
          "id": 1091157,
          "author_name": "omniking1999",
          "author_url": "",
          "post_date": "11/25/2020 19:59:08",
          "content": "<p><a href=\"https://www.kaggle.com/carlosdg\" target=\"_blank\">@carlosdg</a> Thanks for your Support </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1093682,
      "author_name": "michaelj8",
      "author_url": "",
      "post_date": "11/27/2020 23:21:34",
      "content": "<p>if using R, I have had some luck using ffbase &amp; ff packages and biglm package for regression analysis, although traditional regression analysis for this competition probably won't cut it, its been a good learning experience for me.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1094781,
          "author_name": "omniking1999",
          "author_url": "",
          "post_date": "11/28/2020 23:06:33",
          "content": "<p><a href=\"https://www.kaggle.com/michaelj8\" target=\"_blank\">@michaelj8</a> can you recommend me some ff packages  biglm package for classification analysis.<br>\nalso thanks for your reply </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1089819": "Hello, I am quite new to this machine learning world and I have never seen dataset as huge as this. Please help me out on how to do data analysis on this much huge dataset, also how to get the best model for this type of dataset, on searching google I ended up on library called PyCaret which is an autoML library helps to find out best models.\n\nPlease, someone, help me out on this that how to do data analytics  on this much huge dataset",
    "1091071": "Hi @omniking1999\n\nI think that this notebook can help you 🙂\n\n[\"Tutorial on reading large datasets\"](https://www.kaggle.com/rohanrao/tutorial-on-reading-large-datasets)\n\nFor the model, I see a lot of people using tree-based models if that helps you 🙂 There are a lot of great notebooks public here where we can see how others are dealing with this dataset",
    "1091157": "carlosdg Thanks for your Support",
    "1093682": "if using R, I have had some luck using ffbase & ff packages and biglm package for regression analysis, although traditional regression analysis for this competition probably won't cut it, its been a good learning experience for me.",
    "1094781": "michaelj8 can you recommend me some ff packages  biglm package for classification analysis.\nalso thanks for your reply"
  },
  "source": "meta"
}