I started this competition with this simple mean correctness model and I was unable to do better than 0.705.
\nIn this simple model, I just submit the content's mean correctness, no matter which user is it ! I finally felt like introducing some user info by submitting an harmonic mean of user and content correctness means and gess what : the LB jumps from 0.705 to 0.735 !!!
\nEven if this model still very basic, it clearly proof that features that incorporate user and content interactions could drastically improve Riiid models.
\nIf you use such kind of features, don't mind reporting its impact on your model in the comments :) .
", "messages": [ { "id": "1041817", "postDate": "10/07/2020 23:30:59", "content": "I started this competition with this simple mean correctness model and I was unable to do better than 0.705.
\nIn this simple model, I just submit the content's mean correctness, no matter which user is it ! I finally felt like introducing some user info by submitting an harmonic mean of user and content correctness means and gess what : the LB jumps from 0.705 to 0.735 !!!
\nEven if this model still very basic, it clearly proof that features that incorporate user and content interactions could drastically improve Riiid models.
\nIf you use such kind of features, don't mind reporting its impact on your model in the comments :) .
", "rawMarkdown": "I started this competition with [this simple mean correctness](https://www.kaggle.com/kneroma/riid-user-and-content-mean-predictor) model and I was unable to do better than **0.705**.\n\nIn this simple model, I just submit the content's mean correctness, no matter which user is it ! I finally felt like introducing some user info by submitting an harmonic mean of user and content correctness means and gess what : the LB jumps from **0.705** to **0.735** !!!\n\nEven if this model still very basic, it clearly proof that features that incorporate **user and content interactions** could drastically improve Riiid models.\n\nIf you use such kind of features, don't mind reporting its impact on your model in the comments :) .", "votes": null }, { "id": "1042136", "postDate": "10/08/2020 04:55:28", "content": "There are not enough features in the competition data, you gotta make them yourself, that makes the competition is all about feature engineering (as of now, might change in the future).
", "rawMarkdown": "There are not enough features in the competition data, you gotta make them yourself, that makes the competition is all about feature engineering (as of now, might change in the future).", "votes": null }, { "id": "1042512", "postDate": "10/08/2020 09:18:09", "content": "Thanks very much for the tip :)
", "rawMarkdown": "Thanks very much for the tip :)", "votes": null }, { "id": "1042606", "postDate": "10/08/2020 10:58:31", "content": "Always happy to share :)
", "rawMarkdown": "Always happy to share :)", "votes": null }, { "id": "1042607", "postDate": "10/08/2020 10:59:27", "content": "You're totally right … a second approach could consist in using NN with a good data understanding and modeling.
", "rawMarkdown": "You're totally right ... a second approach could consist in using NN with a good data understanding and modeling.", "votes": null }, { "id": "1043155", "postDate": "10/08/2020 18:13:13", "content": "Well, have you thought of treating the timestamp data as a time series model (per user) and somewhat applying any time series or any sequential model to that part? I wonder how will it perform!
", "rawMarkdown": "Well, have you thought of treating the timestamp data as a time series model (per user) and somewhat applying any time series or any sequential model to that part? I wonder how will it perform!", "votes": null }, { "id": "1043203", "postDate": "10/08/2020 18:58:18", "content": "You could try hyper parameter optimization, I went from around a .7 to .75 with it (even though I still throw an error when i submit oof).
", "rawMarkdown": "You could try hyper parameter optimization, I went from around a .7 to .75 with it (even though I still throw an error when i submit oof).", "votes": null }, { "id": "1043271", "postDate": "10/08/2020 20:17:52", "content": "Yep I could :) ! But I do feel a little ashamed in probing public LB
", "rawMarkdown": "Yep I could :) ! But I do feel a little ashamed in probing public LB", "votes": null }, { "id": "1043323", "postDate": "10/08/2020 22:04:15", "content": "Yeah I guess to some degree its probing LB, but at least you're not sitting there for 3 weeks fine tuning some random hyperparameter like for OSIC (and I still can't get mine to submit properly so I guess I'm not either lol).
", "rawMarkdown": "Yeah I guess to some degree its probing LB, but at least you're not sitting there for 3 weeks fine tuning some random hyperparameter like for OSIC (and I still can't get mine to submit properly so I guess I'm not either lol).", "votes": null }, { "id": "1044346", "postDate": "10/09/2020 17:22:25", "content": "lol … OSIC was a real mess ! But still less messy than M5 😄
", "rawMarkdown": "lol ... OSIC was a real mess ! But still less messy than M5 😄", "votes": null }, { "id": "1044608", "postDate": "10/10/2020 00:51:32", "content": "Yeah, I wish I was around for M5, it sounds like one of those messes that are still pretty fun.
", "rawMarkdown": "Yeah, I wish I was around for M5, it sounds like one of those messes that are still pretty fun.", "votes": null }, { "id": "1044775", "postDate": "10/10/2020 05:27:28", "content": "I think that incorporating lag features to account for improvement in skill over time is quite important. Rather than looking at mean encoded features for all time; the most recent gauge of performance should have a higher weighting. I tried doing this in a kernel but the public score dropped although I’m sure I made an error somewhere.
\nI think the tags for the questions will create another important feature and I think the current best scores can be improved upon quite a bit since they’re quite basic at this stage. I think other algorithm such as RNN might provide a good alternative to tree methods for ensembling with.
", "rawMarkdown": "I think that incorporating lag features to account for improvement in skill over time is quite important. Rather than looking at mean encoded features for all time; the most recent gauge of performance should have a higher weighting. I tried doing this in a kernel but the public score dropped although I’m sure I made an error somewhere. \n\nI think the tags for the questions will create another important feature and I think the current best scores can be improved upon quite a bit since they’re quite basic at this stage. I think other algorithm such as RNN might provide a good alternative to tree methods for ensembling with.", "votes": null } ], "comments": [ { "id": 1042136, "author_name": "harshitsheoran", "author_url": "", "post_date": "10/08/2020 04:55:28", "content": "There are not enough features in the competition data, you gotta make them yourself, that makes the competition is all about feature engineering (as of now, might change in the future).
", "votes": null, "replies": [ { "id": 1042607, "author_name": "kneroma", "author_url": "", "post_date": "10/08/2020 10:59:27", "content": "You're totally right … a second approach could consist in using NN with a good data understanding and modeling.
", "votes": null, "replies": [] } ] }, { "id": 1042512, "author_name": "domizianostingi", "author_url": "", "post_date": "10/08/2020 09:18:09", "content": "Thanks very much for the tip :)
", "votes": null, "replies": [ { "id": 1042606, "author_name": "kneroma", "author_url": "", "post_date": "10/08/2020 10:58:31", "content": "Always happy to share :)
", "votes": null, "replies": [] } ] }, { "id": 1043155, "author_name": "mrutyunjaybiswal", "author_url": "", "post_date": "10/08/2020 18:13:13", "content": "Well, have you thought of treating the timestamp data as a time series model (per user) and somewhat applying any time series or any sequential model to that part? I wonder how will it perform!
", "votes": null, "replies": [] }, { "id": 1043203, "author_name": "abhaykatoch", "author_url": "", "post_date": "10/08/2020 18:58:18", "content": "You could try hyper parameter optimization, I went from around a .7 to .75 with it (even though I still throw an error when i submit oof).
", "votes": null, "replies": [ { "id": 1043271, "author_name": "kneroma", "author_url": "", "post_date": "10/08/2020 20:17:52", "content": "Yep I could :) ! But I do feel a little ashamed in probing public LB
", "votes": null, "replies": [] }, { "id": 1043323, "author_name": "abhaykatoch", "author_url": "", "post_date": "10/08/2020 22:04:15", "content": "Yeah I guess to some degree its probing LB, but at least you're not sitting there for 3 weeks fine tuning some random hyperparameter like for OSIC (and I still can't get mine to submit properly so I guess I'm not either lol).
", "votes": null, "replies": [] }, { "id": 1044346, "author_name": "kneroma", "author_url": "", "post_date": "10/09/2020 17:22:25", "content": "lol … OSIC was a real mess ! But still less messy than M5 😄
", "votes": null, "replies": [] }, { "id": 1044608, "author_name": "abhaykatoch", "author_url": "", "post_date": "10/10/2020 00:51:32", "content": "Yeah, I wish I was around for M5, it sounds like one of those messes that are still pretty fun.
", "votes": null, "replies": [] } ] }, { "id": 1044775, "author_name": "lgreig", "author_url": "", "post_date": "10/10/2020 05:27:28", "content": "I think that incorporating lag features to account for improvement in skill over time is quite important. Rather than looking at mean encoded features for all time; the most recent gauge of performance should have a higher weighting. I tried doing this in a kernel but the public score dropped although I’m sure I made an error somewhere.
\nI think the tags for the questions will create another important feature and I think the current best scores can be improved upon quite a bit since they’re quite basic at this stage. I think other algorithm such as RNN might provide a good alternative to tree methods for ensembling with.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1041817": "I started this competition with [this simple mean correctness](https://www.kaggle.com/kneroma/riid-user-and-content-mean-predictor) model and I was unable to do better than **0.705**.\n\nIn this simple model, I just submit the content's mean correctness, no matter which user is it ! I finally felt like introducing some user info by submitting an harmonic mean of user and content correctness means and gess what : the LB jumps from **0.705** to **0.735** !!!\n\nEven if this model still very basic, it clearly proof that features that incorporate **user and content interactions** could drastically improve Riiid models.\n\nIf you use such kind of features, don't mind reporting its impact on your model in the comments :) .", "1042136": "There are not enough features in the competition data, you gotta make them yourself, that makes the competition is all about feature engineering (as of now, might change in the future).", "1042512": "Thanks very much for the tip :)", "1042606": "Always happy to share :)", "1042607": "You're totally right ... a second approach could consist in using NN with a good data understanding and modeling.", "1043155": "Well, have you thought of treating the timestamp data as a time series model (per user) and somewhat applying any time series or any sequential model to that part? I wonder how will it perform!", "1043203": "You could try hyper parameter optimization, I went from around a .7 to .75 with it (even though I still throw an error when i submit oof).", "1043271": "Yep I could :) ! But I do feel a little ashamed in probing public LB", "1043323": "Yeah I guess to some degree its probing LB, but at least you're not sitting there for 3 weeks fine tuning some random hyperparameter like for OSIC (and I still can't get mine to submit properly so I guess I'm not either lol).", "1044346": "lol ... OSIC was a real mess ! But still less messy than M5 😄", "1044608": "Yeah, I wish I was around for M5, it sounds like one of those messes that are still pretty fun.", "1044775": "I think that incorporating lag features to account for improvement in skill over time is quite important. Rather than looking at mean encoded features for all time; the most recent gauge of performance should have a higher weighting. I tried doing this in a kernel but the public score dropped although I’m sure I made an error somewhere. \n\nI think the tags for the questions will create another important feature and I think the current best scores can be improved upon quite a bit since they’re quite basic at this stage. I think other algorithm such as RNN might provide a good alternative to tree methods for ensembling with." }, "source": "meta" }