parquet dataset (without having to read in json), read more hereπ‘ Also, do note that there is a repo on GitHub under which the data for this competition has been shared. It contains preprocessing code and information beyond what is available on Kaggle in the competition tabs (you can find most if not all of it in the discussions though that I link to above)!
\nThe competition is shaping up really nicely and I am super excited about it! π₯³
\nThanks so much for all the hard work and the amazing resources that so many people have shared! π)
\nEDIT: I wrote this post a week or two ago. Adding a couple more resources that can be helpful that have been added in the meantime:
\nA couple of related resources you might find useful:
\nparquet dataset (without having to read in json), read more hereπ‘ Also, do note that there is a repo on GitHub under which the data for this competition has been shared. It contains preprocessing code and information beyond what is available on Kaggle in the competition tabs (you can find most if not all of it in the discussions though that I link to above)!
\nThe competition is shaping up really nicely and I am super excited about it! π₯³
\nThanks so much for all the hard work and the amazing resources that so many people have shared! π)
\nEDIT: I wrote this post a week or two ago. Adding a couple more resources that can be helpful that have been added in the meantime:
\nA couple of related resources you might find useful:
\nThanks! for descriptions!
", "rawMarkdown": "Thanks! for descriptions!", "votes": null }, { "id": "2017119", "postDate": "11/04/2022 13:59:55", "content": "The mistake in the metric was embarrassing, but I'm also proud to be part of a community that handles these hiccups with grace and humor.
", "rawMarkdown": "The mistake in the metric was embarrassing, but I'm also proud to be part of a community that handles these hiccups with grace and humor.", "votes": null }, { "id": "2017193", "postDate": "11/04/2022 15:29:32", "content": "there were issues with the competition metric -> I thought someone was stacking xgboosts π
π
Under your illustrious leadership, I wouldn't expect anything less from the community! π
\nMight I say this tiny hiccup added an element of humor that only contributed to the awesome experience that this competition already is!!!
\nPlus who doesn't have bugs in the code they write π Assuming most of us on here are code writers to one extent or another, it would be some very bad optics if we should throw shade on someone due to an honest mistake that had no consequences and that could be quickly corrected π€·ββοΈ I know this is wishful thinking, but may all the bugs we produce be like that π
", "rawMarkdown": "Under your illustrious leadership, I wouldn't expect anything less from the community! π\n\nMight I say this tiny hiccup added an element of humor that only contributed to the awesome experience that this competition already is!!!\n\nPlus who doesn't have bugs in the code they write π Assuming most of us on here are code writers to one extent or another, it would be some very bad optics if we should throw shade on someone due to an honest mistake that had no consequences and that could be quickly corrected π€·ββοΈ I know this is wishful thinking, but may all the bugs we produce be like that π", "votes": null }, { "id": "2017913", "postDate": "11/05/2022 08:07:00", "content": "My pleasure @zvr842! π
", "rawMarkdown": "My pleasure @zvr842! π", "votes": null }, { "id": "2038166", "postDate": "11/21/2022 08:22:35", "content": "Thanks! for descriptions!
", "rawMarkdown": "Thanks! for descriptions!", "votes": null }, { "id": "2038251", "postDate": "11/21/2022 09:34:32", "content": "np at all @xtxxueyan! π Glad I could be of help!
", "rawMarkdown": "np at all @xtxxueyan! π Glad I could be of help!", "votes": null } ], "comments": [ { "id": 2016866, "author_name": "zvr842", "author_url": "", "post_date": "11/04/2022 10:15:32", "content": "Thanks! for descriptions!
", "votes": null, "replies": [ { "id": 2017913, "author_name": "radek1", "author_url": "", "post_date": "11/05/2022 08:07:00", "content": "My pleasure @zvr842! π
", "votes": null, "replies": [] } ] }, { "id": 2017119, "author_name": "inversion", "author_url": "", "post_date": "11/04/2022 13:59:55", "content": "The mistake in the metric was embarrassing, but I'm also proud to be part of a community that handles these hiccups with grace and humor.
", "votes": null, "replies": [ { "id": 2017783, "author_name": "radek1", "author_url": "", "post_date": "11/05/2022 06:35:20", "content": "Under your illustrious leadership, I wouldn't expect anything less from the community! π
\nMight I say this tiny hiccup added an element of humor that only contributed to the awesome experience that this competition already is!!!
\nPlus who doesn't have bugs in the code they write π Assuming most of us on here are code writers to one extent or another, it would be some very bad optics if we should throw shade on someone due to an honest mistake that had no consequences and that could be quickly corrected π€·ββοΈ I know this is wishful thinking, but may all the bugs we produce be like that π
", "votes": null, "replies": [] } ] }, { "id": 2017193, "author_name": "andradaolteanu", "author_url": "", "post_date": "11/04/2022 15:29:32", "content": "there were issues with the competition metric -> I thought someone was stacking xgboosts π
π
Thanks! for descriptions!
", "votes": null, "replies": [ { "id": 2038251, "author_name": "radek1", "author_url": "", "post_date": "11/21/2022 09:34:32", "content": "np at all @xtxxueyan! π Glad I could be of help!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2016849": "# Discussion summary\n\n- Training on test data is okay! π₯³\n - as observed by @cdeotte [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363939)\n - and confirmed by the organizer [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363939)\n- How are sessions defined?\n - A session is all activity by a single user either in train or in test\n - confirmed by the organizer [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363554#2015486)\n- In the beginning, there were issues with the competition metric (someone managed to score ~4.8 on recall on public LBπ )\n - This has now been [corrected](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363772). Thank you, @inversion! π\n- Some good thoughts on the competition [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363874) by @narsil.\n- [π‘ Do not disregard longer sessions -- they contribute disproportionately to the competition metric!](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364375)\n- Good conversation on competition metric [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364064) and [here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364530)\n\n# Notebooks recap\n\n- The outstanding [co-visitation matrix](https://www.kaggle.com/code/vslaykovsky/co-visitation-matrix) by @vslaykovsky! \n- A notebook by @cdeotte building on the co-visitation matrix π and demonstrating the power of training on the test data! (maybe training is too strong of a word, rather using the leak in your calculations)\n- My take on the above approach that is simplified and that runs on a `parquet` dataset (without having to read in json), read [more here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364210)\n- the last 20 AIDs are very powerful! ([original code](https://www.kaggle.com/code/ttahara/last-aid-20), [simplified without need for chunking](https://www.kaggle.com/code/radek1/last-20-aids))\n- An overview of how to set up [local validation](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991)\n- Local Validation is key to improving results -- no one worked on this so [I implemented one here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364216)\n\n# Resources for getting started\n\n- A [great post](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363624) from @andradaolteanu with a link to Andrew Ng's videos on Recommender Systems\n- [Getting started resources](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363814) from yours truly, among other things a link to a legendary lecture by Xavier Amatriain of Netflix fame\n- [Transformers4Rec](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363980) by NVIDIA, deep learning session-based recommendation models, as recommended by @snnclsr! π\n- [A post on RecBole](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363603), a very interesting looking Recommendation Model library implemented in PyTorch π₯(thanks @hidehisaarai1213)\n- [[Starter pack] LGBMRanker with polars πππ](https://www.kaggle.com/competitions/otto-recommender-system/discussion/366194) - the first (and so far only) introduction to using Ranking Models by yours truly\n\nπ‘ Also, do note that there is [a repo on GitHub](https://github.com/otto-de/recsys-dataset) under which the data for this competition has been shared. It contains preprocessing code and information beyond what is available on Kaggle in the competition tabs (you can find most if not all of it in the discussions though that I link to above)!\n\nThe competition is shaping up really nicely and I am super excited about it! π₯³\n\nThanks so much for all the hard work and the amazing resources that so many people have shared! π) \n\n**EDIT**: I wrote this post a week or two ago. Adding a couple more resources that can be helpful that have been added in the meantime:\n\n**A couple of related resources you might find useful:**\n\n* [π‘ [2 methods] How-to ensemble predictions π π π ](https://www.kaggle.com/code/radek1/2-methods-how-to-ensemble-predictions)\n* [local validation tracks public LB perfecty -- here is the setup](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991)\n* [π‘ For my friends from Twitter and LinkedIn -- here is how to dive into this competition π³](https://www.kaggle.com/competitions/otto-recommender-system/discussion/368560)\n* [Full dataset processed to CSV/parquet files with optimized memory footprint](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363843)\n* [co-visitation matrix - simplified, imprvd logic π₯](https://www.kaggle.com/code/radek1/co-visitation-matrix-simplified-imprvd-logic)\n* [π‘ Word2Vec How-to [training and submission]πππ](https://www.kaggle.com/code/radek1/word2vec-how-to-training-and-submission)", "2016866": "Thanks! for descriptions!", "2017119": "The mistake in the metric was embarrassing, but I'm also proud to be part of a community that handles these hiccups with grace and humor.", "2017193": "`there were issues with the competition metric` -> I thought someone was stacking xgboosts π π", "2017783": "Under your illustrious leadership, I wouldn't expect anything less from the community! π\n\nMight I say this tiny hiccup added an element of humor that only contributed to the awesome experience that this competition already is!!!\n\nPlus who doesn't have bugs in the code they write π Assuming most of us on here are code writers to one extent or another, it would be some very bad optics if we should throw shade on someone due to an honest mistake that had no consequences and that could be quickly corrected π€·ββοΈ I know this is wishful thinking, but may all the bugs we produce be like that π", "2017913": "My pleasure @zvr842! π", "2038166": "Thanks! for descriptions!", "2038251": "np at all @xtxxueyan! π Glad I could be of help!" }, "source": "meta" }