Hey!
\nI am super excited for this competition π Have been perusing the world of recommender systems for the last 6 months or so, will interesting to put all that to a test π
\nIn terms of good learning resources to get started with recommender systems, there are not that many, unfortunately.
\nI would highly recommend this lecture by a former colleague of mine, Xavier Amatriain. It provides the best introduction to thinking about recommendations, what recommender systems are, what can be achieved, that I have ever come across.
\nThis particular problem that we will be working on in this competition are session based recommendations. Essentially, from a timeseries of actions taken by a user in a single session (single visit to a website), we want to predict what is the likely next action a user might take (and most importantly, on what item the action is likely to be taken!)
\nSession recommendations are a very hot topic. They pull in a lot of exciting concepts together such as serving real time predictions (versus predictions calculated offline, usually in batches). And their sequential nature lends themselves well to modelling with RNNs or more recently, Transformers!
\nThere is a lot of complexity and interesting ideas to all of this π I am hoping we will have a chance to explore many of them in this competition together!
\nIf you'd like to read more about using Transformers for session based recommendations, here is a very nice blog post from my colleagues.
\nSo looking forward to jumping into this competition π₯³
\nHappy Kaggling!
\nHey!
\nI am super excited for this competition π Have been perusing the world of recommender systems for the last 6 months or so, will interesting to put all that to a test π
\nIn terms of good learning resources to get started with recommender systems, there are not that many, unfortunately.
\nI would highly recommend this lecture by a former colleague of mine, Xavier Amatriain. It provides the best introduction to thinking about recommendations, what recommender systems are, what can be achieved, that I have ever come across.
\nThis particular problem that we will be working on in this competition are session based recommendations. Essentially, from a timeseries of actions taken by a user in a single session (single visit to a website), we want to predict what is the likely next action a user might take (and most importantly, on what item the action is likely to be taken!)
\nSession recommendations are a very hot topic. They pull in a lot of exciting concepts together such as serving real time predictions (versus predictions calculated offline, usually in batches). And their sequential nature lends themselves well to modelling with RNNs or more recently, Transformers!
\nThere is a lot of complexity and interesting ideas to all of this π I am hoping we will have a chance to explore many of them in this competition together!
\nIf you'd like to read more about using Transformers for session based recommendations, here is a very nice blog post from my colleagues.
\nSo looking forward to jumping into this competition π₯³
\nHappy Kaggling!
\nCould you post a notebook on how to get Transformers4Rec running on kaggle?
\nI wanted to try out the following notebook:
\nhttps://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/end-to-end-session-based/01-ETL-with-NVTabular.ipynb
\nbut had problemsβ¦
Will look into that! π
\nThe transformers will soon be coming to Merlin Models (thous will be HuggingFace transformers implemented in Tensorflow) so maybe that will make it a bit easier to run here on Kaggle, we will see π
", "rawMarkdown": "Will look into that! π\n\nThe transformers will soon be coming to Merlin Models (thous will be HuggingFace transformers implemented in Tensorflow) so maybe that will make it a bit easier to run here on Kaggle, we will see π", "votes": null }, { "id": "2016161", "postDate": "11/03/2022 20:21:46", "content": "Thanks a lot for sharing, coincidentally I've also been reading up on it recently because I am interested in trying to apply it on a different domain. Excited for this comp too!
", "rawMarkdown": "Thanks a lot for sharing, coincidentally I've also been reading up on it recently because I am interested in trying to apply it on a different domain. Excited for this comp too!", "votes": null }, { "id": "2016249", "postDate": "11/03/2022 21:32:25", "content": "Thanks, @ollibolli! Really glad you are finding this useful π BTW what domain are you thinking of applying this to? Just curious, don't need to answer if you don't want to share π
", "rawMarkdown": "Thanks, @ollibolli! Really glad you are finding this useful π BTW what domain are you thinking of applying this to? Just curious, don't need to answer if you don't want to share π", "votes": null }, { "id": "2017263", "postDate": "11/04/2022 16:46:42", "content": "Thanks Radek,
\nno worries, I apologize I was being cryptic I didn't intend to be. One of the things I love most about Kaggle is that everyone is sharing, like you!
\nMight sound silly but I've been getting into soccer analytics and I thought about applying it to player scouting and to extension maybe even selection with some tweaks. I am quite the newbie so it might lead nowhere haha.
Have you seen the Bundesliga Dataset yet? Its quite interesting. I watched a live coding on prev. competition about this, maybe you want to check it out. https://youtu.be/fNHW7zvi_F8
", "rawMarkdown": "Have you seen the Bundesliga Dataset yet? Its quite interesting. I watched a live coding on prev. competition about this, maybe you want to check it out. https://youtu.be/fNHW7zvi_F8", "votes": null }, { "id": "2017778", "postDate": "11/05/2022 06:30:32", "content": "Thank you for your reply, @oliver! Sounds like an exciting project! π
", "rawMarkdown": "Thank you for your reply, @oliver! Sounds like an exciting project! π", "votes": null }, { "id": "2018044", "postDate": "11/05/2022 10:56:56", "content": "Hi Ian, cheers, yes, I followed this quite closely, there is a group at my school that actually has a few research projects on this. It's a very exciting time :-).
", "rawMarkdown": "Hi Ian, cheers, yes, I followed this quite closely, there is a group at my school that actually has a few research projects on this. It's a very exciting time :-).", "votes": null }, { "id": "2019123", "postDate": "11/06/2022 11:05:16", "content": "Thanks, even I couldn't find out anything interesting. Will definitely have a look.
", "rawMarkdown": "Thanks, even I couldn't find out anything interesting. Will definitely have a look.", "votes": null }, { "id": "2020487", "postDate": "11/07/2022 14:22:23", "content": "https://github.com/hidasib/GRU4Rec
\nimplementation of the algorithm in \"Session-based Recommendations with Recurrent Neural Networks\" paper.
Gratitude for sharing!
", "rawMarkdown": "Gratitude for sharing!", "votes": null }, { "id": "2020801", "postDate": "11/07/2022 19:33:44", "content": "Really glad you are finding this useful! π
", "rawMarkdown": "Really glad you are finding this useful! π", "votes": null }, { "id": "2023136", "postDate": "11/09/2022 14:58:19", "content": "Hi @radek1 Thank you so much for the resources to help beginners to get started.
\nI have finished watching and note taking the getting started videos on recsys from Andrew Ng (which are relatively easy to follow) and I am half way through those of Xavier Amatriain (two more hours to go for lecture 3+4) which I found many concepts are not very familiar and definitely need to rewatch a few more times.
\nI wonder how much should I understand Xavier's lectures? I know you would approve that the best way to help understanding is to experiment with the models he talked about, but there are no accompany notebooks or codes available from the lectures right?
\nI wonder have you found or implemented those the models or codes when you study the videos? or do you know where I can find them? or do actually I need to study them now for learning recsys in order to prepare me for participating in OTTO competition? Or maybe focusing on experimenting with your notebooks is already enough for beginners?
\nThanks!
", "rawMarkdown": "Hi @radek1 Thank you so much for the resources to help beginners to get started. \n\nI have finished watching and [note taking](https://www.kaggle.com/competitions/otto-recommender-system/discussion/365039) the getting started videos on recsys from Andrew Ng (which are relatively easy to follow) and I am half way through those of Xavier Amatriain (two more hours to go for lecture 3+4) which I found many concepts are not very familiar and definitely need to rewatch a few more times.\n\nI wonder how much should I understand Xavier's lectures? I know you would approve that the best way to help understanding is to experiment with the models he talked about, but there are no accompany notebooks or codes available from the lectures right? \n\nI wonder have you found or implemented those the models or codes when you study the videos? or do you know where I can find them? or do actually I need to study them now for learning recsys in order to prepare me for participating in OTTO competition? Or maybe focusing on experimenting with your notebooks is already enough for beginners?\n\nThanks!", "votes": null }, { "id": "2023699", "postDate": "11/10/2022 01:00:01", "content": "I just found a summary of Xavier's lecture 1 + 2 by @radek1 here! Thanks Radek
", "rawMarkdown": "I just found a summary of Xavier's lecture 1 + 2 by @radek1 [here](https://twitter.com/radekosmulski/status/1565716248083566592)! Thanks Radek", "votes": null }, { "id": "2023718", "postDate": "11/10/2022 01:25:12", "content": "Xavier does a great job in his lectures of explaining the reasoning behind recommender systems, I think that is where the value lies. Very few people do that and nearly no one has experience comparable to his π As in, the thinking behind what you want and why you want to make the decisions that you do.
\nBut on the technical side, I think you will find there is a great disconnect between the ideas and what you can implement in code. Everything is very dataset-specific. In some way, this is another skillset that you have to build up, this ability to transfer those ideas to the problem you are working on. Even one or two ideas can go a long way.
\nHere, for instance, this is a recommender system competition, but the curve ball is the high cardinality of action ids. How do you deal with this? There has been some really good discussion on this here: π the elephant in the room -- high cardinality of targets and what to do about this.
\nOr even more concretely, how do you work with the amount of data that you have available in this competition? This is on a completely different scale than most of the \"toy\" ML problems.
\nSo I think chipping away at this competition and learning from the conversations and the kernels that people post is a great way of getting hands-on experience π It really does go a long way in actual business context.
\nThough some of the challenges that you might have to solve in the real world will be different, the problem-solving methodology that you can work out while trying to improve at a Kaggle competition goes a really long way π
\nMeaning, I feel if you found your way to Kaggle and this competition, you probably are in a great spot to grow your practical set of abilities by working on this problem π
", "rawMarkdown": "Xavier does a great job in his lectures of explaining the reasoning behind recommender systems, I think that is where the value lies. Very few people do that and nearly no one has experience comparable to his π As in, the thinking behind what you want and why you want to make the decisions that you do.\n\nBut on the technical side, I think you will find there is a great disconnect between the ideas and what you can implement in code. Everything is very dataset-specific. In some way, this is another skillset that you have to build up, this ability to transfer those ideas to the problem you are working on. Even one or two ideas can go a long way.\n\nHere, for instance, this is a recommender system competition, but the curve ball is the high cardinality of action ids. How do you deal with this? There has been some really good discussion on this here: [π the elephant in the room -- high cardinality of targets and what to do about this](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364722).\n\nOr even more concretely, how do you work with the amount of data that you have available in this competition? This is on a completely different scale than most of the \"toy\" ML problems.\n\nSo I think chipping away at this competition and learning from the conversations and the kernels that people post is a great way of getting hands-on experience π It really does go a long way in actual business context.\n\nThough some of the challenges that you might have to solve in the real world will be different, the problem-solving methodology that you can work out while trying to improve at a Kaggle competition goes a really long way π \n\nMeaning, I feel if you found your way to Kaggle and this competition, you probably are in a great spot to grow your practical set of abilities by working on this problem π", "votes": null }, { "id": "2033005", "postDate": "11/17/2022 01:23:36", "content": "@simonveitner Did you ever figure it out on getting Transformers4Rec to work? I was trying to do the same thing with trying the example notebook from the merlin github repo but unfortunately, I've been running into issues after installing nvtabular with the following error ModuleNotFoundError: No module named 'merlin.dag.executors'. Were you also running into this error?
Update: I was able to successfully install the packages using the following pip install commands that resolved the error i was getting. FYI, kaggle already has cudf install when using a GPU accelerator.
\n!pip install transformers4rec[pytorch,nvtabular]
\n!pip install -U nvtabular==1.3.3
Could you post a notebook on how to get Transformers4Rec running on kaggle?
\nI wanted to try out the following notebook:
\nhttps://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/end-to-end-session-based/01-ETL-with-NVTabular.ipynb
\nbut had problemsβ¦
Will look into that! π
\nThe transformers will soon be coming to Merlin Models (thous will be HuggingFace transformers implemented in Tensorflow) so maybe that will make it a bit easier to run here on Kaggle, we will see π
", "votes": null, "replies": [] }, { "id": 2033005, "author_name": "dillonquan", "author_url": "", "post_date": "11/17/2022 01:23:36", "content": "@simonveitner Did you ever figure it out on getting Transformers4Rec to work? I was trying to do the same thing with trying the example notebook from the merlin github repo but unfortunately, I've been running into issues after installing nvtabular with the following error ModuleNotFoundError: No module named 'merlin.dag.executors'. Were you also running into this error?
Update: I was able to successfully install the packages using the following pip install commands that resolved the error i was getting. FYI, kaggle already has cudf install when using a GPU accelerator.
\n!pip install transformers4rec[pytorch,nvtabular]
\n!pip install -U nvtabular==1.3.3
Thanks a lot for sharing, coincidentally I've also been reading up on it recently because I am interested in trying to apply it on a different domain. Excited for this comp too!
", "votes": null, "replies": [ { "id": 2016249, "author_name": "radek1", "author_url": "", "post_date": "11/03/2022 21:32:25", "content": "Thanks, @ollibolli! Really glad you are finding this useful π BTW what domain are you thinking of applying this to? Just curious, don't need to answer if you don't want to share π
", "votes": null, "replies": [] }, { "id": 2017263, "author_name": "ollibolli", "author_url": "", "post_date": "11/04/2022 16:46:42", "content": "Thanks Radek,
\nno worries, I apologize I was being cryptic I didn't intend to be. One of the things I love most about Kaggle is that everyone is sharing, like you!
\nMight sound silly but I've been getting into soccer analytics and I thought about applying it to player scouting and to extension maybe even selection with some tweaks. I am quite the newbie so it might lead nowhere haha.
Have you seen the Bundesliga Dataset yet? Its quite interesting. I watched a live coding on prev. competition about this, maybe you want to check it out. https://youtu.be/fNHW7zvi_F8
", "votes": null, "replies": [] }, { "id": 2017778, "author_name": "radek1", "author_url": "", "post_date": "11/05/2022 06:30:32", "content": "Thank you for your reply, @oliver! Sounds like an exciting project! π
", "votes": null, "replies": [] }, { "id": 2018044, "author_name": "ollibolli", "author_url": "", "post_date": "11/05/2022 10:56:56", "content": "Hi Ian, cheers, yes, I followed this quite closely, there is a group at my school that actually has a few research projects on this. It's a very exciting time :-).
", "votes": null, "replies": [] } ] }, { "id": 2019123, "author_name": "priyanshuprajapatis", "author_url": "", "post_date": "11/06/2022 11:05:16", "content": "Thanks, even I couldn't find out anything interesting. Will definitely have a look.
", "votes": null, "replies": [] }, { "id": 2020487, "author_name": "ridwanultanvir", "author_url": "", "post_date": "11/07/2022 14:22:23", "content": "https://github.com/hidasib/GRU4Rec
\nimplementation of the algorithm in \"Session-based Recommendations with Recurrent Neural Networks\" paper.
Gratitude for sharing!
", "votes": null, "replies": [ { "id": 2020801, "author_name": "radek1", "author_url": "", "post_date": "11/07/2022 19:33:44", "content": "Really glad you are finding this useful! π
", "votes": null, "replies": [] } ] }, { "id": 2023136, "author_name": "danielliao", "author_url": "", "post_date": "11/09/2022 14:58:19", "content": "Hi @radek1 Thank you so much for the resources to help beginners to get started.
\nI have finished watching and note taking the getting started videos on recsys from Andrew Ng (which are relatively easy to follow) and I am half way through those of Xavier Amatriain (two more hours to go for lecture 3+4) which I found many concepts are not very familiar and definitely need to rewatch a few more times.
\nI wonder how much should I understand Xavier's lectures? I know you would approve that the best way to help understanding is to experiment with the models he talked about, but there are no accompany notebooks or codes available from the lectures right?
\nI wonder have you found or implemented those the models or codes when you study the videos? or do you know where I can find them? or do actually I need to study them now for learning recsys in order to prepare me for participating in OTTO competition? Or maybe focusing on experimenting with your notebooks is already enough for beginners?
\nThanks!
", "votes": null, "replies": [ { "id": 2023699, "author_name": "danielliao", "author_url": "", "post_date": "11/10/2022 01:00:01", "content": "I just found a summary of Xavier's lecture 1 + 2 by @radek1 here! Thanks Radek
", "votes": null, "replies": [] }, { "id": 2023718, "author_name": "radek1", "author_url": "", "post_date": "11/10/2022 01:25:12", "content": "Xavier does a great job in his lectures of explaining the reasoning behind recommender systems, I think that is where the value lies. Very few people do that and nearly no one has experience comparable to his π As in, the thinking behind what you want and why you want to make the decisions that you do.
\nBut on the technical side, I think you will find there is a great disconnect between the ideas and what you can implement in code. Everything is very dataset-specific. In some way, this is another skillset that you have to build up, this ability to transfer those ideas to the problem you are working on. Even one or two ideas can go a long way.
\nHere, for instance, this is a recommender system competition, but the curve ball is the high cardinality of action ids. How do you deal with this? There has been some really good discussion on this here: π the elephant in the room -- high cardinality of targets and what to do about this.
\nOr even more concretely, how do you work with the amount of data that you have available in this competition? This is on a completely different scale than most of the \"toy\" ML problems.
\nSo I think chipping away at this competition and learning from the conversations and the kernels that people post is a great way of getting hands-on experience π It really does go a long way in actual business context.
\nThough some of the challenges that you might have to solve in the real world will be different, the problem-solving methodology that you can work out while trying to improve at a Kaggle competition goes a really long way π
\nMeaning, I feel if you found your way to Kaggle and this competition, you probably are in a great spot to grow your practical set of abilities by working on this problem π
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2015354": "Hey!\n\nI am super excited for this competition π Have been perusing the world of recommender systems for the last 6 months or so, will interesting to put all that to a test π \n\nIn terms of good learning resources to get started with recommender systems, there are not that many, unfortunately. \n \nI would highly recommend [this lecture](https://youtu.be/bLhq63ygoU8) by a former colleague of mine, Xavier Amatriain. It provides the best introduction to thinking about recommendations, what recommender systems are, what can be achieved, that I have ever come across.\n\nThis particular problem that we will be working on in this competition are session based recommendations. Essentially, from a timeseries of actions taken by a user in a single session (single visit to a website), we want to predict what is the likely next action a user might take (and most importantly, on what item the action is likely to be taken!)\n\nSession recommendations are a very hot topic. They pull in a lot of exciting concepts together such as serving real time predictions (versus predictions calculated offline, usually in batches). And their sequential nature lends themselves well to modelling with RNNs or more recently, Transformers!\n\nThere is a lot of complexity and interesting ideas to all of this π I am hoping we will have a chance to explore many of them in this competition together!\n\nIf you'd like to read more about using Transformers for session based recommendations, here is [a very nice blog post](https://medium.com/nvidia-merlin/transformers4rec-4523cc7d8fa8) from my colleagues.\n\nSo looking forward to jumping into this competition π₯³\n\nHappy Kaggling!\n\n### Other resources you might find useful:\n\n* [π‘ [2 methods] How-to ensemble predictions π π π ](https://www.kaggle.com/code/radek1/2-methods-how-to-ensemble-predictions)\n* [local validation tracks public LB perfecty -- here is the setup](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991)\n* [π‘ For my friends from Twitter and LinkedIn -- here is how to dive into this competition π³](https://www.kaggle.com/competitions/otto-recommender-system/discussion/368560)\n* [Full dataset processed to CSV/parquet files with optimized memory footprint](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363843)\n* [co-visitation matrix - simplified, imprvd logic π₯](https://www.kaggle.com/code/radek1/co-visitation-matrix-simplified-imprvd-logic)\n* [π‘ Word2Vec How-to [training and submission]πππ](https://www.kaggle.com/code/radek1/word2vec-how-to-training-and-submission)", "2015715": "Could you post a notebook on how to get Transformers4Rec running on kaggle?\nI wanted to try out the following notebook:\nhttps://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/end-to-end-session-based/01-ETL-with-NVTabular.ipynb\nbut had problems...", "2015722": "Will look into that! π\n\nThe transformers will soon be coming to Merlin Models (thous will be HuggingFace transformers implemented in Tensorflow) so maybe that will make it a bit easier to run here on Kaggle, we will see π", "2016161": "Thanks a lot for sharing, coincidentally I've also been reading up on it recently because I am interested in trying to apply it on a different domain. Excited for this comp too!", "2016249": "Thanks, @ollibolli! Really glad you are finding this useful π BTW what domain are you thinking of applying this to? Just curious, don't need to answer if you don't want to share π", "2017263": "Thanks Radek, \nno worries, I apologize I was being cryptic I didn't intend to be. One of the things I love most about Kaggle is that everyone is sharing, like you!\nMight sound silly but I've been getting into soccer analytics and I thought about applying it to player scouting and to extension maybe even selection with some tweaks. I am quite the newbie so it might lead nowhere haha.", "2017463": "Have you seen the Bundesliga Dataset yet? Its quite interesting. I watched a live coding on prev. competition about this, maybe you want to check it out. https://youtu.be/fNHW7zvi_F8", "2017778": "Thank you for your reply, @oliver! Sounds like an exciting project! π", "2018044": "Hi Ian, cheers, yes, I followed this quite closely, there is a group at my school that actually has a few research projects on this. It's a very exciting time :-).", "2019123": "Thanks, even I couldn't find out anything interesting. Will definitely have a look.", "2020487": "https://github.com/hidasib/GRU4Rec \nimplementation of the algorithm in \"Session-based Recommendations with Recurrent Neural Networks\" paper.", "2020557": "Gratitude for sharing!", "2020801": "Really glad you are finding this useful! π", "2023136": "Hi @radek1 Thank you so much for the resources to help beginners to get started. \n\nI have finished watching and [note taking](https://www.kaggle.com/competitions/otto-recommender-system/discussion/365039) the getting started videos on recsys from Andrew Ng (which are relatively easy to follow) and I am half way through those of Xavier Amatriain (two more hours to go for lecture 3+4) which I found many concepts are not very familiar and definitely need to rewatch a few more times.\n\nI wonder how much should I understand Xavier's lectures? I know you would approve that the best way to help understanding is to experiment with the models he talked about, but there are no accompany notebooks or codes available from the lectures right? \n\nI wonder have you found or implemented those the models or codes when you study the videos? or do you know where I can find them? or do actually I need to study them now for learning recsys in order to prepare me for participating in OTTO competition? Or maybe focusing on experimenting with your notebooks is already enough for beginners?\n\nThanks!", "2023699": "I just found a summary of Xavier's lecture 1 + 2 by @radek1 [here](https://twitter.com/radekosmulski/status/1565716248083566592)! Thanks Radek", "2023718": "Xavier does a great job in his lectures of explaining the reasoning behind recommender systems, I think that is where the value lies. Very few people do that and nearly no one has experience comparable to his π As in, the thinking behind what you want and why you want to make the decisions that you do.\n\nBut on the technical side, I think you will find there is a great disconnect between the ideas and what you can implement in code. Everything is very dataset-specific. In some way, this is another skillset that you have to build up, this ability to transfer those ideas to the problem you are working on. Even one or two ideas can go a long way.\n\nHere, for instance, this is a recommender system competition, but the curve ball is the high cardinality of action ids. How do you deal with this? There has been some really good discussion on this here: [π the elephant in the room -- high cardinality of targets and what to do about this](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364722).\n\nOr even more concretely, how do you work with the amount of data that you have available in this competition? This is on a completely different scale than most of the \"toy\" ML problems.\n\nSo I think chipping away at this competition and learning from the conversations and the kernels that people post is a great way of getting hands-on experience π It really does go a long way in actual business context.\n\nThough some of the challenges that you might have to solve in the real world will be different, the problem-solving methodology that you can work out while trying to improve at a Kaggle competition goes a really long way π \n\nMeaning, I feel if you found your way to Kaggle and this competition, you probably are in a great spot to grow your practical set of abilities by working on this problem π", "2033005": "simonveitner Did you ever figure it out on getting Transformers4Rec to work? I was trying to do the same thing with trying the example notebook from the merlin github repo but unfortunately, I've been running into issues after installing nvtabular with the following error `ModuleNotFoundError: No module named 'merlin.dag.executors'`. Were you also running into this error?\n\nUpdate: I was able to successfully install the packages using the following pip install commands that resolved the error i was getting. FYI, kaggle already has cudf install when using a GPU accelerator.\n\n!pip install transformers4rec[pytorch,nvtabular]\n!pip install -U nvtabular==1.3.3" }, "source": "meta" }