{
  "id": 383121,
  "title": "17th place solution",
  "url": "/competitions/otto-recommender-system/writeups/goodra-17th-place-solution",
  "author_name": "",
  "post_date": "2023-02-10T00:37:44.280Z",
  "votes": 13,
  "comment_count": 1,
  "views": 0,
  "content": "<p>First, thanks to OTTO and Kaggle for organizing this great competition.</p>\n<p>Here I would like to share my part.</p>\n<h1>Overview</h1>\n<ul>\n<li>My part<ul>\n<li>Two-stage recommendation </li>\n<li>100-200 candidates per session</li>\n<li>100 features for ranking model</li>\n<li>CV strategy<ul>\n<li>For local validation, train by week3 and validate by week4</li>\n<li>For submission, train by week4</li></ul></li>\n<li>Score -&gt; public: 0.598, private: 0.597</li></ul></li>\n<li>Team Solution<ul>\n<li>Ensemble the outputs of my model and those of my teammates' models with rank vote</li>\n<li>Score -&gt; public: 0.601, private: 0.600</li></ul></li>\n</ul>\n<h1>Candidates generation</h1>\n<p>Generate candidates by following methods. </p>\n<p>Median numbers of candidates per session are 100 for clicks, 180 for carts and 200 for orders.</p>\n<ul>\n<li>Previously actioned</li>\n<li>Co-visitation<ul>\n<li>matrix<ul>\n<li>action2action</li>\n<li>action2click, action2cart, action2order</li>\n<li>action2buy</li>\n<li>buy2buy</li></ul></li>\n<li>weight<ul>\n<li>type</li>\n<li>recency</li>\n<li>time interval</li></ul></li></ul></li>\n<li>word2vec (use Gensim Word2Vec)</li>\n<li>node2vec (use PyTorch Geometric Node2Vec)</li>\n</ul>\n<h1>Feature engineering</h1>\n<p>Around 100 features were created to train models. </p>\n<p>I think my features are not special compared to other competitors, but some examples are given below.</p>\n<ul>\n<li>CF score<ul>\n<li>candidates score</li>\n<li>candidates rank</li></ul></li>\n<li>Item<ul>\n<li>number of {type}*</li>\n<li>ratio of number of {type} to number of actions</li>\n<li>average number of {type} in one session</li>\n<li>probability of {type} after {type} in one session</li></ul></li>\n<li>User<ul>\n<li>session size</li>\n<li>unique item number</li>\n<li>ratio of unique item number to session size</li></ul></li>\n<li>Item x User<ul>\n<li>number of {type}</li>\n<li>number of {type} weighted by recency</li>\n<li>ratio of number of {type} to session size</li>\n<li>elapsed time from last {type}</li></ul></li>\n<li>Item similarity<ul>\n<li>cosine similarity of embeddings by word2vec and node2vec<ul>\n<li>last aid to candidate</li>\n<li>average of last 3 aids to candidate</li>\n<li>average of last 5 aids to candidate</li></ul></li></ul></li>\n</ul>\n<p>*{type} denotes clicks, carts or orders.</p>\n<h1>Ranker model</h1>\n<ul>\n<li>I used LightGBM ranker<ul>\n<li>objective: lambdarank</li>\n<li>boosting_type: gbdt</li></ul></li>\n</ul>\n<h1>Ensemble</h1>\n<ul>\n<li>For each type, ensemble the output of two LightGBM rankers with slightly different candidate generation<ul>\n<li>Score -&gt; public: 0.598, private: 0.597</li></ul></li>\n<li>After that, ensemble the outputs of my model and those of my teammates' models with rank vote<ul>\n<li><a href=\"https://www.kaggle.com/dehokanta\" target=\"_blank\">@dehokanta</a> &amp; <a href=\"https://www.kaggle.com/zakopur0\" target=\"_blank\">@zakopur0</a> model<ul>\n<li>Score -&gt; public: 0.598, private: 0.597</li>\n<li>Details -&gt; <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/383493\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/383493</a></li></ul></li>\n<li><a href=\"https://www.kaggle.com/t88take\" target=\"_blank\">@t88take</a> model<ul>\n<li>Score -&gt; public: 0.594, private: 0.594</li>\n<li>Details -&gt; <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/382886\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/382886</a></li></ul></li>\n<li>Final score -&gt; <strong>public: 0.601, private: 0.600</strong></li></ul></li>\n</ul>\n<h1>What worked</h1>\n<ul>\n<li>Item similarity between last aid and candidate by word2vec contributes significantly. This feature boosts my score <br>\n+0.003.</li>\n<li>Inspired by <a href=\"https://arxiv.org/abs/1804.04212\" target=\"_blank\">this paper</a>, I tuned hyperparameters of word2vec (such as epochs, window, ns_exponent,,,) and this increased my score +0.001.</li>\n<li>I used different N of co-visit candidates based on recency and gained +0.0015. Take action2click matrix as an example, <ul>\n<li>For last aid in a session, join top 50 </li>\n<li>For aids within 30 minutes of last action, join top 20</li>\n<li>For aids older than 30 minutes, no candidates are joined</li></ul></li>\n</ul>\n<h1>What did not work</h1>\n<ul>\n<li>In a recent tabular competition I participated in (H&amp;M and Amex), LightGBM dart was better than gbdt. So, I tried dart but accuracy did not improve. It just made training time longer.</li>\n<li>I tried different models (LightGBM classifier, CatBoost classifier, CatBoost ranker) but LightGBM ranker was best in my case.</li>\n</ul>\n<h1>Environment</h1>\n<p>Google Colab Pro+</p>\n<h1>Acknowledgement</h1>\n<p>Thank <a href=\"https://www.kaggle.com/radek1\" target=\"_blank\">@radek1</a> <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/366194\" target=\"_blank\">introducing Polars</a>. I did not know Polars until taking part in this competition. Polars helped me to accelerate experiments and try many ideas.</p>\n<p>Thank <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> for sharing very helpful knowledge and codes. My co-visit matrix candidates generation is totally based on <a href=\"https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575\" target=\"_blank\">his notebook</a>.</p>",
  "messages": [
    {
      "id": "2126522",
      "postDate": "02/02/2023 09:25:18",
      "content": "<p>First, thanks to OTTO and Kaggle for organizing this great competition.</p>\n<p>Here I would like to share my part.</p>\n<h1>Overview</h1>\n<ul>\n<li>My part<ul>\n<li>Two-stage recommendation </li>\n<li>100-200 candidates per session</li>\n<li>100 features for ranking model</li>\n<li>CV strategy<ul>\n<li>For local validation, train by week3 and validate by week4</li>\n<li>For submission, train by week4</li></ul></li>\n<li>Score -&gt; public: 0.598, private: 0.597</li></ul></li>\n<li>Team Solution<ul>\n<li>Ensemble the outputs of my model and those of my teammates' models with rank vote</li>\n<li>Score -&gt; public: 0.601, private: 0.600</li></ul></li>\n</ul>\n<h1>Candidates generation</h1>\n<p>Generate candidates by following methods. </p>\n<p>Median numbers of candidates per session are 100 for clicks, 180 for carts and 200 for orders.</p>\n<ul>\n<li>Previously actioned</li>\n<li>Co-visitation<ul>\n<li>matrix<ul>\n<li>action2action</li>\n<li>action2click, action2cart, action2order</li>\n<li>action2buy</li>\n<li>buy2buy</li></ul></li>\n<li>weight<ul>\n<li>type</li>\n<li>recency</li>\n<li>time interval</li></ul></li></ul></li>\n<li>word2vec (use Gensim Word2Vec)</li>\n<li>node2vec (use PyTorch Geometric Node2Vec)</li>\n</ul>\n<h1>Feature engineering</h1>\n<p>Around 100 features were created to train models. </p>\n<p>I think my features are not special compared to other competitors, but some examples are given below.</p>\n<ul>\n<li>CF score<ul>\n<li>candidates score</li>\n<li>candidates rank</li></ul></li>\n<li>Item<ul>\n<li>number of {type}*</li>\n<li>ratio of number of {type} to number of actions</li>\n<li>average number of {type} in one session</li>\n<li>probability of {type} after {type} in one session</li></ul></li>\n<li>User<ul>\n<li>session size</li>\n<li>unique item number</li>\n<li>ratio of unique item number to session size</li></ul></li>\n<li>Item x User<ul>\n<li>number of {type}</li>\n<li>number of {type} weighted by recency</li>\n<li>ratio of number of {type} to session size</li>\n<li>elapsed time from last {type}</li></ul></li>\n<li>Item similarity<ul>\n<li>cosine similarity of embeddings by word2vec and node2vec<ul>\n<li>last aid to candidate</li>\n<li>average of last 3 aids to candidate</li>\n<li>average of last 5 aids to candidate</li></ul></li></ul></li>\n</ul>\n<p>*{type} denotes clicks, carts or orders.</p>\n<h1>Ranker model</h1>\n<ul>\n<li>I used LightGBM ranker<ul>\n<li>objective: lambdarank</li>\n<li>boosting_type: gbdt</li></ul></li>\n</ul>\n<h1>Ensemble</h1>\n<ul>\n<li>For each type, ensemble the output of two LightGBM rankers with slightly different candidate generation<ul>\n<li>Score -&gt; public: 0.598, private: 0.597</li></ul></li>\n<li>After that, ensemble the outputs of my model and those of my teammates' models with rank vote<ul>\n<li><a href=\"https://www.kaggle.com/dehokanta\" target=\"_blank\">@dehokanta</a> &amp; <a href=\"https://www.kaggle.com/zakopur0\" target=\"_blank\">@zakopur0</a> model<ul>\n<li>Score -&gt; public: 0.598, private: 0.597</li>\n<li>Details -&gt; <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/383493\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/383493</a></li></ul></li>\n<li><a href=\"https://www.kaggle.com/t88take\" target=\"_blank\">@t88take</a> model<ul>\n<li>Score -&gt; public: 0.594, private: 0.594</li>\n<li>Details -&gt; <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/382886\" target=\"_blank\">https://www.kaggle.com/competitions/otto-recommender-system/discussion/382886</a></li></ul></li>\n<li>Final score -&gt; <strong>public: 0.601, private: 0.600</strong></li></ul></li>\n</ul>\n<h1>What worked</h1>\n<ul>\n<li>Item similarity between last aid and candidate by word2vec contributes significantly. This feature boosts my score <br>\n+0.003.</li>\n<li>Inspired by <a href=\"https://arxiv.org/abs/1804.04212\" target=\"_blank\">this paper</a>, I tuned hyperparameters of word2vec (such as epochs, window, ns_exponent,,,) and this increased my score +0.001.</li>\n<li>I used different N of co-visit candidates based on recency and gained +0.0015. Take action2click matrix as an example, <ul>\n<li>For last aid in a session, join top 50 </li>\n<li>For aids within 30 minutes of last action, join top 20</li>\n<li>For aids older than 30 minutes, no candidates are joined</li></ul></li>\n</ul>\n<h1>What did not work</h1>\n<ul>\n<li>In a recent tabular competition I participated in (H&amp;M and Amex), LightGBM dart was better than gbdt. So, I tried dart but accuracy did not improve. It just made training time longer.</li>\n<li>I tried different models (LightGBM classifier, CatBoost classifier, CatBoost ranker) but LightGBM ranker was best in my case.</li>\n</ul>\n<h1>Environment</h1>\n<p>Google Colab Pro+</p>\n<h1>Acknowledgement</h1>\n<p>Thank <a href=\"https://www.kaggle.com/radek1\" target=\"_blank\">@radek1</a> <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/366194\" target=\"_blank\">introducing Polars</a>. I did not know Polars until taking part in this competition. Polars helped me to accelerate experiments and try many ideas.</p>\n<p>Thank <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> for sharing very helpful knowledge and codes. My co-visit matrix candidates generation is totally based on <a href=\"https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575\" target=\"_blank\">his notebook</a>.</p>",
      "rawMarkdown": "First, thanks to OTTO and Kaggle for organizing this great competition.\n\nHere I would like to share my part.\n\n# Overview\n- My part\n    - Two-stage recommendation \n    - 100-200 candidates per session\n    - 100 features for ranking model\n    - CV strategy\n        - For local validation, train by week3 and validate by week4\n        - For submission, train by week4\n    - Score -> public: 0.598, private: 0.597\n- Team Solution\n    - Ensemble the outputs of my model and those of my teammates' models with rank vote\n    - Score -> public: 0.601, private: 0.600\n\n# Candidates generation\nGenerate candidates by following methods. \n\nMedian numbers of candidates per session are 100 for clicks, 180 for carts and 200 for orders.\n- Previously actioned\n- Co-visitation\n    - matrix\n        - action2action\n        - action2click, action2cart, action2order\n        - action2buy\n        - buy2buy\n    - weight\n        - type\n        - recency\n        - time interval\n- word2vec (use Gensim Word2Vec)\n- node2vec (use PyTorch Geometric Node2Vec)\n\n# Feature engineering\nAround 100 features were created to train models. \n\nI think my features are not special compared to other competitors, but some examples are given below.\n- CF score\n    - candidates score\n    - candidates rank\n- Item\n    - number of {type}*\n    - ratio of number of {type} to number of actions\n    - average number of {type} in one session\n    - probability of {type} after {type} in one session\n- User\n    - session size\n    - unique item number\n    - ratio of unique item number to session size\n- Item x User\n    - number of {type}\n    - number of {type} weighted by recency\n    - ratio of number of {type} to session size\n    - elapsed time from last {type}\n- Item similarity\n    - cosine similarity of embeddings by word2vec and node2vec\n        - last aid to candidate\n        - average of last 3 aids to candidate\n        - average of last 5 aids to candidate\n\n*{type} denotes clicks, carts or orders.\n\n# Ranker model\n- I used LightGBM ranker\n    - objective: lambdarank\n    - boosting_type: gbdt\n\n# Ensemble\n- For each type, ensemble the output of two LightGBM rankers with slightly different candidate generation\n    - Score -> public: 0.598, private: 0.597\n- After that, ensemble the outputs of my model and those of my teammates' models with rank vote\n    - @dehokanta & @zakopur0 model\n        - Score -> public: 0.598, private: 0.597\n        - Details -> https://www.kaggle.com/competitions/otto-recommender-system/discussion/383493\n    - @t88take model\n        - Score -> public: 0.594, private: 0.594\n        - Details -> https://www.kaggle.com/competitions/otto-recommender-system/discussion/382886\n    - Final score -> **public: 0.601, private: 0.600**\n\n# What worked\n- Item similarity between last aid and candidate by word2vec contributes significantly. This feature boosts my score \n+0.003.\n- Inspired by [this paper](https://arxiv.org/abs/1804.04212), I tuned hyperparameters of word2vec (such as epochs, window, ns_exponent,,,) and this increased my score +0.001.\n- I used different N of co-visit candidates based on recency and gained +0.0015. Take action2click matrix as an example, \n    - For last aid in a session, join top 50 \n    - For aids within 30 minutes of last action, join top 20\n    - For aids older than 30 minutes, no candidates are joined\n\n# What did not work\n- In a recent tabular competition I participated in (H&M and Amex), LightGBM dart was better than gbdt. So, I tried dart but accuracy did not improve. It just made training time longer.\n- I tried different models (LightGBM classifier, CatBoost classifier, CatBoost ranker) but LightGBM ranker was best in my case.\n\n# Environment\nGoogle Colab Pro+\n\n# Acknowledgement\nThank @radek1 [introducing Polars](https://www.kaggle.com/competitions/otto-recommender-system/discussion/366194). I did not know Polars until taking part in this competition. Polars helped me to accelerate experiments and try many ideas.\n\nThank @cdeotte for sharing very helpful knowledge and codes. My co-visit matrix candidates generation is totally based on [his notebook](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575).",
      "votes": null
    },
    {
      "id": "2161720",
      "postDate": "02/27/2023 17:25:32",
      "content": "<p>excuse me, may i see you code? my solution is the same as yours roughly, but my ranker doesn't work</p>",
      "rawMarkdown": "excuse me, may i see you code? my solution is the same as yours roughly, but my ranker doesn't work",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2161720,
      "author_name": "wcqglhf",
      "author_url": "",
      "post_date": "02/27/2023 17:25:32",
      "content": "<p>excuse me, may i see you code? my solution is the same as yours roughly, but my ranker doesn't work</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2126522": "First, thanks to OTTO and Kaggle for organizing this great competition.\n\nHere I would like to share my part.\n\n# Overview\n- My part\n    - Two-stage recommendation \n    - 100-200 candidates per session\n    - 100 features for ranking model\n    - CV strategy\n        - For local validation, train by week3 and validate by week4\n        - For submission, train by week4\n    - Score -> public: 0.598, private: 0.597\n- Team Solution\n    - Ensemble the outputs of my model and those of my teammates' models with rank vote\n    - Score -> public: 0.601, private: 0.600\n\n# Candidates generation\nGenerate candidates by following methods. \n\nMedian numbers of candidates per session are 100 for clicks, 180 for carts and 200 for orders.\n- Previously actioned\n- Co-visitation\n    - matrix\n        - action2action\n        - action2click, action2cart, action2order\n        - action2buy\n        - buy2buy\n    - weight\n        - type\n        - recency\n        - time interval\n- word2vec (use Gensim Word2Vec)\n- node2vec (use PyTorch Geometric Node2Vec)\n\n# Feature engineering\nAround 100 features were created to train models. \n\nI think my features are not special compared to other competitors, but some examples are given below.\n- CF score\n    - candidates score\n    - candidates rank\n- Item\n    - number of {type}*\n    - ratio of number of {type} to number of actions\n    - average number of {type} in one session\n    - probability of {type} after {type} in one session\n- User\n    - session size\n    - unique item number\n    - ratio of unique item number to session size\n- Item x User\n    - number of {type}\n    - number of {type} weighted by recency\n    - ratio of number of {type} to session size\n    - elapsed time from last {type}\n- Item similarity\n    - cosine similarity of embeddings by word2vec and node2vec\n        - last aid to candidate\n        - average of last 3 aids to candidate\n        - average of last 5 aids to candidate\n\n*{type} denotes clicks, carts or orders.\n\n# Ranker model\n- I used LightGBM ranker\n    - objective: lambdarank\n    - boosting_type: gbdt\n\n# Ensemble\n- For each type, ensemble the output of two LightGBM rankers with slightly different candidate generation\n    - Score -> public: 0.598, private: 0.597\n- After that, ensemble the outputs of my model and those of my teammates' models with rank vote\n    - @dehokanta & @zakopur0 model\n        - Score -> public: 0.598, private: 0.597\n        - Details -> https://www.kaggle.com/competitions/otto-recommender-system/discussion/383493\n    - @t88take model\n        - Score -> public: 0.594, private: 0.594\n        - Details -> https://www.kaggle.com/competitions/otto-recommender-system/discussion/382886\n    - Final score -> **public: 0.601, private: 0.600**\n\n# What worked\n- Item similarity between last aid and candidate by word2vec contributes significantly. This feature boosts my score \n+0.003.\n- Inspired by [this paper](https://arxiv.org/abs/1804.04212), I tuned hyperparameters of word2vec (such as epochs, window, ns_exponent,,,) and this increased my score +0.001.\n- I used different N of co-visit candidates based on recency and gained +0.0015. Take action2click matrix as an example, \n    - For last aid in a session, join top 50 \n    - For aids within 30 minutes of last action, join top 20\n    - For aids older than 30 minutes, no candidates are joined\n\n# What did not work\n- In a recent tabular competition I participated in (H&M and Amex), LightGBM dart was better than gbdt. So, I tried dart but accuracy did not improve. It just made training time longer.\n- I tried different models (LightGBM classifier, CatBoost classifier, CatBoost ranker) but LightGBM ranker was best in my case.\n\n# Environment\nGoogle Colab Pro+\n\n# Acknowledgement\nThank @radek1 [introducing Polars](https://www.kaggle.com/competitions/otto-recommender-system/discussion/366194). I did not know Polars until taking part in this competition. Polars helped me to accelerate experiments and try many ideas.\n\nThank @cdeotte for sharing very helpful knowledge and codes. My co-visit matrix candidates generation is totally based on [his notebook](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575).",
    "2161720": "excuse me, may i see you code? my solution is the same as yours roughly, but my ranker doesn't work"
  },
  "source": "meta"
}