Thanks to my teammates @ootake, @ryoichi0917, @xia464, especially @tyonemoto for inviting me.
\nI publish the entire code.
\ncode: https://github.com/Y-Haneji/kaggle-handm
I think that the key points of this competition are these.
\nLightGBM lambdarank
\nThis competition gave no training datasets, only transactions datas and some metadatas. I needed to generate training datasets by myself. The dataset was like this.
\nThe features of customers and articles are purchase and sales of …
\nLabel is whether the article was bought target_week.
\nThe details of strategies to generate candidates is written in below.
\nOther parameters is default. I tried Optuna but it didn't work well for lambdarank because it attempted to LOWER the MAP.
\nTO BE UPDATED: Some strategies will be written in some days.
\nTo recommend popular item was effecitve especially for cold start customers. Different kind of customers should have different preference. So we split customers into some groups by their features, like age, sex, and so on.
\nIn the experiment phase, we used age, price, and transaction channel for binning. The number of splitting is 10 for age, 20 for price, and 3 for channels. Recall of this strategy was 0.06750.
\nBecause of the deadline, we have no choice but to adopt only age bins.
\nThe collaboration articles with Kangol was launched a few weeks ago from the submission week. We recommended popular collaboration articles to customers who had bought some collaboration articles. Recall of this strategy was 0.00094.
\nThe metadatas of articles from the file of articles.csv and embedding vectors of detail descriptions by tf-idf and bert are compressed to two dimensions by t-SNE. We used count encoding method to deal with the category metadatas columns.
\nWe used bert pretrained model named \"bert-uncased\", \"MRPC\", and \"SQuAD\" to extract features from articles' detail description.
\nI extracted features from images using VGG because their vectors were relatively low dimensions, which can avoid a memory problem. But even they reached 10GB files (when compressed to .npy).
\nVGG KNN doesn't worked, but I wonder if I could use other bigger models to extract features.
The true purpose of this competition is not to explore strategies which have better Precision and Recall, but to obtain better MAP@12 for submission, so I compared each strategy based on MAP@12.
\n| Strategy | \nMAP@12 change | \n
|---|---|
| trending | \nbase lines | \n
| recently | \nbase lines | \n
| popular | \nbase lines | \n
| last week | \n-0.000131 | \n
| t-SNE | \n+0.00053 | \n
| umap | \n-0.0001 | \n
| t-SNE & umap | \n-0.000898 | \n
| also bought this | \n+0.000533 | \n
| albert | \n+0.000241 | \n
| collaboration | \n+0.000134 | \n
| Bert | \n+0.000051 (didn't correlate with LB) | \n
| sequence Word2Vec w/ compression | \n+0.000367 | \n
| bin popular | \n+0.00058 | \n
| bert4rec | \n-0.00037 | \n
| gru4rec | \n-0.000347 | \n
| Implicit ALS | \n+0.000142 | \n
| sequence Word2Vec | \n-0.000563 | \n
| VGG | \n-0.000375 | \n
Note: This is the PROCESS of adding or subtracting the strategies rather than the result of comparision experiment.
\nThis is my first medal in Kaggle, so I'm very happy with my place!
\nI am new to a recommend problem, but my teammates helped me a lot, and @paweljankiewicz and @lihaorocky was so kind to answer my questions in discussion.
", "messages": [ { "id": "1783657", "postDate": "05/10/2022 15:00:05", "content": "Thanks to my teammates @ootake, @ryoichi0917, @xia464, especially @tyonemoto for inviting me.
\nI publish the entire code.
\ncode: https://github.com/Y-Haneji/kaggle-handm
I think that the key points of this competition are these.
\nLightGBM lambdarank
\nThis competition gave no training datasets, only transactions datas and some metadatas. I needed to generate training datasets by myself. The dataset was like this.
\nThe features of customers and articles are purchase and sales of …
\nLabel is whether the article was bought target_week.
\nThe details of strategies to generate candidates is written in below.
\nOther parameters is default. I tried Optuna but it didn't work well for lambdarank because it attempted to LOWER the MAP.
\nTO BE UPDATED: Some strategies will be written in some days.
\nTo recommend popular item was effecitve especially for cold start customers. Different kind of customers should have different preference. So we split customers into some groups by their features, like age, sex, and so on.
\nIn the experiment phase, we used age, price, and transaction channel for binning. The number of splitting is 10 for age, 20 for price, and 3 for channels. Recall of this strategy was 0.06750.
\nBecause of the deadline, we have no choice but to adopt only age bins.
\nThe collaboration articles with Kangol was launched a few weeks ago from the submission week. We recommended popular collaboration articles to customers who had bought some collaboration articles. Recall of this strategy was 0.00094.
\nThe metadatas of articles from the file of articles.csv and embedding vectors of detail descriptions by tf-idf and bert are compressed to two dimensions by t-SNE. We used count encoding method to deal with the category metadatas columns.
\nWe used bert pretrained model named \"bert-uncased\", \"MRPC\", and \"SQuAD\" to extract features from articles' detail description.
\nI extracted features from images using VGG because their vectors were relatively low dimensions, which can avoid a memory problem. But even they reached 10GB files (when compressed to .npy).
\nVGG KNN doesn't worked, but I wonder if I could use other bigger models to extract features.
The true purpose of this competition is not to explore strategies which have better Precision and Recall, but to obtain better MAP@12 for submission, so I compared each strategy based on MAP@12.
\n| Strategy | \nMAP@12 change | \n
|---|---|
| trending | \nbase lines | \n
| recently | \nbase lines | \n
| popular | \nbase lines | \n
| last week | \n-0.000131 | \n
| t-SNE | \n+0.00053 | \n
| umap | \n-0.0001 | \n
| t-SNE & umap | \n-0.000898 | \n
| also bought this | \n+0.000533 | \n
| albert | \n+0.000241 | \n
| collaboration | \n+0.000134 | \n
| Bert | \n+0.000051 (didn't correlate with LB) | \n
| sequence Word2Vec w/ compression | \n+0.000367 | \n
| bin popular | \n+0.00058 | \n
| bert4rec | \n-0.00037 | \n
| gru4rec | \n-0.000347 | \n
| Implicit ALS | \n+0.000142 | \n
| sequence Word2Vec | \n-0.000563 | \n
| VGG | \n-0.000375 | \n
Note: This is the PROCESS of adding or subtracting the strategies rather than the result of comparision experiment.
\nThis is my first medal in Kaggle, so I'm very happy with my place!
\nI am new to a recommend problem, but my teammates helped me a lot, and @paweljankiewicz and @lihaorocky was so kind to answer my questions in discussion.
", "rawMarkdown": "# 63th Place Solution\n\nThanks to my teammates @ootake, @ryoichi0917, @xia464, especially @tyonemoto for inviting me.\n\n## Code (updated in 2022-07-12)\n\nI publish the entire code.\ncode: https://github.com/Y-Haneji/kaggle-handm\n\n## Last Submission\n\n- (lr=0.1) : Private LB: 0.02813, Public LB: 0.02826, Validation: 0.033623\n- (lr=0.02): Private LB: 0.02772, Public LB: 0.0277, Validation: 0.033814\n\n## Summary\n\nI think that the key points of this competition are these.\n\n- Stable validation.\n- Two stage: generating candidates by some strategies, then ranking them.\n- Better strategies and scores from them.\n- Using full information like not only transactions but also detail descriptions, images, and so on.\n\n## Ranking Model (me)\n\n### Model\n\nLightGBM lambdarank\n\n### Dataset\n\nThis competition gave no training datasets, only transactions datas and some metadatas. I needed to generate training datasets by myself. The dataset was like this.\n\n- customer_id\n- article_id (candidate)\n- score from generative strategy (may not just one score)\n- flags wheter the candidate included each strategy\n- other features of customers\n- other features of articles\n- label (0 or 1)\n\nThe features of customers and articles are purchase and sales of ...\n\n- each week\n- agregation (sum, mean, max, min) of all weeks\n- difference and division between each week and aggregation\n\nLabel is whether the article was bought target_week.\n\nThe details of strategies to generate candidates is written in below.\n\n### Training\n\n- 5 target weeks, this means 5 folds training dataset.\n- For training, I only generated candidates for the customers who had some purchases in the target week.\n- Model Validation: NDCG@12. The dataset of next target week was validation dataset. The last target week's training dataset had no validation datasets.\n- Submission Validation: MAP@12 for last week's transactions.\n\n\n\n### Parameter\n\n- objective: lambdarank\n- lr: 0.1 or 0.02\n- feature_fraction: 0.8\n- baggin_fraction: 0.8\n- early_stopping: 10 or 50 rounds (depend on lr)\n\nOther parameters is default. I tried Optuna but it didn't work well for lambdarank because it attempted to LOWER the MAP.\n\n### Prediction (Submission)\n\n- For prediction, I should generate candidates for all the customers, which causes a memory problem.\n- Split the customers into batches.\n- Ensemble 5 target week's models.\n- Generally take longer time than training\n- If validation was not good, I didn't compute submission.\n\n## Candidates Strategies\n\n### Worked (included last submission)\n\n- trending (original: [here](https://www.kaggle.com/code/byfone/h-m-trending-products-weekly))\n- recently purchased\n- bin popular\n- last week for each customer\n- also bought this (original: [here](https://www.kaggle.com/code/cdeotte/customers-who-bought-this-frequently-buy-this))\n- collabo\n- t-SNE KNN\n- Albert KNN\n- sequence Word2Vec\n\n### Not Worked\n\n- Bert KNN\n- VGG KNN\n- Gru4Rec, Bert4Rec (LSTM) (original: [here](https://www.kaggle.com/code/astrung/lstm-model-with-item-infor-fix-missing-last-item))\n- Implicit ALS\n\n### Details of Some Strategies\n\nTO BE UPDATED: Some strategies will be written in some days.\n\n#### bin popular (by me and by @tyonemoto)\n\nTo recommend popular item was effecitve especially for cold start customers. Different kind of customers should have different preference. So we split customers into some groups by their features, like age, sex, and so on.\n\nIn the experiment phase, we used age, price, and transaction channel for binning. The number of splitting is 10 for age, 20 for price, and 3 for channels. Recall of this strategy was 0.06750.\n\nBecause of the deadline, we have no choice but to adopt only age bins.\n\n#### collabo (by @tyonemoto)\n\nThe collaboration articles with Kangol was launched a few weeks ago from the submission week. We recommended popular collaboration articles to customers who had bought some collaboration articles. Recall of this strategy was 0.00094.\n\n#### t-SNE (by @tyonemoto)\n\nThe metadatas of articles from the file of articles.csv and embedding vectors of detail descriptions by tf-idf and bert are compressed to two dimensions by t-SNE. We used count encoding method to deal with the category metadatas columns.\n\n#### Albert (by @ryoichi0917)\n\n#### sequence Word2Vec (by @tyonemoto)\n\n#### Bert (by @tyonemoto and @ootake)\n\nWe used bert pretrained model named \"bert-uncased\", \"MRPC\", and \"SQuAD\" to extract features from articles' detail description.\n\n#### VGG (by me)\n\nI extracted features from images using VGG because their vectors were relatively low dimensions, which can avoid a memory problem. But even they reached 10GB files (when compressed to .npy).\nVGG KNN doesn't worked, but I wonder if I could use other bigger models to extract features.\n\n#### Implicit ALS (by @ootake)\n\n### Comparision\n\nThe true purpose of this competition is not to explore strategies which have better Precision and Recall, but to obtain better MAP@12 for submission, so I compared each strategy based on MAP@12.\n\n| Strategy | MAP@12 change |\n| --- | --- |\n| trending | base lines |\n| recently | base lines |\n| popular | base lines |\n| last week | -0.000131 |\n| t-SNE | +0.00053 |\n| umap | -0.0001 |\n| t-SNE & umap | -0.000898 |\n| also bought this | +0.000533 |\n| albert | +0.000241 |\n| collaboration | +0.000134 |\n| Bert | +0.000051 (didn't correlate with LB) |\n| sequence Word2Vec w/ compression | +0.000367 |\n| bin popular | +0.00058 |\n| bert4rec | -0.00037 |\n| gru4rec | -0.000347 |\n| Implicit ALS | +0.000142 |\n| sequence Word2Vec | -0.000563 |\n| VGG | -0.000375 |\n\nNote: This is the PROCESS of adding or subtracting the strategies rather than the result of comparision experiment.\n\n## Computing Resources\n\n- Untill 2 weeks from deadlines: MacBook Air (M1, 2020) (8 core CPU, 16 GM RAM, no GPU for deep learning).\n- Last 2 weeks: GCP instances to deal with more candidates. (48 core CPU, 384 GB RAM)\n\n## Some Story\n\nThis is my first medal in Kaggle, so I'm very happy with my place! \n\nI am new to a recommend problem, but my teammates helped me a lot, and @paweljankiewicz and @lihaorocky was so kind to answer my questions in discussion.", "votes": null }, { "id": "1784101", "postDate": "05/10/2022 23:55:41", "content": "Congratulations on your medal! This is an impressive and informative post. Your teammates seem to have been a great help, and it's great that you were able to learn from others in the community. Recommendation problems can be tough, but it sounds like you did a great job. Keep up the good work!
", "rawMarkdown": "Congratulations on your medal! This is an impressive and informative post. Your teammates seem to have been a great help, and it's great that you were able to learn from others in the community. Recommendation problems can be tough, but it sounds like you did a great job. Keep up the good work!", "votes": null }, { "id": "1784510", "postDate": "05/11/2022 08:10:49", "content": "@satoshidatamoto
\nThank you for your celebrating!
Congratulations on your medal! This is an impressive and informative post. Your teammates seem to have been a great help, and it's great that you were able to learn from others in the community. Recommendation problems can be tough, but it sounds like you did a great job. Keep up the good work!
", "votes": null, "replies": [ { "id": 1784510, "author_name": "hanejiyuto", "author_url": "", "post_date": "05/11/2022 08:10:49", "content": "@satoshidatamoto
\nThank you for your celebrating!