{ "id": 324223, "title": "10th place solution", "url": "/competitions/h-and-m-personalized-fashion-recommendations/writeups/flg2k-10th-place-solution", "author_name": "", "post_date": "2022-05-10T15:19:01.825471Z", "votes": 25, "comment_count": 3, "views": 0, "content": "

First, thanks for the great competition to H&M and Kaggle. I enjoyed it a lot, it's a very nice kind of challenge, as it's not heavily compute-gated and allows for a wide range of solutions.

\n

Many Kagglers already shared their solution and since I followed the same framework as most, I will only mention some differences.

\n
CV-LB correlation
\n
    \n
  1. Many people reported a gap between CV and LB or a \"slowdown\" when their models improved. I never experienced that, a linear fit between my CV and LB gives: f(x) = 0.793x + c with R²=98.96%.
    2. \n
  2. The problem is most likely caused by information leaking from the future. For example when I train a CF-model on all data up to 09-15 and later use this to make predictions for the week 09-01, I will boost my CV by implicitly using purchase information from the future.
    3. \n
  3. I avoided this by training a full set of models for every week. I tagged the models with the last date used to create it. That way it was impossible to leak any information.
    4. \n
  4. While this sounds good, it also was my \"downfall\". I simply did not have enough compute to do this for many different models and weeks. Which is why my recall methods were much less diverse than those of the top solutions and which is why I only used four weeks of training data in the end.
    5. \n
\n
Retrieval strategy / Features
\n
    \n
  1. In addition to what many people used (rebuys, I2I, CF, Top-Sellers etc.), I created a simple sales prediction model. Based on recent sales of an article it predicted next weeks sales.
    2. \n
  2. The main use of this model was helping to detect products that are phasing in or out (since we don't have that data).
    3. \n
  3. Example: an article that is only available for a short time (special collaboration etc.) might have daily sales like: 0 0 28 431 389 105 32 11. Using recent popularity will massively overestimate next weeks sales while the prediction model learned the article is phasing out (out of stock).
    4. \n
  4. It was in the end one of the most used features, probably since it covers popularity and (future) availability in one.
    5. \n
  5. My work horse for similarity was LightFM (BPR and WARP, with and without item features). I used it for recommendation and item-to-item similarity (by comparing feature embeddings).
    6. \n
\n
Hardware / Optimization
\n
    \n
  1. Desktop with 12C and 64GB Ram.
    2. \n
  2. I used feature storing liberally.
    3. \n
  3. To decouple training data size from Ram available, I trained my LightGBM model from HDF5.
    4. \n
  4. Final model was an ensemble of four LightGBM models. But the improvement was rather small, ~1% compared to the single best model.
    5. \n
", "messages": [ { "id": "1783676", "postDate": "05/10/2022 15:19:01", "content": "

First, thanks for the great competition to H&M and Kaggle. I enjoyed it a lot, it's a very nice kind of challenge, as it's not heavily compute-gated and allows for a wide range of solutions.

\n

Many Kagglers already shared their solution and since I followed the same framework as most, I will only mention some differences.

\n
CV-LB correlation
\n
    \n
  1. Many people reported a gap between CV and LB or a \"slowdown\" when their models improved. I never experienced that, a linear fit between my CV and LB gives: f(x) = 0.793x + c with R²=98.96%.
    2. \n
  2. The problem is most likely caused by information leaking from the future. For example when I train a CF-model on all data up to 09-15 and later use this to make predictions for the week 09-01, I will boost my CV by implicitly using purchase information from the future.
    3. \n
  3. I avoided this by training a full set of models for every week. I tagged the models with the last date used to create it. That way it was impossible to leak any information.
    4. \n
  4. While this sounds good, it also was my \"downfall\". I simply did not have enough compute to do this for many different models and weeks. Which is why my recall methods were much less diverse than those of the top solutions and which is why I only used four weeks of training data in the end.
    5. \n
\n
Retrieval strategy / Features
\n
    \n
  1. In addition to what many people used (rebuys, I2I, CF, Top-Sellers etc.), I created a simple sales prediction model. Based on recent sales of an article it predicted next weeks sales.
    2. \n
  2. The main use of this model was helping to detect products that are phasing in or out (since we don't have that data).
    3. \n
  3. Example: an article that is only available for a short time (special collaboration etc.) might have daily sales like: 0 0 28 431 389 105 32 11. Using recent popularity will massively overestimate next weeks sales while the prediction model learned the article is phasing out (out of stock).
    4. \n
  4. It was in the end one of the most used features, probably since it covers popularity and (future) availability in one.
    5. \n
  5. My work horse for similarity was LightFM (BPR and WARP, with and without item features). I used it for recommendation and item-to-item similarity (by comparing feature embeddings).
    6. \n
\n
Hardware / Optimization
\n
    \n
  1. Desktop with 12C and 64GB Ram.
    2. \n
  2. I used feature storing liberally.
    3. \n
  3. To decouple training data size from Ram available, I trained my LightGBM model from HDF5.
    4. \n
  4. Final model was an ensemble of four LightGBM models. But the improvement was rather small, ~1% compared to the single best model.
    5. \n
", "rawMarkdown": "First, thanks for the great competition to H&M and Kaggle. I enjoyed it a lot, it's a very nice kind of challenge, as it's not heavily compute-gated and allows for a wide range of solutions.\n\nMany Kagglers already shared their solution and since I followed the same framework as most, I will only mention some differences.\n\n###### CV-LB correlation\n1. Many people reported a gap between CV and LB or a \"slowdown\" when their models improved. I never experienced that, a linear fit between my CV and LB gives: f(x) = 0.793x + c with R²=98.96%. \n2. The problem is most likely caused by information leaking from the future. For example when I train a CF-model on all data up to 09-15 and later use this to make predictions for the week 09-01, I will boost my CV by implicitly using purchase information from the future.\n3. I avoided this by training a full set of models for every week. I tagged the models with the last date used to create it. That way it was impossible to leak any information.\n4. While this sounds good, it also was my \"downfall\". I simply did not have enough compute to do this for many different models and weeks. Which is why my recall methods were much less diverse than those of the top solutions and which is why I only used four weeks of training data in the end.\n\n###### Retrieval strategy / Features\n1. In addition to what many people used (rebuys, I2I, CF, Top-Sellers etc.), I created a simple sales prediction model. Based on recent sales of an article it predicted next weeks sales.\n2. The main use of this model was helping to detect products that are phasing in or out (since we don't have that data).\n3. Example: an article that is only available for a short time (special collaboration etc.) might have daily sales like: 0 0 28 431 389 105 32 11. Using recent popularity will massively overestimate next weeks sales while the prediction model learned the article is phasing out (out of stock).\n4. It was in the end one of the most used features, probably since it covers popularity and (future) availability in one.\n5. My work horse for similarity was LightFM (BPR and WARP, with and without item features). I used it for recommendation and item-to-item similarity (by comparing feature embeddings).\n\n###### Hardware / Optimization\n1. Desktop with 12C and 64GB Ram.\n2. I used feature storing liberally.\n3. To decouple training data size from Ram available, I trained my LightGBM model from HDF5.\n4. Final model was an ensemble of four LightGBM models. But the improvement was rather small, ~1% compared to the single best model.", "votes": null }, { "id": "1783753", "postDate": "05/10/2022 16:38:28", "content": "

The prediction model to try to capture out of stock items seems a very good idea. Thanks for sharing and congratulations for gold.

", "rawMarkdown": "The prediction model to try to capture out of stock items seems a very good idea. Thanks for sharing and congratulations for gold.", "votes": null }, { "id": "1783948", "postDate": "05/10/2022 19:40:07", "content": "

Congrats for achieving that position/Gold medal and thank you for sharing your valuable solution.

", "rawMarkdown": "Congrats for achieving that position/Gold medal and thank you for sharing your valuable solution.", "votes": null }, { "id": "1784103", "postDate": "05/10/2022 23:56:55", "content": "

Great solution! You have inspired me to try a similar technique for the next competition :)

", "rawMarkdown": "Great solution! You have inspired me to try a similar technique for the next competition :)", "votes": null } ], "comments": [ { "id": 1783753, "author_name": "igorkf", "author_url": "", "post_date": "05/10/2022 16:38:28", "content": "

The prediction model to try to capture out of stock items seems a very good idea. Thanks for sharing and congratulations for gold.

", "votes": null, "replies": [] }, { "id": 1783948, "author_name": "mpwolke", "author_url": "", "post_date": "05/10/2022 19:40:07", "content": "

Congrats for achieving that position/Gold medal and thank you for sharing your valuable solution.

", "votes": null, "replies": [] }, { "id": 1784103, "author_name": "", "author_url": "", "post_date": "05/10/2022 23:56:55", "content": "

Great solution! You have inspired me to try a similar technique for the next competition :)

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1783676": "First, thanks for the great competition to H&M and Kaggle. I enjoyed it a lot, it's a very nice kind of challenge, as it's not heavily compute-gated and allows for a wide range of solutions.\n\nMany Kagglers already shared their solution and since I followed the same framework as most, I will only mention some differences.\n\n###### CV-LB correlation\n1. Many people reported a gap between CV and LB or a \"slowdown\" when their models improved. I never experienced that, a linear fit between my CV and LB gives: f(x) = 0.793x + c with R²=98.96%. \n2. The problem is most likely caused by information leaking from the future. For example when I train a CF-model on all data up to 09-15 and later use this to make predictions for the week 09-01, I will boost my CV by implicitly using purchase information from the future.\n3. I avoided this by training a full set of models for every week. I tagged the models with the last date used to create it. That way it was impossible to leak any information.\n4. While this sounds good, it also was my \"downfall\". I simply did not have enough compute to do this for many different models and weeks. Which is why my recall methods were much less diverse than those of the top solutions and which is why I only used four weeks of training data in the end.\n\n###### Retrieval strategy / Features\n1. In addition to what many people used (rebuys, I2I, CF, Top-Sellers etc.), I created a simple sales prediction model. Based on recent sales of an article it predicted next weeks sales.\n2. The main use of this model was helping to detect products that are phasing in or out (since we don't have that data).\n3. Example: an article that is only available for a short time (special collaboration etc.) might have daily sales like: 0 0 28 431 389 105 32 11. Using recent popularity will massively overestimate next weeks sales while the prediction model learned the article is phasing out (out of stock).\n4. It was in the end one of the most used features, probably since it covers popularity and (future) availability in one.\n5. My work horse for similarity was LightFM (BPR and WARP, with and without item features). I used it for recommendation and item-to-item similarity (by comparing feature embeddings).\n\n###### Hardware / Optimization\n1. Desktop with 12C and 64GB Ram.\n2. I used feature storing liberally.\n3. To decouple training data size from Ram available, I trained my LightGBM model from HDF5.\n4. Final model was an ensemble of four LightGBM models. But the improvement was rather small, ~1% compared to the single best model.", "1783753": "The prediction model to try to capture out of stock items seems a very good idea. Thanks for sharing and congratulations for gold.", "1783948": "Congrats for achieving that position/Gold medal and thank you for sharing your valuable solution.", "1784103": "Great solution! You have inspired me to try a similar technique for the next competition :)" }, "source": "meta" }