-No performance improvements when using 1% of negatives samples vs all negative samples. --> Use subsampling.
\n-When using a relative low learning rate ( 0.1 ) my carts recall got higher whereas my orders recall got lower, and vice-versa --> maybe train a slow learner for carts and fast learner for orders ?
\n-Interactions features are more important than users and items features ( from the feature importance of the GBDT ).
\n-The Ranker might be better than heuristic approaches in some cases, and vice-versa --> you can use both of them.
\n-Scale pos weight parameter ( ratio between negatives and positive samples ) does not help improving my recall score.
\nThis competition is kind of painful when using only kaggle kernel π€
", "messages": [ { "id": "2116944", "postDate": "01/26/2023 20:39:51", "content": "-No performance improvements when using 1% of negatives samples vs all negative samples. --> Use subsampling.
\n-When using a relative low learning rate ( 0.1 ) my carts recall got higher whereas my orders recall got lower, and vice-versa --> maybe train a slow learner for carts and fast learner for orders ?
\n-Interactions features are more important than users and items features ( from the feature importance of the GBDT ).
\n-The Ranker might be better than heuristic approaches in some cases, and vice-versa --> you can use both of them.
\n-Scale pos weight parameter ( ratio between negatives and positive samples ) does not help improving my recall score.
\nThis competition is kind of painful when using only kaggle kernel π€
", "rawMarkdown": "No performance improvements when using 1% of negatives samples vs all negative samples. --> Use subsampling.\n\n-When using a relative low learning rate ( 0.1 ) my carts recall got higher whereas my orders recall got lower, and vice-versa --> maybe train a slow learner for carts and fast learner for orders ?\n\n-Interactions features are more important than users and items features ( from the feature importance of the GBDT ).\n\n-The Ranker might be better than heuristic approaches in some cases, and vice-versa --> you can use both of them.\n\n-Scale pos weight parameter ( ratio between negatives and positive samples ) does not help improving my recall score.\n\n\nThis competition is kind of painful when using only kaggle kernel π€", "votes": null }, { "id": "2116996", "postDate": "01/26/2023 21:49:52", "content": "Tuning downsample percentage also didn't work for me. I can manipulate CV with it, but cannot improve LB.
", "rawMarkdown": "Tuning downsample percentage also didn't work for me. I can manipulate CV with it, but cannot improve LB.", "votes": null }, { "id": "2117006", "postDate": "01/26/2023 22:17:21", "content": "Totally agree on this point, thanks for the remark !
", "rawMarkdown": "Totally agree on this point, thanks for the remark !", "votes": null }, { "id": "2117117", "postDate": "01/27/2023 02:53:58", "content": "But improved the training speed when you have a lot of candidates π
", "rawMarkdown": "But improved the training speed when you have a lot of candidates π", "votes": null }, { "id": "2117118", "postDate": "01/27/2023 02:57:40", "content": "I am surprised that you can do this competition using only kaggle kernelπ
", "rawMarkdown": "I am surprised that you can do this competition using only kaggle kernelπ", "votes": null }, { "id": "2117441", "postDate": "01/27/2023 09:22:29", "content": "I can assure you it's true π. I have become an expert in CHUNK, I'm also using a dozen notebooks for the processing of each step.
\nIf you don't mind @buumoo, What are you using for this competition ( i mean your setup ) ? I heard about google colab Pro + in some threads but don't know if it's enough for kaggle competitions, however I'm sure that it's better than my actual setup on kaggle kernel XD
", "rawMarkdown": "I can assure you it's true π. I have become an expert in CHUNK, I'm also using a dozen notebooks for the processing of each step.\n\nIf you don't mind @buumoo, What are you using for this competition ( i mean your setup ) ? I heard about google colab Pro + in some threads but don't know if it's enough for kaggle competitions, however I'm sure that it's better than my actual setup on kaggle kernel XD", "votes": null }, { "id": "2117454", "postDate": "01/27/2023 09:30:44", "content": "@gongbi How many candidates did you extract from the co-visitation matrices ??
", "rawMarkdown": "gongbi How many candidates did you extract from the co-visitation matrices ??", "votes": null }, { "id": "2117516", "postDate": "01/27/2023 10:28:03", "content": "My personal pc is poor as well, only 48gb ram with a 3070 GPU. So I have to rent cloud GPU server to do this competition. Previously I used google colab before it changed from monthly subscription to current compute units system. It becomes much more expensive in this way, for me at least, which depands on how much of your usage.
\nFor this competition, 128gb ram with a 3090 GPU is enough for all my methods.
\nGCP and AWS are secure but expensive.
\nSome small private cloud platforms are much cheaper, although you may face some security issue. Maybe your code/solution will be stolen. such as https://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/337853, so I will not advertise or mention any of them here.
May I ask how you do neg sample? by filter out those who do not have any cart/order labels?(e.g. A session whose 100 candidates all fail to hit the groundtruth)
", "rawMarkdown": "May I ask how you do neg sample? by filter out those who do not have any cart/order labels?(e.g. A session whose 100 candidates all fail to hit the groundtruth)", "votes": null }, { "id": "2118567", "postDate": "01/28/2023 05:48:35", "content": "@yuzhang0422 Exactly, I just randomly remove samples that have target=0. I do that for each dataframe (clicks, carts and orders):
\nprint(cand_carts.shape)\n\ncand_carts = pl.concat([\n cand_carts.filter(~(pl.col(\"cart_target\") == 0)),\n cand_carts.filter((pl.col(\"cart_target\") == 0)).sample(frac=CFG.carts_frac, seed=CFG.seed)\n ]).sort(['session'])\n\nprint(cand_carts.shape)\nTuning downsample percentage also didn't work for me. I can manipulate CV with it, but cannot improve LB.
", "votes": null, "replies": [ { "id": 2117006, "author_name": "rayanaay", "author_url": "", "post_date": "01/26/2023 22:17:21", "content": "Totally agree on this point, thanks for the remark !
", "votes": null, "replies": [] }, { "id": 2117117, "author_name": "gongbi", "author_url": "", "post_date": "01/27/2023 02:53:58", "content": "But improved the training speed when you have a lot of candidates π
", "votes": null, "replies": [ { "id": 2117454, "author_name": "rayanaay", "author_url": "", "post_date": "01/27/2023 09:30:44", "content": "@gongbi How many candidates did you extract from the co-visitation matrices ??
", "votes": null, "replies": [] } ] }, { "id": 2118502, "author_name": "yuzhang0422", "author_url": "", "post_date": "01/28/2023 04:02:10", "content": "May I ask how you do neg sample? by filter out those who do not have any cart/order labels?(e.g. A session whose 100 candidates all fail to hit the groundtruth)
", "votes": null, "replies": [ { "id": 2118567, "author_name": "hinepo", "author_url": "", "post_date": "01/28/2023 05:48:35", "content": "@yuzhang0422 Exactly, I just randomly remove samples that have target=0. I do that for each dataframe (clicks, carts and orders):
\nprint(cand_carts.shape)\n\ncand_carts = pl.concat([\n cand_carts.filter(~(pl.col(\"cart_target\") == 0)),\n cand_carts.filter((pl.col(\"cart_target\") == 0)).sample(frac=CFG.carts_frac, seed=CFG.seed)\n ]).sort(['session'])\n\nprint(cand_carts.shape)\nI am surprised that you can do this competition using only kaggle kernelπ
", "votes": null, "replies": [ { "id": 2117441, "author_name": "rayanaay", "author_url": "", "post_date": "01/27/2023 09:22:29", "content": "I can assure you it's true π. I have become an expert in CHUNK, I'm also using a dozen notebooks for the processing of each step.
\nIf you don't mind @buumoo, What are you using for this competition ( i mean your setup ) ? I heard about google colab Pro + in some threads but don't know if it's enough for kaggle competitions, however I'm sure that it's better than my actual setup on kaggle kernel XD
", "votes": null, "replies": [ { "id": 2117516, "author_name": "buumoo", "author_url": "", "post_date": "01/27/2023 10:28:03", "content": "My personal pc is poor as well, only 48gb ram with a 3070 GPU. So I have to rent cloud GPU server to do this competition. Previously I used google colab before it changed from monthly subscription to current compute units system. It becomes much more expensive in this way, for me at least, which depands on how much of your usage.
\nFor this competition, 128gb ram with a 3090 GPU is enough for all my methods.
\nGCP and AWS are secure but expensive.
\nSome small private cloud platforms are much cheaper, although you may face some security issue. Maybe your code/solution will be stolen. such as https://www.kaggle.com/competitions/us-patent-phrase-to-phrase-matching/discussion/337853, so I will not advertise or mention any of them here.