Hello community, I want to share with you my experience with this competition, and the big/dumb mistake which that made me lose 1 month in this competition.
\nI think that many of us struggled with the ranker, and that the task was not so easy. Personnally, I have realized several of my errors during the last month, but this one is definitely the worst.
\nThe mistake is about the candidate generation:
\n \n aids2 = (itertools.chain(*[top_20_buys[aid] aid unique_aids aid top_20_buys]))\n \n aids3 = (itertools.chain(*[top_20_buy2buy[aid] aid unique_buys aid top_20_buy2buy]))\n \n top_aids2 = [aid2 aid2, cnt Counter(aids2+aids3).most_common(candidates_number) aid2 unique_aids] \n result = ((unique_aids + top_aids2[:candidates_number - (unique_aids)]))\n top_orderss = [aid aid top_orders aid result ]\n session, result + top_orderss[:candidates_number - (result)]\nDid you see it ?
\nIn order to avoid any kind of duplicates at the level of my candidates ( for caution ), I used set, but I totally forgot that the actual order of the candidates has a capital importance, an order that has been altered because set actually rearranged my candidates in ascending order.
That's all for me, I noticed this error a little late I admit, I tried to investigate elsewhere but it never really solved my barrier to crossing the benchmark made by the heuristic approach.
\nBy removing the famous set, the ranker goes from 0.565 to 0.579 in the LB with a single model. However, with 5 submissions left, there is little chance that I will improve further, with all the tests and the time it takes to execute.
Finally, I wanted to say thanks to all the kagglers who gave me directions in this competition, I learned a lot from you!
\nAnd you ! how do you make sure every step is correct ?
\nPS: be careful with every line of code you write, this advice is for me first haha
", "messages": [ { "id": "2121565", "postDate": "01/30/2023 12:07:50", "content": "Hello community, I want to share with you my experience with this competition, and the big/dumb mistake which that made me lose 1 month in this competition.
\nI think that many of us struggled with the ranker, and that the task was not so easy. Personnally, I have realized several of my errors during the last month, but this one is definitely the worst.
\nThe mistake is about the candidate generation:
\n \n aids2 = (itertools.chain(*[top_20_buys[aid] aid unique_aids aid top_20_buys]))\n \n aids3 = (itertools.chain(*[top_20_buy2buy[aid] aid unique_buys aid top_20_buy2buy]))\n \n top_aids2 = [aid2 aid2, cnt Counter(aids2+aids3).most_common(candidates_number) aid2 unique_aids] \n result = ((unique_aids + top_aids2[:candidates_number - (unique_aids)]))\n top_orderss = [aid aid top_orders aid result ]\n session, result + top_orderss[:candidates_number - (result)]\nDid you see it ?
\nIn order to avoid any kind of duplicates at the level of my candidates ( for caution ), I used set, but I totally forgot that the actual order of the candidates has a capital importance, an order that has been altered because set actually rearranged my candidates in ascending order.
That's all for me, I noticed this error a little late I admit, I tried to investigate elsewhere but it never really solved my barrier to crossing the benchmark made by the heuristic approach.
\nBy removing the famous set, the ranker goes from 0.565 to 0.579 in the LB with a single model. However, with 5 submissions left, there is little chance that I will improve further, with all the tests and the time it takes to execute.
Finally, I wanted to say thanks to all the kagglers who gave me directions in this competition, I learned a lot from you!
\nAnd you ! how do you make sure every step is correct ?
\nPS: be careful with every line of code you write, this advice is for me first haha
", "rawMarkdown": "Hello community, I want to share with you my experience with this competition, and the big/dumb mistake which that made me lose 1 month in this competition.\n\nI think that many of us struggled with the ranker, and that the task was not so easy. Personnally, I have realized several of my errors during the last month, but this one is definitely the worst.\n\nThe mistake is about the candidate generation:\n\n```python\n # USE \"CART ORDER\" CO-VISITATION MATRIX\n aids2 = list(itertools.chain(*[top_20_buys[aid] for aid in unique_aids if aid in top_20_buys]))\n # USE \"BUY2BUY\" CO-VISITATION MATRIX\n aids3 = list(itertools.chain(*[top_20_buy2buy[aid] for aid in unique_buys if aid in top_20_buy2buy]))\n # RERANK CANDIDATES\n top_aids2 = [aid2 for aid2, cnt in Counter(aids2+aids3).most_common(candidates_number) if aid2 not in unique_aids] \n result = list(set(unique_aids + top_aids2[:candidates_number - len(unique_aids)]))\n top_orderss = [aid for aid in top_orders if aid not in result ]\n return session, result + top_orderss[:candidates_number - len(result)]\n```\nDid you see it ?\nIn order to avoid any kind of duplicates at the level of my candidates ( for caution ), I used `set`, but I totally forgot that the actual order of the candidates has a capital importance, an order that has been altered because `set` actually rearranged my candidates in ascending order.\n\n\nThat's all for me, I noticed this error a little late I admit, I tried to investigate elsewhere but it never really solved my barrier to crossing the benchmark made by the heuristic approach.\n\nBy removing the famous `set`, the ranker goes from 0.565 to 0.579 in the LB with a single model. However, with 5 submissions left, there is little chance that I will improve further, with all the tests and the time it takes to execute.\n\nFinally, I wanted to say thanks to all the kagglers who gave me directions in this competition, I learned a lot from you!\n\n*And you ! how do you make sure every step is correct ?*\n\n**PS: be careful with every line of code you write, this advice is for me first haha**", "votes": null }, { "id": "2121644", "postDate": "01/30/2023 13:13:52", "content": "I know this might be hard or might sound a bit insane but this is the truth:
\nLosing a month of work happens to many of us in the field..
\nThe only thing it means is that you are practicing and leveling up your skill. Look on the bright side even if it is hard to see.
And about the sorting mistake itself, it can happen to anyone it is really tricky.
\nTake care!
", "rawMarkdown": "I know this might be hard or might sound a bit insane but this is the truth:\nLosing a month of work happens to many of us in the field.. \nThe only thing it means is that you are practicing and leveling up your skill. Look on the bright side even if it is hard to see. \n\nAnd about the sorting mistake itself, it can happen to anyone it is really tricky.\n\nTake care!", "votes": null }, { "id": "2121699", "postDate": "01/30/2023 13:53:20", "content": "Where are the sort in your code?
\nAnd how you fixed it?
", "rawMarkdown": "Where are the **sort** in your code? \n\nAnd how you fixed it?", "votes": null }, { "id": "2121783", "postDate": "01/30/2023 14:47:57", "content": "\n\nWhere are the sort in your code?
\nAnd how you fixed it?
\n
I don't think it is the sort, but rather the set creation. Sets do not preserve or guarantee order.
", "rawMarkdown": "> Where are the **sort** in your code? \n> \n> And how you fixed it?\n\nI don't think it is the sort, but rather the set creation. Sets do not preserve or guarantee order.", "votes": null }, { "id": "2121790", "postDate": "01/30/2023 14:52:04", "content": "My bad, I corrected it, yes it's the set not the sort.
\nThanks @group16
", "rawMarkdown": "My bad, I corrected it, yes it's the **set** not the sort.\n\nThanks @group16", "votes": null }, { "id": "2121791", "postDate": "01/30/2023 14:53:45", "content": "@joaopmpeinado I just removed the set to fix it, because the order of the candidate was important, this way the ranker can learn better from the heuristic approach
Hi! Thank you for sharing your experience and the mistake that cost you a month in the competition. It's great to see you reflecting on what you learned and sharing with the community. The mistake with candidate generation is a common issue and it's important to remember that the order of the candidates matters. Your hard work paid off with the improved ranker score, and the lesson learned will be valuable in future competitions. Keep up the great work!
", "rawMarkdown": "Hi! Thank you for sharing your experience and the mistake that cost you a month in the competition. It's great to see you reflecting on what you learned and sharing with the community. The mistake with candidate generation is a common issue and it's important to remember that the order of the candidates matters. Your hard work paid off with the improved ranker score, and the lesson learned will be valuable in future competitions. Keep up the great work!", "votes": null }, { "id": "2122367", "postDate": "01/30/2023 22:04:13", "content": "Yeah, from Deotte's code. He uses
\nlist(dict.fromkeys(aids[::-1]))
\nto keep the order while removing duplicates. The [::-1] just puts the most recent aid upfront, and you have to make sure the following logic follows the pattern.
Indeed. It is so difficult to make sure every step is correct. This kind of implict bug is very difficult to detect, especially when the whole pipeline is long. It does not break the program, it runs smoothly, but it will destroy your model performance.
", "rawMarkdown": "Indeed. It is so difficult to make sure every step is correct. This kind of implict bug is very difficult to detect, especially when the whole pipeline is long. It does not break the program, it runs smoothly, but it will destroy your model performance.", "votes": null }, { "id": "2122634", "postDate": "01/31/2023 04:17:04", "content": "That's a good lesson. You have to know what each line does in your code.
", "rawMarkdown": "That's a good lesson. You have to know what each line does in your code.", "votes": null }, { "id": "2122709", "postDate": "01/31/2023 05:25:06", "content": "Thank you for sharing your experience. I'm new in this space and I'm looking at building my own skills too. Looking forward to learning from you.
", "rawMarkdown": "Thank you for sharing your experience. I'm new in this space and I'm looking at building my own skills too. Looking forward to learning from you.", "votes": null }, { "id": "2123077", "postDate": "01/31/2023 10:17:16", "content": "\n\nAnd you ! how do you make sure every step is correct?
\n
1) Try making some assertions:
\nFor example, in your case:
\n# Make sure unique_aids are in front of top_aids2\nassert result[0] == unique_aids[0]\nassert result[len(unique_aids)-1] == unique_aids[-1]\n\n# Make sure top_aids2 order is ok (not a full check)\nresult[len(unique_aids)] == top_aids2[:candidates_number - len(unique_aids)][0]\nresult[-1] == top_aids2[:candidates_number - len(unique_aids)][-1]\n2) If time is allowed, write solid tests for critical functions…
\nAlso, after you've debugged enough code, you should be able to tell the part of the code that is prompted to error. When I saw this kind of code from a public notebook, I feel so insecure to use it directly, so I always rewrite it.
", "rawMarkdown": "> And you ! how do you make sure every step is correct?\n\n1) Try making some assertions:\n\nFor example, in your case:\n```\n# Make sure unique_aids are in front of top_aids2\nassert result[0] == unique_aids[0]\nassert result[len(unique_aids)-1] == unique_aids[-1]\n\n# Make sure top_aids2 order is ok (not a full check)\nresult[len(unique_aids)] == top_aids2[:candidates_number - len(unique_aids)][0]\nresult[-1] == top_aids2[:candidates_number - len(unique_aids)][-1]\n```\n\n2) If time is allowed, write solid tests for critical functions...\n\n\nAlso, after you've debugged enough code, you should be able to tell the part of the code that is prompted to error. When I saw this kind of code from a public notebook, I feel so insecure to use it directly, so I always rewrite it.", "votes": null } ], "comments": [ { "id": 2121644, "author_name": "thedevastator", "author_url": "", "post_date": "01/30/2023 13:13:52", "content": "I know this might be hard or might sound a bit insane but this is the truth:
\nLosing a month of work happens to many of us in the field..
\nThe only thing it means is that you are practicing and leveling up your skill. Look on the bright side even if it is hard to see.
And about the sorting mistake itself, it can happen to anyone it is really tricky.
\nTake care!
", "votes": null, "replies": [] }, { "id": 2121699, "author_name": "joaopmpeinado", "author_url": "", "post_date": "01/30/2023 13:53:20", "content": "Where are the sort in your code?
\nAnd how you fixed it?
", "votes": null, "replies": [ { "id": 2121783, "author_name": "group16", "author_url": "", "post_date": "01/30/2023 14:47:57", "content": "\n\nWhere are the sort in your code?
\nAnd how you fixed it?
\n
I don't think it is the sort, but rather the set creation. Sets do not preserve or guarantee order.
", "votes": null, "replies": [ { "id": 2121790, "author_name": "rayanaay", "author_url": "", "post_date": "01/30/2023 14:52:04", "content": "My bad, I corrected it, yes it's the set not the sort.
\nThanks @group16
", "votes": null, "replies": [ { "id": 2121791, "author_name": "rayanaay", "author_url": "", "post_date": "01/30/2023 14:53:45", "content": "@joaopmpeinado I just removed the set to fix it, because the order of the candidate was important, this way the ranker can learn better from the heuristic approach
Hi! Thank you for sharing your experience and the mistake that cost you a month in the competition. It's great to see you reflecting on what you learned and sharing with the community. The mistake with candidate generation is a common issue and it's important to remember that the order of the candidates matters. Your hard work paid off with the improved ranker score, and the lesson learned will be valuable in future competitions. Keep up the great work!
", "votes": null, "replies": [] }, { "id": 2122367, "author_name": "datadote", "author_url": "", "post_date": "01/30/2023 22:04:13", "content": "Yeah, from Deotte's code. He uses
\nlist(dict.fromkeys(aids[::-1]))
\nto keep the order while removing duplicates. The [::-1] just puts the most recent aid upfront, and you have to make sure the following logic follows the pattern.
Indeed. It is so difficult to make sure every step is correct. This kind of implict bug is very difficult to detect, especially when the whole pipeline is long. It does not break the program, it runs smoothly, but it will destroy your model performance.
", "votes": null, "replies": [] }, { "id": 2122634, "author_name": "gunesevitan", "author_url": "", "post_date": "01/31/2023 04:17:04", "content": "That's a good lesson. You have to know what each line does in your code.
", "votes": null, "replies": [] }, { "id": 2122709, "author_name": "ridwanoyekunle", "author_url": "", "post_date": "01/31/2023 05:25:06", "content": "Thank you for sharing your experience. I'm new in this space and I'm looking at building my own skills too. Looking forward to learning from you.
", "votes": null, "replies": [] }, { "id": 2123077, "author_name": "kingychiu", "author_url": "", "post_date": "01/31/2023 10:17:16", "content": "\n\nAnd you ! how do you make sure every step is correct?
\n
1) Try making some assertions:
\nFor example, in your case:
\n# Make sure unique_aids are in front of top_aids2\nassert result[0] == unique_aids[0]\nassert result[len(unique_aids)-1] == unique_aids[-1]\n\n# Make sure top_aids2 order is ok (not a full check)\nresult[len(unique_aids)] == top_aids2[:candidates_number - len(unique_aids)][0]\nresult[-1] == top_aids2[:candidates_number - len(unique_aids)][-1]\n2) If time is allowed, write solid tests for critical functions…
\nAlso, after you've debugged enough code, you should be able to tell the part of the code that is prompted to error. When I saw this kind of code from a public notebook, I feel so insecure to use it directly, so I always rewrite it.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2121565": "Hello community, I want to share with you my experience with this competition, and the big/dumb mistake which that made me lose 1 month in this competition.\n\nI think that many of us struggled with the ranker, and that the task was not so easy. Personnally, I have realized several of my errors during the last month, but this one is definitely the worst.\n\nThe mistake is about the candidate generation:\n\n```python\n # USE \"CART ORDER\" CO-VISITATION MATRIX\n aids2 = list(itertools.chain(*[top_20_buys[aid] for aid in unique_aids if aid in top_20_buys]))\n # USE \"BUY2BUY\" CO-VISITATION MATRIX\n aids3 = list(itertools.chain(*[top_20_buy2buy[aid] for aid in unique_buys if aid in top_20_buy2buy]))\n # RERANK CANDIDATES\n top_aids2 = [aid2 for aid2, cnt in Counter(aids2+aids3).most_common(candidates_number) if aid2 not in unique_aids] \n result = list(set(unique_aids + top_aids2[:candidates_number - len(unique_aids)]))\n top_orderss = [aid for aid in top_orders if aid not in result ]\n return session, result + top_orderss[:candidates_number - len(result)]\n```\nDid you see it ?\nIn order to avoid any kind of duplicates at the level of my candidates ( for caution ), I used `set`, but I totally forgot that the actual order of the candidates has a capital importance, an order that has been altered because `set` actually rearranged my candidates in ascending order.\n\n\nThat's all for me, I noticed this error a little late I admit, I tried to investigate elsewhere but it never really solved my barrier to crossing the benchmark made by the heuristic approach.\n\nBy removing the famous `set`, the ranker goes from 0.565 to 0.579 in the LB with a single model. However, with 5 submissions left, there is little chance that I will improve further, with all the tests and the time it takes to execute.\n\nFinally, I wanted to say thanks to all the kagglers who gave me directions in this competition, I learned a lot from you!\n\n*And you ! how do you make sure every step is correct ?*\n\n**PS: be careful with every line of code you write, this advice is for me first haha**", "2121644": "I know this might be hard or might sound a bit insane but this is the truth:\nLosing a month of work happens to many of us in the field.. \nThe only thing it means is that you are practicing and leveling up your skill. Look on the bright side even if it is hard to see. \n\nAnd about the sorting mistake itself, it can happen to anyone it is really tricky.\n\nTake care!", "2121699": "Where are the **sort** in your code? \n\nAnd how you fixed it?", "2121783": "> Where are the **sort** in your code? \n> \n> And how you fixed it?\n\nI don't think it is the sort, but rather the set creation. Sets do not preserve or guarantee order.", "2121790": "My bad, I corrected it, yes it's the **set** not the sort.\n\nThanks @group16", "2121791": "joaopmpeinado I just removed the `set` to fix it, because the order of the candidate was important, this way the ranker can learn better from the heuristic approach", "2122101": "Hi! Thank you for sharing your experience and the mistake that cost you a month in the competition. It's great to see you reflecting on what you learned and sharing with the community. The mistake with candidate generation is a common issue and it's important to remember that the order of the candidates matters. Your hard work paid off with the improved ranker score, and the lesson learned will be valuable in future competitions. Keep up the great work!", "2122367": "Yeah, from Deotte's code. He uses \nlist(dict.fromkeys(aids[::-1]))\nto keep the order while removing duplicates. The [::-1] just puts the most recent aid upfront, and you have to make sure the following logic follows the pattern.", "2122536": "Indeed. It is so difficult to make sure every step is correct. This kind of implict bug is very difficult to detect, especially when the whole pipeline is long. It does not break the program, it runs smoothly, but it will destroy your model performance.", "2122634": "That's a good lesson. You have to know what each line does in your code.", "2122709": "Thank you for sharing your experience. I'm new in this space and I'm looking at building my own skills too. Looking forward to learning from you.", "2123077": "> And you ! how do you make sure every step is correct?\n\n1) Try making some assertions:\n\nFor example, in your case:\n```\n# Make sure unique_aids are in front of top_aids2\nassert result[0] == unique_aids[0]\nassert result[len(unique_aids)-1] == unique_aids[-1]\n\n# Make sure top_aids2 order is ok (not a full check)\nresult[len(unique_aids)] == top_aids2[:candidates_number - len(unique_aids)][0]\nresult[-1] == top_aids2[:candidates_number - len(unique_aids)][-1]\n```\n\n2) If time is allowed, write solid tests for critical functions...\n\n\nAlso, after you've debugged enough code, you should be able to tell the part of the code that is prompted to error. When I saw this kind of code from a public notebook, I feel so insecure to use it directly, so I always rewrite it." }, "source": "meta" }