Hello community, I'm asking this question because obviously the majority of the interactions features are null values because of the scarcity of the right candidates.
So, how much did you get with only users and items features ?
\nI would love to hear about your results !
", "messages": [ { "id": "2108837", "postDate": "01/20/2023 21:19:40", "content": "Hello community, I'm asking this question because obviously the majority of the interactions features are null values because of the scarcity of the right candidates.
So, how much did you get with only users and items features ?
\nI would love to hear about your results !
", "rawMarkdown": "Hello community, I'm asking this question because obviously the majority of the interactions features are `null` values because of the scarcity of the right candidates.\n\nSo, how much did you get with only users and items features ?\n\nI would love to hear about your results !", "votes": null }, { "id": "2109150", "postDate": "01/21/2023 07:09:00", "content": "My candidate generation val score is 0.5692 and lb score is 0.58 without ranker model. I'm currently using 25 aid and session features and I haven't started creating interaction features yet. This model's oof score is 0.575. It would make sense to share your own score when you are starting a topic like this.
", "rawMarkdown": "My candidate generation val score is 0.5692 and lb score is 0.58 without ranker model. I'm currently using 25 aid and session features and I haven't started creating interaction features yet. This model's oof score is 0.575. It would make sense to share your own score when you are starting a topic like this.", "votes": null }, { "id": "2109316", "postDate": "01/21/2023 10:01:30", "content": "Great, I'm using user / item / interaction features, and I'm still get tinga score under the benchmark of top20 co-visitation matrix. I don't really understand why it overfit even with strong regularization strategies.
\nLet's talk about orders recall only:
\nWithout interactions features i'm getting a very bad recall of 0.42
\nWith interactions features I'm getting a score near the benchmark ( 0.639 vs ~0.649 ) --> but it's still not enough to improve the LB
", "rawMarkdown": "Great, I'm using user / item / interaction features, and I'm still get tinga score under the benchmark of top20 co-visitation matrix. I don't really understand why it overfit even with strong regularization strategies.\n\nLet's talk about orders recall only:\n\nWithout interactions features i'm getting a very bad recall of 0.42\n\nWith interactions features I'm getting a score near the benchmark ( 0.639 vs ~0.649 ) --> but it's still not enough to improve the LB", "votes": null }, { "id": "2109557", "postDate": "01/21/2023 13:27:50", "content": "0.42 order score is pretty bad. You can try feeding candidate scores into your model. It would help your model to converge faster.
", "rawMarkdown": "0.42 order score is pretty bad. You can try feeding candidate scores into your model. It would help your model to converge faster.", "votes": null }, { "id": "2109799", "postDate": "01/21/2023 17:10:55", "content": "Yes, i did. Did you consider it as an integer or float ?
\nI'm personnally re-ranking candidates from top 100 extracted from the co-visitation matrix, and convert it to float32 in order to consider it as continuous variable by the model. What do you think about it ?
Being it float or integer wouldn't make any difference. My guess is you might be using only covisitation candidates. You also have to generate unique aids in sessions as candidates. That's how you reach 0.486 in the first place.
", "rawMarkdown": "Being it float or integer wouldn't make any difference. My guess is you might be using only covisitation candidates. You also have to generate unique aids in sessions as candidates. That's how you reach 0.486 in the first place.", "votes": null }, { "id": "2109945", "postDate": "01/21/2023 19:22:25", "content": "@gunesevitan thanks for sharing the info. Would you mind elaborating what do you mean by candidate score?
", "rawMarkdown": "gunesevitan thanks for sharing the info. Would you mind elaborating what do you mean by *candidate score*?", "votes": null }, { "id": "2110297", "postDate": "01/22/2023 05:28:42", "content": "If you are using covisitation for candidate generation then occurrence counts of aids can be used as candidate scores. If you are using some kind of similarity search then the distance metric can be used as candidate score and etc.
", "rawMarkdown": "If you are using covisitation for candidate generation then occurrence counts of aids can be used as candidate scores. If you are using some kind of similarity search then the distance metric can be used as candidate score and etc.", "votes": null }, { "id": "2110483", "postDate": "01/22/2023 08:22:05", "content": "Have you experienced that after adding some interaction features, it will get a good local cv, but LB will come to 0.1. I don't know how this happened. My validation is spilited from the training data.
", "rawMarkdown": "Have you experienced that after adding some interaction features, it will get a good local cv, but LB will come to 0.1. I don't know how this happened. My validation is spilited from the training data.", "votes": null }, { "id": "2110924", "postDate": "01/22/2023 14:34:08", "content": "How did you split your train test data ? If you're getting a good local cv, may be you're leaking data in your train data
", "rawMarkdown": "How did you split your train test data ? If you're getting a good local cv, may be you're leaking data in your train data", "votes": null }, { "id": "2111010", "postDate": "01/22/2023 15:13:53", "content": "I spilt 4/5 of the last week data for training and remain 1/5 for local cv. It can perform well on local cv but poor on LB.
", "rawMarkdown": "I spilt 4/5 of the last week data for training and remain 1/5 for local cv. It can perform well on local cv but poor on LB.", "votes": null }, { "id": "2111053", "postDate": "01/22/2023 15:32:19", "content": "What about your interactions features ? you should not include the last week to build your interactions features, did you verify this ?? I told you this because I did this error too
", "rawMarkdown": "What about your interactions features ? you should not include the last week to build your interactions features, did you verify this ?? I told you this because I did this error too", "votes": null }, { "id": "2111067", "postDate": "01/22/2023 15:40:09", "content": "I use the data provided by Radek. The first half is the training data(test.parquet) and the second half is ground truth(test_labels.parquet). And I split 1/5 from the first half(test.parquet) to be the valid set. I also generate some interactions features from the first half., just some features groupby ['session', 'aid'], like ['session', 'aid'] click_count, cart_count, order_count, session length and so on. Are there any errors? Please point out to me. Thanks a lot!
", "rawMarkdown": "I use the data provided by Radek. The first half is the training data(test.parquet) and the second half is ground truth(test_labels.parquet). And I split 1/5 from the first half(test.parquet) to be the valid set. I also generate some interactions features from the first half., just some features groupby ['session', 'aid'], like ['session', 'aid'] click_count, cart_count, order_count, session length and so on. Are there any errors? Please point out to me. Thanks a lot!", "votes": null }, { "id": "2115546", "postDate": "01/25/2023 20:42:44", "content": "Hello @kimoyami sorry for the late reply.
\nFrom what you said, everything seems good without any leaks. You should may be review your validation strategy. Also, when generating candidates --> be careful with the leakage.
\nUse one co-visitation matrix for train, and one another for train
", "rawMarkdown": "Hello @kimoyami sorry for the late reply.\n\nFrom what you said, everything seems good without any leaks. You should may be review your validation strategy. Also, when generating candidates --> be careful with the leakage.\n\nUse one co-visitation matrix for train, and one another for train", "votes": null } ], "comments": [ { "id": 2109150, "author_name": "gunesevitan", "author_url": "", "post_date": "01/21/2023 07:09:00", "content": "My candidate generation val score is 0.5692 and lb score is 0.58 without ranker model. I'm currently using 25 aid and session features and I haven't started creating interaction features yet. This model's oof score is 0.575. It would make sense to share your own score when you are starting a topic like this.
", "votes": null, "replies": [ { "id": 2109316, "author_name": "rayanaay", "author_url": "", "post_date": "01/21/2023 10:01:30", "content": "Great, I'm using user / item / interaction features, and I'm still get tinga score under the benchmark of top20 co-visitation matrix. I don't really understand why it overfit even with strong regularization strategies.
\nLet's talk about orders recall only:
\nWithout interactions features i'm getting a very bad recall of 0.42
\nWith interactions features I'm getting a score near the benchmark ( 0.639 vs ~0.649 ) --> but it's still not enough to improve the LB
", "votes": null, "replies": [ { "id": 2109557, "author_name": "gunesevitan", "author_url": "", "post_date": "01/21/2023 13:27:50", "content": "0.42 order score is pretty bad. You can try feeding candidate scores into your model. It would help your model to converge faster.
", "votes": null, "replies": [ { "id": 2109799, "author_name": "rayanaay", "author_url": "", "post_date": "01/21/2023 17:10:55", "content": "Yes, i did. Did you consider it as an integer or float ?
\nI'm personnally re-ranking candidates from top 100 extracted from the co-visitation matrix, and convert it to float32 in order to consider it as continuous variable by the model. What do you think about it ?
Being it float or integer wouldn't make any difference. My guess is you might be using only covisitation candidates. You also have to generate unique aids in sessions as candidates. That's how you reach 0.486 in the first place.
", "votes": null, "replies": [ { "id": 2109945, "author_name": "parthpankajtiwary", "author_url": "", "post_date": "01/21/2023 19:22:25", "content": "@gunesevitan thanks for sharing the info. Would you mind elaborating what do you mean by candidate score?
", "votes": null, "replies": [ { "id": 2110297, "author_name": "gunesevitan", "author_url": "", "post_date": "01/22/2023 05:28:42", "content": "If you are using covisitation for candidate generation then occurrence counts of aids can be used as candidate scores. If you are using some kind of similarity search then the distance metric can be used as candidate score and etc.
", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2110483, "author_name": "kimoyami", "author_url": "", "post_date": "01/22/2023 08:22:05", "content": "Have you experienced that after adding some interaction features, it will get a good local cv, but LB will come to 0.1. I don't know how this happened. My validation is spilited from the training data.
", "votes": null, "replies": [ { "id": 2110924, "author_name": "rayanaay", "author_url": "", "post_date": "01/22/2023 14:34:08", "content": "How did you split your train test data ? If you're getting a good local cv, may be you're leaking data in your train data
", "votes": null, "replies": [ { "id": 2111010, "author_name": "kimoyami", "author_url": "", "post_date": "01/22/2023 15:13:53", "content": "I spilt 4/5 of the last week data for training and remain 1/5 for local cv. It can perform well on local cv but poor on LB.
", "votes": null, "replies": [ { "id": 2111053, "author_name": "rayanaay", "author_url": "", "post_date": "01/22/2023 15:32:19", "content": "What about your interactions features ? you should not include the last week to build your interactions features, did you verify this ?? I told you this because I did this error too
", "votes": null, "replies": [ { "id": 2111067, "author_name": "kimoyami", "author_url": "", "post_date": "01/22/2023 15:40:09", "content": "I use the data provided by Radek. The first half is the training data(test.parquet) and the second half is ground truth(test_labels.parquet). And I split 1/5 from the first half(test.parquet) to be the valid set. I also generate some interactions features from the first half., just some features groupby ['session', 'aid'], like ['session', 'aid'] click_count, cart_count, order_count, session length and so on. Are there any errors? Please point out to me. Thanks a lot!
", "votes": null, "replies": [ { "id": 2115546, "author_name": "rayanaay", "author_url": "", "post_date": "01/25/2023 20:42:44", "content": "Hello @kimoyami sorry for the late reply.
\nFrom what you said, everything seems good without any leaks. You should may be review your validation strategy. Also, when generating candidates --> be careful with the leakage.
\nUse one co-visitation matrix for train, and one another for train
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2108837": "Hello community, I'm asking this question because obviously the majority of the interactions features are `null` values because of the scarcity of the right candidates.\n\nSo, how much did you get with only users and items features ?\n\nI would love to hear about your results !", "2109150": "My candidate generation val score is 0.5692 and lb score is 0.58 without ranker model. I'm currently using 25 aid and session features and I haven't started creating interaction features yet. This model's oof score is 0.575. It would make sense to share your own score when you are starting a topic like this.", "2109316": "Great, I'm using user / item / interaction features, and I'm still get tinga score under the benchmark of top20 co-visitation matrix. I don't really understand why it overfit even with strong regularization strategies.\n\nLet's talk about orders recall only:\n\nWithout interactions features i'm getting a very bad recall of 0.42\n\nWith interactions features I'm getting a score near the benchmark ( 0.639 vs ~0.649 ) --> but it's still not enough to improve the LB", "2109557": "0.42 order score is pretty bad. You can try feeding candidate scores into your model. It would help your model to converge faster.", "2109799": "Yes, i did. Did you consider it as an integer or float ? \nI'm personnally re-ranking candidates from top 100 extracted from the co-visitation matrix, and convert it to float32 in order to consider it as continuous variable by the model. What do you think about it ?", "2109858": "Being it float or integer wouldn't make any difference. My guess is you might be using only covisitation candidates. You also have to generate unique aids in sessions as candidates. That's how you reach 0.486 in the first place.", "2109945": "gunesevitan thanks for sharing the info. Would you mind elaborating what do you mean by *candidate score*?", "2110297": "If you are using covisitation for candidate generation then occurrence counts of aids can be used as candidate scores. If you are using some kind of similarity search then the distance metric can be used as candidate score and etc.", "2110483": "Have you experienced that after adding some interaction features, it will get a good local cv, but LB will come to 0.1. I don't know how this happened. My validation is spilited from the training data.", "2110924": "How did you split your train test data ? If you're getting a good local cv, may be you're leaking data in your train data", "2111010": "I spilt 4/5 of the last week data for training and remain 1/5 for local cv. It can perform well on local cv but poor on LB.", "2111053": "What about your interactions features ? you should not include the last week to build your interactions features, did you verify this ?? I told you this because I did this error too", "2111067": "I use the data provided by Radek. The first half is the training data(test.parquet) and the second half is ground truth(test_labels.parquet). And I split 1/5 from the first half(test.parquet) to be the valid set. I also generate some interactions features from the first half., just some features groupby ['session', 'aid'], like ['session', 'aid'] click_count, cart_count, order_count, session length and so on. Are there any errors? Please point out to me. Thanks a lot!", "2115546": "Hello @kimoyami sorry for the late reply.\n\nFrom what you said, everything seems good without any leaks. You should may be review your validation strategy. Also, when generating candidates --> be careful with the leakage.\n\nUse one co-visitation matrix for train, and one another for train" }, "source": "meta" }