![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4624515%2Fed57783c4103c84f9c54148604c2e970%2F2023-02-02%202.15.20.png?generation=1675271741794866&alt=media\")
Our team's 34th solution and overview are summarized at the following link:
\n\nIn this thread, I'll explain about my stacking method which boosted both our local CV and LB score.
\nI'm not sure this method is commonly used or well known so I named this method \"Cross Target Stacking\"
\nbecause this is like a stacking using the different type of the target labels.
The following figure shows that the ordinary training/prediction strategies.
\nI think a lot of participants apply these kind of strategies.
\nI also try this strategies at first.
At the training phase, the GBDT framework (like the LightGBM) is used for training and three independent models are generated for order/click/cart, respectively.
\nThe common candidates/features are used for the training but the target labels are separated for each type (order/cart/click).
\nAt the prediction phase, each models are used for the inference and we can generate the three independent score.
\nFinally, we can get the final submission results by concatenating there score.
During the competition, I was wondering how to treat three different target labels.
\nIn the ordinary method, three different type of the labels (order/click/cart) are independent.
\nIn other words, we can generate \"order\" prediction score without click/cart target labels and we only need the
\n\"order\" target label and features when we train/predict \"order\" labels.
However, I guessed click/carts target information might be made use of the order prediction.
\nFor the same reason, click/order might be made use of the \"cart\" prediction and cart/order might be made use of the \"click\" prediction as well.
In order to check this hypothesis, I tried to add the cart/click target labels for \"order\" training on purpose.
\nObviously this features cause leakage but I wanted to know cart/click target information are important to predict \"order\" or not.
\nBy adding there \"leakage\" features, my local CV score was drastically improved.
\nThis means that cart/click target label information is important to predict \"order\".
Of course, we can't use target label itself as a feature, so I tried to use the prediction score of cross target.
\nIn my definition, \"cross target\" represent the different type of target.
\nFor example, the cross target of \"order\" means \"cart/clicks\".
\nThis method is similar to the \"stacking\" method.
\nWhereas the ordinary stacking method uses the \"same\" type of prediction score, my stacking method uses the
\ndifferent type of target.
\nThat's why I named this method \"cross target stacking\".
The following figure shows the overview of this method for training.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4624515%2Fdf4f95f5eae6d29d98cab90092ff7c39%2F2023-02-02%202.14.31.png?generation=1675271696594580&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4624515%2F275b803de2abd2a0d9d7cf372aa223a8%2F2023-02-02%202.17.03.png?generation=1675271844836461&alt=media\")
There are two training/prediction phase.
\nThe first phase is similar to the ordinary training/prediction strategies.
After the first training, we can get the OOF score for each type by using the cross validation.
\n(In my case number of fold=5)
These prediction score can be added for the second training as features.
\nIn order to avoid overfitting, I only add cross target prediction score.
For example, I added cart/click prediction scores to the \"order\" training but didn't add the \"order\"
\nprediction score itself.
By using this procedures, our local CV score and LB are improved especially for the \"order\" training.
\nIn fact, these strategies could boosted our score but I'm not sure this method is popular or not.
\nI'm also not sure this kind of method is adopted or tried by the Top-Kaggler in this competition.
I would be happy if I could discuss this theme in this thread.
\nAt the end, I appreciate the collaboration with my team members and this competition supported by the organizers.
\nComments are welcome:)
Our team's 34th solution and overview are summarized at the following link:
\n\nIn this thread, I'll explain about my stacking method which boosted both our local CV and LB score.
\nI'm not sure this method is commonly used or well known so I named this method \"Cross Target Stacking\"
\nbecause this is like a stacking using the different type of the target labels.
The following figure shows that the ordinary training/prediction strategies.
\nI think a lot of participants apply these kind of strategies.
\nI also try this strategies at first.
At the training phase, the GBDT framework (like the LightGBM) is used for training and three independent models are generated for order/click/cart, respectively.
\nThe common candidates/features are used for the training but the target labels are separated for each type (order/cart/click).
\nAt the prediction phase, each models are used for the inference and we can generate the three independent score.
\nFinally, we can get the final submission results by concatenating there score.
During the competition, I was wondering how to treat three different target labels.
\nIn the ordinary method, three different type of the labels (order/click/cart) are independent.
\nIn other words, we can generate \"order\" prediction score without click/cart target labels and we only need the
\n\"order\" target label and features when we train/predict \"order\" labels.
However, I guessed click/carts target information might be made use of the order prediction.
\nFor the same reason, click/order might be made use of the \"cart\" prediction and cart/order might be made use of the \"click\" prediction as well.
In order to check this hypothesis, I tried to add the cart/click target labels for \"order\" training on purpose.
\nObviously this features cause leakage but I wanted to know cart/click target information are important to predict \"order\" or not.
\nBy adding there \"leakage\" features, my local CV score was drastically improved.
\nThis means that cart/click target label information is important to predict \"order\".
Of course, we can't use target label itself as a feature, so I tried to use the prediction score of cross target.
\nIn my definition, \"cross target\" represent the different type of target.
\nFor example, the cross target of \"order\" means \"cart/clicks\".
\nThis method is similar to the \"stacking\" method.
\nWhereas the ordinary stacking method uses the \"same\" type of prediction score, my stacking method uses the
\ndifferent type of target.
\nThat's why I named this method \"cross target stacking\".
The following figure shows the overview of this method for training.
\n
There are two training/prediction phase.
\nThe first phase is similar to the ordinary training/prediction strategies.
After the first training, we can get the OOF score for each type by using the cross validation.
\n(In my case number of fold=5)
These prediction score can be added for the second training as features.
\nIn order to avoid overfitting, I only add cross target prediction score.
For example, I added cart/click prediction scores to the \"order\" training but didn't add the \"order\"
\nprediction score itself.
By using this procedures, our local CV score and LB are improved especially for the \"order\" training.
\nIn fact, these strategies could boosted our score but I'm not sure this method is popular or not.
\nI'm also not sure this kind of method is adopted or tried by the Top-Kaggler in this competition.
I would be happy if I could discuss this theme in this thread.
\nAt the end, I appreciate the collaboration with my team members and this competition supported by the organizers.
\nComments are welcome:)
I used the same approach for \"orders\" model and it gave me ~0.0015.
", "rawMarkdown": "I used the same approach for \"orders\" model and it gave me ~0.0015.", "votes": null }, { "id": "2125641", "postDate": "02/01/2023 19:02:03", "content": "Hello @tetsuro731 , I'm happy you shared a detailed post of this approach.
\nI think I saw something similar in the RecSys Challenge 2021 (video presentation of recsys 2021 top teams, the link should bring to the time 1:26:45 where a similar approach is shown, it is not in 2 phases like yours).
This approach can bring many advantages. That year after the end of RecSys Challenge 2021 I tried training a classifier that predicted the absence of any kind of interaction and using it as an additional feature and saw a 0.5% improvement on the validation score, I would be really interested in knowing if this approach could also improve your solution in this challenge.
\nMy intuitive understanding of the usefulness of this approach is that the prediction of a model trained on another target type contains in itself manipulations of the features useful for that specific target, those manipulations may also be useful for other targets and so adding the score of the model as a feature can make easier to find those aspects that may be harder to find only seeing the current target type.
\nThe use of prediction from a model that predicts absence of interactions helps to more easily ignore candidates that are not interesting for any interaction type making their score lower and so less likely to end up in the top 20.
\nLastly, Congratulations for your position in the challenge!
", "rawMarkdown": "Hello @tetsuro731 , I'm happy you shared a detailed post of this approach. \nI think I saw something similar in the RecSys Challenge 2021 ([video presentation of recsys 2021 top teams](https://youtu.be/8ms6ZBXDx3E?t=5205), the link should bring to the time 1:26:45 where a similar approach is shown, it is not in 2 phases like yours).\n\nThis approach can bring many advantages. That year after the end of RecSys Challenge 2021 I tried training a classifier that predicted the absence of any kind of interaction and using it as an additional feature and saw a 0.5% improvement on the validation score, I would be really interested in knowing if this approach could also improve your solution in this challenge.\n\nMy intuitive understanding of the usefulness of this approach is that the prediction of a model trained on another target type contains in itself manipulations of the features useful for that specific target, those manipulations may also be useful for other targets and so adding the score of the model as a feature can make easier to find those aspects that may be harder to find only seeing the current target type.\n\nThe use of prediction from a model that predicts absence of interactions helps to more easily ignore candidates that are not interesting for any interaction type making their score lower and so less likely to end up in the top 20.\n\nLastly, Congratulations for your position in the challenge!", "votes": null }, { "id": "2125902", "postDate": "02/02/2023 01:33:11", "content": "Thanks for sharing. I used the same idea here. In local cv, it boosted by almost 0.001. But didn't work on lb in my case.. confused..
", "rawMarkdown": "Thanks for sharing. I used the same idea here. In local cv, it boosted by almost 0.001. But didn't work on lb in my case.. confused..", "votes": null }, { "id": "2127672", "postDate": "02/03/2023 03:28:21", "content": "Thank you for your comment.
\nThat's consistent with my situation.
\nI applied to this method to \"order\" which could boost my lb from 0.593 to 0.594.
Thank you for your comment.
\nIn my case, these methods can boost our local CV.
\nHowever, as for LB, \"order\" stacking was the most effective to boost our LB but the other type stacking couldn't boost our LB effectively.
\nI think the effect of this method is depend on the \"type\".
Thank you for your informative comment/suggestion!
\nI think that's so interesting research.
\nIn this competition, these method can boost our LB ~0.001 which is not a large effect.
\nHowever, I think this kind of method is interesting and can be make use of the other situation as you said.
I also congratulate you for your silver medal!
", "rawMarkdown": "Thank you for your informative comment/suggestion!\nI think that's so interesting research.\nIn this competition, these method can boost our LB ~0.001 which is not a large effect.\nHowever, I think this kind of method is interesting and can be make use of the other situation as you said.\n\nI also congratulate you for your silver medal!", "votes": null }, { "id": "2127871", "postDate": "02/03/2023 09:33:23", "content": "Stacking features generated in this way helped a bit also my model (not more 0.001 aniway).
\nIt is a sort of surrogate for a multioutput rank/classifier.
\nIf NN had been competitive to rank candidate with this dataset, we would certainly have used a schema like this to train:
\nX:session, candidate \ny:(candidate_is_a_session_order,candidate_is_a_session_cart,candidate_is_session_a_click)\nI used the same approach for \"orders\" model and it gave me ~0.0015.
", "votes": null, "replies": [ { "id": 2127672, "author_name": "tetsuro731", "author_url": "", "post_date": "02/03/2023 03:28:21", "content": "Thank you for your comment.
\nThat's consistent with my situation.
\nI applied to this method to \"order\" which could boost my lb from 0.593 to 0.594.
Hello @tetsuro731 , I'm happy you shared a detailed post of this approach.
\nI think I saw something similar in the RecSys Challenge 2021 (video presentation of recsys 2021 top teams, the link should bring to the time 1:26:45 where a similar approach is shown, it is not in 2 phases like yours).
This approach can bring many advantages. That year after the end of RecSys Challenge 2021 I tried training a classifier that predicted the absence of any kind of interaction and using it as an additional feature and saw a 0.5% improvement on the validation score, I would be really interested in knowing if this approach could also improve your solution in this challenge.
\nMy intuitive understanding of the usefulness of this approach is that the prediction of a model trained on another target type contains in itself manipulations of the features useful for that specific target, those manipulations may also be useful for other targets and so adding the score of the model as a feature can make easier to find those aspects that may be harder to find only seeing the current target type.
\nThe use of prediction from a model that predicts absence of interactions helps to more easily ignore candidates that are not interesting for any interaction type making their score lower and so less likely to end up in the top 20.
\nLastly, Congratulations for your position in the challenge!
", "votes": null, "replies": [ { "id": 2127676, "author_name": "tetsuro731", "author_url": "", "post_date": "02/03/2023 03:37:30", "content": "Thank you for your informative comment/suggestion!
\nI think that's so interesting research.
\nIn this competition, these method can boost our LB ~0.001 which is not a large effect.
\nHowever, I think this kind of method is interesting and can be make use of the other situation as you said.
I also congratulate you for your silver medal!
", "votes": null, "replies": [] } ] }, { "id": 2125902, "author_name": "hookman", "author_url": "", "post_date": "02/02/2023 01:33:11", "content": "Thanks for sharing. I used the same idea here. In local cv, it boosted by almost 0.001. But didn't work on lb in my case.. confused..
", "votes": null, "replies": [ { "id": 2127675, "author_name": "tetsuro731", "author_url": "", "post_date": "02/03/2023 03:31:51", "content": "Thank you for your comment.
\nIn my case, these methods can boost our local CV.
\nHowever, as for LB, \"order\" stacking was the most effective to boost our LB but the other type stacking couldn't boost our LB effectively.
\nI think the effect of this method is depend on the \"type\".
Stacking features generated in this way helped a bit also my model (not more 0.001 aniway).
\nIt is a sort of surrogate for a multioutput rank/classifier.
\nIf NN had been competitive to rank candidate with this dataset, we would certainly have used a schema like this to train:
\nX:session, candidate \ny:(candidate_is_a_session_order,candidate_is_a_session_cart,candidate_is_session_a_click)\n