Thank you Amex for hosting the amazing competition. Thank you to the whole Kaggle community and Kagglers who have attributed to this competition and provided the amazing opportunity to compete and learn!!
\nI, fortunately, got the 30th place with 720 places shake-up. I think it is an interesting case since no one in the upper rankers has got the shake-up as much as I did. I'd like to share the experience of that for the record, and please comment if you guys have any ideas why this huge shake-up happened. Also, I'm looking for a team to join the Kaggle competitions. If there's anyone who considers it, feel free to contact me.
\nI couldn't believe it when I woke up and saw on the private leaderboard that my submission was in 30th place. Last night when I checked it, the public leader board was 750th place😨. For your information, I am a university student in japan and there's 20 hours difference😃. I thought that the ensemble of NN + LGBM was the most likely solution, therefore I didn't really care about the public leader board though it struck me with a major surprise. It proves that we don't know what's gonna happen in the private leader board, and most of the participants were overfitting for the public leader board.
\nMy solution was similar to his. The main idea was the 50%/50% ensemble of LGBM and NN Transformer.
\nMr. Chris Deotte's solution
\nIn my case, I'm a broke university student, and I couldn't afford the computational resource. I used the free quote of Kaggle GPU and the resource was very limited. I attached Efficientnet with a transformer, and I could only use 60% of the competition data for the training session. That's one thing I regret. For LGBM models, I simply used the public LGBM notebook for the LGBM ensemble. I think the reason why I got this place was the NN model part. Although I had poor resources and couldn't do my best, I think that the transformer+efficientnet structure was the reason for my rank. That shows the infinite possibility of the NN!!
I myself cannot believe that I got the top 0.6% place in this competition with this poor resource. Even though I could only use 60% of training data for NN, and public notebooks for the LGBM model, the private score was good. I think it means that this solution was one of the best answers for this competition. I'd love to see what happens if I could have used the full 100% of the training data for the NN models and properly train the LGBM models. Could I win the gold medal, maybe?
\nThanks again to everyone who contributed to this competition. And again, I am desperately looking for a team. I'd love to learn from the team and I believe I can be helpful to the whole team. Please contact me if you can let me join your team😊
", "messages": [ { "id": "1913250", "postDate": "08/25/2022 08:57:00", "content": "Thank you Amex for hosting the amazing competition. Thank you to the whole Kaggle community and Kagglers who have attributed to this competition and provided the amazing opportunity to compete and learn!!
\nI, fortunately, got the 30th place with 720 places shake-up. I think it is an interesting case since no one in the upper rankers has got the shake-up as much as I did. I'd like to share the experience of that for the record, and please comment if you guys have any ideas why this huge shake-up happened. Also, I'm looking for a team to join the Kaggle competitions. If there's anyone who considers it, feel free to contact me.
\nI couldn't believe it when I woke up and saw on the private leaderboard that my submission was in 30th place. Last night when I checked it, the public leader board was 750th place😨. For your information, I am a university student in japan and there's 20 hours difference😃. I thought that the ensemble of NN + LGBM was the most likely solution, therefore I didn't really care about the public leader board though it struck me with a major surprise. It proves that we don't know what's gonna happen in the private leader board, and most of the participants were overfitting for the public leader board.
\nMy solution was similar to his. The main idea was the 50%/50% ensemble of LGBM and NN Transformer.
\nMr. Chris Deotte's solution
\nIn my case, I'm a broke university student, and I couldn't afford the computational resource. I used the free quote of Kaggle GPU and the resource was very limited. I attached Efficientnet with a transformer, and I could only use 60% of the competition data for the training session. That's one thing I regret. For LGBM models, I simply used the public LGBM notebook for the LGBM ensemble. I think the reason why I got this place was the NN model part. Although I had poor resources and couldn't do my best, I think that the transformer+efficientnet structure was the reason for my rank. That shows the infinite possibility of the NN!!
I myself cannot believe that I got the top 0.6% place in this competition with this poor resource. Even though I could only use 60% of training data for NN, and public notebooks for the LGBM model, the private score was good. I think it means that this solution was one of the best answers for this competition. I'd love to see what happens if I could have used the full 100% of the training data for the NN models and properly train the LGBM models. Could I win the gold medal, maybe?
\nThanks again to everyone who contributed to this competition. And again, I am desperately looking for a team. I'd love to learn from the team and I believe I can be helpful to the whole team. Please contact me if you can let me join your team😊
", "rawMarkdown": "Thank you Amex for hosting the amazing competition. Thank you to the whole Kaggle community and Kagglers who have attributed to this competition and provided the amazing opportunity to compete and learn!!\n\n## Abstract\nI, fortunately, got the 30th place with 720 places shake-up. I think it is an interesting case since no one in the upper rankers has got the shake-up as much as I did. I'd like to share the experience of that for the record, and please comment if you guys have any ideas why this huge shake-up happened. Also, I'm looking for a team to join the Kaggle competitions. If there's anyone who considers it, feel free to contact me.\n\n## 720 places Shake-up\nI couldn't believe it when I woke up and saw on the private leaderboard that my submission was in 30th place. Last night when I checked it, the public leader board was 750th place😨. For your information, I am a university student in japan and there's 20 hours difference😃. I thought that the ensemble of NN + LGBM was the most likely solution, therefore I didn't really care about the public leader board though it struck me with a major surprise. It proves that we don't know what's gonna happen in the private leader board, and most of the participants were overfitting for the public leader board.\n\n## Solution overview\nMy solution was similar to his. The main idea was the 50%/50% ensemble of LGBM and NN Transformer.\n[Mr. Chris Deotte's solution](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347641)\nIn my case, I'm a broke university student, and I couldn't afford the computational resource. I used the free quote of Kaggle GPU and the resource was very limited. I attached Efficientnet with a transformer, and I could only use 60% of the competition data for the training session. That's one thing I regret. For LGBM models, I simply used the public LGBM notebook for the LGBM ensemble. I think the reason why I got this place was the NN model part. Although I had poor resources and couldn't do my best, I think that the transformer+efficientnet structure was the reason for my rank. That shows the infinite possibility of the NN!!\n\n## What I learned\nI myself cannot believe that I got the top 0.6% place in this competition with this poor resource. Even though I could only use 60% of training data for NN, and public notebooks for the LGBM model, the private score was good. I think it means that this solution was one of the best answers for this competition. I'd love to see what happens if I could have used the full 100% of the training data for the NN models and properly train the LGBM models. Could I win the gold medal, maybe?\n\n## Closure\nThanks again to everyone who contributed to this competition. And again, I am desperately looking for a team. I'd love to learn from the team and I believe I can be helpful to the whole team. Please contact me if you can let me join your team😊", "votes": null }, { "id": "1913355", "postDate": "08/25/2022 09:38:01", "content": "That's a neat decision from the beginning to do 50/50 of tree and nn.
\nIf you're willing to share your NN training setup, I can try to run it on 100% data and share with you the results.
Thanks for sharing! Congrats!
", "rawMarkdown": "Thanks for sharing! Congrats!", "votes": null }, { "id": "1913759", "postDate": "08/25/2022 13:44:02", "content": "Damnn congrats @tatsumicrub !!
", "rawMarkdown": "Damnn congrats @tatsumicrub !!", "votes": null }, { "id": "1913772", "postDate": "08/25/2022 13:56:08", "content": "Congratulations! Great model and great result. Can you explain more about
\n\n\nI attached Efficientnet with a transformer
\n
Did you convert the competition data into 2D images or something for the EfficientNet CNN to process?
", "rawMarkdown": "Congratulations! Great model and great result. Can you explain more about\n>I attached Efficientnet with a transformer\n\nDid you convert the competition data into 2D images or something for the EfficientNet CNN to process?", "votes": null }, { "id": "1913866", "postDate": "08/25/2022 14:50:46", "content": "Amazing results with limited resources
", "rawMarkdown": "Amazing results with limited resources", "votes": null }, { "id": "1913959", "postDate": "08/25/2022 15:50:19", "content": "@nyleve
\nI'll try to train with the rest o 40% data! Though it is really kind of you!! Thank you so much!!
@cdeotte
\nThank you for your comment! Your notebook helped us a lot! I attached the efficientnet at the output layer rather than the input layer. So the data preparation is exactly the same as your transformer notebook. Since the transformer models can handle continuous data better than efficientnet, first I fed the data to the transformer. Then I flatten the output from the transformer, reshaped the data into the efficienet input shape, and fed it to the efficientnet.
Thank you guys for the comments! Hope we'll do our best in upcoming competitions!
", "rawMarkdown": "Thank you guys for the comments! Hope we'll do our best in upcoming competitions!", "votes": null }, { "id": "1913997", "postDate": "08/25/2022 16:20:19", "content": "That's awesome, great idea. Did you train with any pseudo labeled test data? Or does your solution only use the train data?
", "rawMarkdown": "That's awesome, great idea. Did you train with any pseudo labeled test data? Or does your solution only use the train data?", "votes": null } ], "comments": [ { "id": 1913355, "author_name": "nyleve", "author_url": "", "post_date": "08/25/2022 09:38:01", "content": "That's a neat decision from the beginning to do 50/50 of tree and nn.
\nIf you're willing to share your NN training setup, I can try to run it on 100% data and share with you the results.
Thanks for sharing! Congrats!
", "votes": null, "replies": [] }, { "id": 1913759, "author_name": "ashvanths", "author_url": "", "post_date": "08/25/2022 13:44:02", "content": "Damnn congrats @tatsumicrub !!
", "votes": null, "replies": [] }, { "id": 1913772, "author_name": "cdeotte", "author_url": "", "post_date": "08/25/2022 13:56:08", "content": "Congratulations! Great model and great result. Can you explain more about
\n\n\nI attached Efficientnet with a transformer
\n
Did you convert the competition data into 2D images or something for the EfficientNet CNN to process?
", "votes": null, "replies": [] }, { "id": 1913866, "author_name": "aninda", "author_url": "", "post_date": "08/25/2022 14:50:46", "content": "Amazing results with limited resources
", "votes": null, "replies": [] }, { "id": 1913959, "author_name": "tatsumicrub", "author_url": "", "post_date": "08/25/2022 15:50:19", "content": "@nyleve
\nI'll try to train with the rest o 40% data! Though it is really kind of you!! Thank you so much!!
@cdeotte
\nThank you for your comment! Your notebook helped us a lot! I attached the efficientnet at the output layer rather than the input layer. So the data preparation is exactly the same as your transformer notebook. Since the transformer models can handle continuous data better than efficientnet, first I fed the data to the transformer. Then I flatten the output from the transformer, reshaped the data into the efficienet input shape, and fed it to the efficientnet.
That's awesome, great idea. Did you train with any pseudo labeled test data? Or does your solution only use the train data?
", "votes": null, "replies": [] } ] }, { "id": 1913983, "author_name": "tatsumicrub", "author_url": "", "post_date": "08/25/2022 16:08:48", "content": "Thank you guys for the comments! Hope we'll do our best in upcoming competitions!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1913250": "Thank you Amex for hosting the amazing competition. Thank you to the whole Kaggle community and Kagglers who have attributed to this competition and provided the amazing opportunity to compete and learn!!\n\n## Abstract\nI, fortunately, got the 30th place with 720 places shake-up. I think it is an interesting case since no one in the upper rankers has got the shake-up as much as I did. I'd like to share the experience of that for the record, and please comment if you guys have any ideas why this huge shake-up happened. Also, I'm looking for a team to join the Kaggle competitions. If there's anyone who considers it, feel free to contact me.\n\n## 720 places Shake-up\nI couldn't believe it when I woke up and saw on the private leaderboard that my submission was in 30th place. Last night when I checked it, the public leader board was 750th place😨. For your information, I am a university student in japan and there's 20 hours difference😃. I thought that the ensemble of NN + LGBM was the most likely solution, therefore I didn't really care about the public leader board though it struck me with a major surprise. It proves that we don't know what's gonna happen in the private leader board, and most of the participants were overfitting for the public leader board.\n\n## Solution overview\nMy solution was similar to his. The main idea was the 50%/50% ensemble of LGBM and NN Transformer.\n[Mr. Chris Deotte's solution](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347641)\nIn my case, I'm a broke university student, and I couldn't afford the computational resource. I used the free quote of Kaggle GPU and the resource was very limited. I attached Efficientnet with a transformer, and I could only use 60% of the competition data for the training session. That's one thing I regret. For LGBM models, I simply used the public LGBM notebook for the LGBM ensemble. I think the reason why I got this place was the NN model part. Although I had poor resources and couldn't do my best, I think that the transformer+efficientnet structure was the reason for my rank. That shows the infinite possibility of the NN!!\n\n## What I learned\nI myself cannot believe that I got the top 0.6% place in this competition with this poor resource. Even though I could only use 60% of training data for NN, and public notebooks for the LGBM model, the private score was good. I think it means that this solution was one of the best answers for this competition. I'd love to see what happens if I could have used the full 100% of the training data for the NN models and properly train the LGBM models. Could I win the gold medal, maybe?\n\n## Closure\nThanks again to everyone who contributed to this competition. And again, I am desperately looking for a team. I'd love to learn from the team and I believe I can be helpful to the whole team. Please contact me if you can let me join your team😊", "1913355": "That's a neat decision from the beginning to do 50/50 of tree and nn.\nIf you're willing to share your NN training setup, I can try to run it on 100% data and share with you the results.", "1913378": "Thanks for sharing! Congrats!", "1913759": "Damnn congrats @tatsumicrub !!", "1913772": "Congratulations! Great model and great result. Can you explain more about\n>I attached Efficientnet with a transformer\n\nDid you convert the competition data into 2D images or something for the EfficientNet CNN to process?", "1913866": "Amazing results with limited resources", "1913959": "nyleve \nI'll try to train with the rest o 40% data! Though it is really kind of you!! Thank you so much!!", "1913977": "cdeotte \nThank you for your comment! Your notebook helped us a lot! I attached the efficientnet at the output layer rather than the input layer. So the data preparation is exactly the same as your transformer notebook. Since the transformer models can handle continuous data better than efficientnet, first I fed the data to the transformer. Then I flatten the output from the transformer, reshaped the data into the efficienet input shape, and fed it to the efficientnet.", "1913983": "Thank you guys for the comments! Hope we'll do our best in upcoming competitions!", "1913997": "That's awesome, great idea. Did you train with any pseudo labeled test data? Or does your solution only use the train data?" }, "source": "meta" }