Once the dust settles can Kaggle / LANL gives us feedback in terms of what went right and what you would do differently?
\n\nI am not sure why this isn't done for every competition.
", "messages": [ { "id": "543123", "postDate": "06/04/2019 10:41:57", "content": "Once the dust settles can Kaggle / LANL gives us feedback in terms of what went right and what you would do differently?
\n\nI am not sure why this isn't done for every competition.
", "rawMarkdown": "Once the dust settles can Kaggle / LANL gives us feedback in terms of what went right and what you would do differently?\n\nI am not sure why this isn't done for every competition.", "votes": null }, { "id": "543269", "postDate": "06/04/2019 12:30:29", "content": "With all due respect I would be surprised if the solutions perform as good as they were on any of the LB for any competition by any participant. Real world is different, totally different. Approaches like Gradient Boosted Decision Trees (GBDT) which win almost all the competitions make a lot of assumptions which are met in Test and Training set. It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment.
\n\nFor real world scenarios approaches like Markov Chains, Hamiltonian Monte Carlo etc. are suitable. But again depends on how one sees the problem. I prefer having least or no assumptions if possible.
\n\nEdit 1: One competition which I have come across which resembles real world scenario is https://www.kaggle.com/c/two-sigma-financial-news/overview/submission-instructions \nIt was regarding time series and predictions were made for future data in real world (stage 2). Being a realistic competition this was not very popular I was surprised to see no big named Kagglers in this one. Unfortunately I didn't qualify because of stage 2 kernel failure (in short didn't follow a rule of the competition) but it was great experience.
", "rawMarkdown": "With all due respect I would be surprised if the solutions perform as good as they were on any of the LB for any competition by any participant. Real world is different, totally different. Approaches like Gradient Boosted Decision Trees (GBDT) which win almost all the competitions make a lot of assumptions which are met in Test and Training set. It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment.\n\nFor real world scenarios approaches like Markov Chains, Hamiltonian Monte Carlo etc. are suitable. But again depends on how one sees the problem. I prefer having least or no assumptions if possible. \n\nEdit 1: One competition which I have come across which resembles real world scenario is https://www.kaggle.com/c/two-sigma-financial-news/overview/submission-instructions \nIt was regarding time series and predictions were made for future data in real world (stage 2). Being a realistic competition this was not very popular I was surprised to see no big named Kagglers in this one. Unfortunately I didn't qualify because of stage 2 kernel failure (in short didn't follow a rule of the competition) but it was great experience.", "votes": null }, { "id": "543277", "postDate": "06/04/2019 12:38:12", "content": "How is this relevant to feedback from Kaggle and LANL?
\n\nAha - \"It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment\" - got you this is relevant - apologies! ;)
", "rawMarkdown": "How is this relevant to feedback from Kaggle and LANL?\n\nAha - \"It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment\" - got you this is relevant - apologies! ;)", "votes": null }, { "id": "543780", "postDate": "06/04/2019 20:15:26", "content": "Personally, I like the approach in the link. Not a fan of micro-manipulations just for the sake of winning the competition. I have learned a lot. I compiled several models. Models performed great in my local testing, but did not generalize well because test data were quite different from training data. Kind of disappointed that the organizer did it that way. Would like to know their opinions on the competition and what they could learn from it.\nhttps://www.kaggle.com/wimwim/wavenet-lstm
", "rawMarkdown": "Personally, I like the approach in the link. Not a fan of micro-manipulations just for the sake of winning the competition. I have learned a lot. I compiled several models. Models performed great in my local testing, but did not generalize well because test data were quite different from training data. Kind of disappointed that the organizer did it that way. Would like to know their opinions on the competition and what they could learn from it.\nhttps://www.kaggle.com/wimwim/wavenet-lstm", "votes": null } ], "comments": [ { "id": 543269, "author_name": "cyberia", "author_url": "", "post_date": "06/04/2019 12:30:29", "content": "With all due respect I would be surprised if the solutions perform as good as they were on any of the LB for any competition by any participant. Real world is different, totally different. Approaches like Gradient Boosted Decision Trees (GBDT) which win almost all the competitions make a lot of assumptions which are met in Test and Training set. It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment.
\n\nFor real world scenarios approaches like Markov Chains, Hamiltonian Monte Carlo etc. are suitable. But again depends on how one sees the problem. I prefer having least or no assumptions if possible.
\n\nEdit 1: One competition which I have come across which resembles real world scenario is https://www.kaggle.com/c/two-sigma-financial-news/overview/submission-instructions \nIt was regarding time series and predictions were made for future data in real world (stage 2). Being a realistic competition this was not very popular I was surprised to see no big named Kagglers in this one. Unfortunately I didn't qualify because of stage 2 kernel failure (in short didn't follow a rule of the competition) but it was great experience.
", "votes": null, "replies": [ { "id": 543277, "author_name": "scirpus", "author_url": "", "post_date": "06/04/2019 12:38:12", "content": "How is this relevant to feedback from Kaggle and LANL?
\n\nAha - \"It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment\" - got you this is relevant - apologies! ;)
", "votes": null, "replies": [] } ] }, { "id": 543780, "author_name": "joejeo1", "author_url": "", "post_date": "06/04/2019 20:15:26", "content": "Personally, I like the approach in the link. Not a fan of micro-manipulations just for the sake of winning the competition. I have learned a lot. I compiled several models. Models performed great in my local testing, but did not generalize well because test data were quite different from training data. Kind of disappointed that the organizer did it that way. Would like to know their opinions on the competition and what they could learn from it.\nhttps://www.kaggle.com/wimwim/wavenet-lstm
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "543123": "Once the dust settles can Kaggle / LANL gives us feedback in terms of what went right and what you would do differently?\n\nI am not sure why this isn't done for every competition.", "543269": "With all due respect I would be surprised if the solutions perform as good as they were on any of the LB for any competition by any participant. Real world is different, totally different. Approaches like Gradient Boosted Decision Trees (GBDT) which win almost all the competitions make a lot of assumptions which are met in Test and Training set. It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment.\n\nFor real world scenarios approaches like Markov Chains, Hamiltonian Monte Carlo etc. are suitable. But again depends on how one sees the problem. I prefer having least or no assumptions if possible. \n\nEdit 1: One competition which I have come across which resembles real world scenario is https://www.kaggle.com/c/two-sigma-financial-news/overview/submission-instructions \nIt was regarding time series and predictions were made for future data in real world (stage 2). Being a realistic competition this was not very popular I was surprised to see no big named Kagglers in this one. Unfortunately I didn't qualify because of stage 2 kernel failure (in short didn't follow a rule of the competition) but it was great experience.", "543277": "How is this relevant to feedback from Kaggle and LANL?\n\nAha - \"It would be great if Kaggle could tell how well the solution did after a test implementation in their real world dev environment\" - got you this is relevant - apologies! ;)", "543780": "Personally, I like the approach in the link. Not a fan of micro-manipulations just for the sake of winning the competition. I have learned a lot. I compiled several models. Models performed great in my local testing, but did not generalize well because test data were quite different from training data. Kind of disappointed that the organizer did it that way. Would like to know their opinions on the competition and what they could learn from it.\nhttps://www.kaggle.com/wimwim/wavenet-lstm" }, "source": "meta" }