First, I would like to congratulate the winners and all the participants.\nThe Private LeaderBoard is a lot different from the public one, that was happening also in the last competition \"Microsoft Malware detection\"
\n\nIs this normal ??
", "messages": [ { "id": "496156", "postDate": "03/22/2019 00:09:20", "content": "First, I would like to congratulate the winners and all the participants.\nThe Private LeaderBoard is a lot different from the public one, that was happening also in the last competition \"Microsoft Malware detection\"
\n\nIs this normal ??
", "rawMarkdown": "First, I would like to congratulate the winners and all the participants.\nThe Private LeaderBoard is a lot different from the public one, that was happening also in the last competition \"Microsoft Malware detection\"\n\nIs this normal ??", "votes": null }, { "id": "496157", "postDate": "03/22/2019 00:10:04", "content": "Definitely not
", "rawMarkdown": "Definitely not", "votes": null }, { "id": "496163", "postDate": "03/22/2019 00:11:34", "content": "This is not normal, but happens sometimes :)
", "rawMarkdown": "This is not normal, but happens sometimes :)", "votes": null }, { "id": "496164", "postDate": "03/22/2019 00:11:39", "content": "This was bigger than the last shake. But had a good experience using RNN's
", "rawMarkdown": "This was bigger than the last shake. But had a good experience using RNN's", "votes": null }, { "id": "496172", "postDate": "03/22/2019 00:28:29", "content": "This is not normal, huge shake-ups making kaggle competitions more interesting. Next time I will enter a competition, trying to do something in first month then I will just wait. Maybe with that way I can win a gold medal.
", "rawMarkdown": "This is not normal, huge shake-ups making kaggle competitions more interesting. Next time I will enter a competition, trying to do something in first month then I will just wait. Maybe with that way I can win a gold medal.", "votes": null }, { "id": "496280", "postDate": "03/22/2019 02:50:04", "content": "I thought it's not normal but it's 2 consecutive shake down for me in MS and VSB... Now I realize why so many kagglers suggest \"Trust your CV\".
", "rawMarkdown": "I thought it's not normal but it's 2 consecutive shake down for me in MS and VSB... Now I realize why so many kagglers suggest \"Trust your CV\".", "votes": null }, { "id": "496287", "postDate": "03/22/2019 02:56:07", "content": "Trust your cv is what all you should do.
", "rawMarkdown": "Trust your cv is what all you should do.", "votes": null }, { "id": "496299", "postDate": "03/22/2019 03:06:54", "content": "To my friends that say “trust your CV”, would you mind sharing your thought here : \nhttps://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/85167
\n\n(issue 2, on the CV vs. LB where LB number of data is actually higher)
", "rawMarkdown": "To my friends that say “trust your CV”, would you mind sharing your thought here : \nhttps://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/85167\n\n(issue 2, on the CV vs. LB where LB number of data is actually higher)", "votes": null }, { "id": "496381", "postDate": "03/22/2019 05:15:54", "content": "I've been in a lot of Kaggle competitions (this is my 36th!), and several have had big shakeups. I've had the good fortune of mostly being \"shaken up\" as opposed to \"shaken down\". Based on this contest's preliminary standings I went from rank 1010 on the Public LB to 50 on the Private. I didn't expect this result, since I'm not an expert on this kind of signal processing, and for most of the competition I felt like I was floundering, with little idea of what methods would work or how to improve my results. But I did stick to a general principle that has served me well in previous competitions: don't rely on the published kernels except for interesting ideas worth incorporating into my own algorithms. Competitions with big shakeups have usually featured one or more popular kernels with good Public LB scores that were due at least partly to overfitting. Those kernels have often dropped way down on the Private LB, taking their adherents with them. This appears to have happened to some extent in this contest as well. For example, there were a lot of participants who scored 0.694 (the score of a popular kernel) on the Public LB that dropped to 0.609 on the Private LB.\nBut the above only partially explains my final ranking. At the end of a competition I often try to do a \"post-mortem\" exam to figure out what went wrong. In this case I am faced with having to figure out what went right, which is still largely a mystery.
", "rawMarkdown": "I've been in a lot of Kaggle competitions (this is my 36th!), and several have had big shakeups. I've had the good fortune of mostly being \"shaken up\" as opposed to \"shaken down\". Based on this contest's preliminary standings I went from rank 1010 on the Public LB to 50 on the Private. I didn't expect this result, since I'm not an expert on this kind of signal processing, and for most of the competition I felt like I was floundering, with little idea of what methods would work or how to improve my results. But I did stick to a general principle that has served me well in previous competitions: don't rely on the published kernels except for interesting ideas worth incorporating into my own algorithms. Competitions with big shakeups have usually featured one or more popular kernels with good Public LB scores that were due at least partly to overfitting. Those kernels have often dropped way down on the Private LB, taking their adherents with them. This appears to have happened to some extent in this contest as well. For example, there were a lot of participants who scored 0.694 (the score of a popular kernel) on the Public LB that dropped to 0.609 on the Private LB.\nBut the above only partially explains my final ranking. At the end of a competition I often try to do a \"post-mortem\" exam to figure out what went wrong. In this case I am faced with having to figure out what went right, which is still largely a mystery.", "votes": null }, { "id": "496719", "postDate": "03/22/2019 14:09:17", "content": "Having read some of the other posts about the shakeup, I'm not so sure that the drop in score from 0.694 on Public LB to 0.609 on Private LB of the \"5-fold LSTM Attention (fully commented)\" kernel was just due to overfitting as I suggested above. The differences between Public and Private scores for many participants, including myself, are rather puzzling, and perhaps there are factors that involve the differences between the Public and Private test datasets, together with the characteristics of the MCC metric, that combined to produce these odd results.
\n\nIt would be interesting if the organizers could at some point reveal the test labels and the Public/Private data split to help explain the shakeup.
", "rawMarkdown": "Having read some of the other posts about the shakeup, I'm not so sure that the drop in score from 0.694 on Public LB to 0.609 on Private LB of the \"5-fold LSTM Attention (fully commented)\" kernel was just due to overfitting as I suggested above. The differences between Public and Private scores for many participants, including myself, are rather puzzling, and perhaps there are factors that involve the differences between the Public and Private test datasets, together with the characteristics of the MCC metric, that combined to produce these odd results.\n\nIt would be interesting if the organizers could at some point reveal the test labels and the Public/Private data split to help explain the shakeup.", "votes": null } ], "comments": [ { "id": 496157, "author_name": "strideradu", "author_url": "", "post_date": "03/22/2019 00:10:04", "content": "Definitely not
", "votes": null, "replies": [] }, { "id": 496163, "author_name": "titericz", "author_url": "", "post_date": "03/22/2019 00:11:34", "content": "This is not normal, but happens sometimes :)
", "votes": null, "replies": [] }, { "id": 496164, "author_name": "roydatascience", "author_url": "", "post_date": "03/22/2019 00:11:39", "content": "This was bigger than the last shake. But had a good experience using RNN's
", "votes": null, "replies": [] }, { "id": 496172, "author_name": "fatall", "author_url": "", "post_date": "03/22/2019 00:28:29", "content": "This is not normal, huge shake-ups making kaggle competitions more interesting. Next time I will enter a competition, trying to do something in first month then I will just wait. Maybe with that way I can win a gold medal.
", "votes": null, "replies": [] }, { "id": 496280, "author_name": "playif1", "author_url": "", "post_date": "03/22/2019 02:50:04", "content": "I thought it's not normal but it's 2 consecutive shake down for me in MS and VSB... Now I realize why so many kagglers suggest \"Trust your CV\".
", "votes": null, "replies": [] }, { "id": 496287, "author_name": "junyun1002", "author_url": "", "post_date": "03/22/2019 02:56:07", "content": "Trust your cv is what all you should do.
", "votes": null, "replies": [] }, { "id": 496299, "author_name": "ratthachat", "author_url": "", "post_date": "03/22/2019 03:06:54", "content": "To my friends that say “trust your CV”, would you mind sharing your thought here : \nhttps://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/85167
\n\n(issue 2, on the CV vs. LB where LB number of data is actually higher)
", "votes": null, "replies": [] }, { "id": 496381, "author_name": "dslate", "author_url": "", "post_date": "03/22/2019 05:15:54", "content": "I've been in a lot of Kaggle competitions (this is my 36th!), and several have had big shakeups. I've had the good fortune of mostly being \"shaken up\" as opposed to \"shaken down\". Based on this contest's preliminary standings I went from rank 1010 on the Public LB to 50 on the Private. I didn't expect this result, since I'm not an expert on this kind of signal processing, and for most of the competition I felt like I was floundering, with little idea of what methods would work or how to improve my results. But I did stick to a general principle that has served me well in previous competitions: don't rely on the published kernels except for interesting ideas worth incorporating into my own algorithms. Competitions with big shakeups have usually featured one or more popular kernels with good Public LB scores that were due at least partly to overfitting. Those kernels have often dropped way down on the Private LB, taking their adherents with them. This appears to have happened to some extent in this contest as well. For example, there were a lot of participants who scored 0.694 (the score of a popular kernel) on the Public LB that dropped to 0.609 on the Private LB.\nBut the above only partially explains my final ranking. At the end of a competition I often try to do a \"post-mortem\" exam to figure out what went wrong. In this case I am faced with having to figure out what went right, which is still largely a mystery.
", "votes": null, "replies": [ { "id": 496719, "author_name": "dslate", "author_url": "", "post_date": "03/22/2019 14:09:17", "content": "Having read some of the other posts about the shakeup, I'm not so sure that the drop in score from 0.694 on Public LB to 0.609 on Private LB of the \"5-fold LSTM Attention (fully commented)\" kernel was just due to overfitting as I suggested above. The differences between Public and Private scores for many participants, including myself, are rather puzzling, and perhaps there are factors that involve the differences between the Public and Private test datasets, together with the characteristics of the MCC metric, that combined to produce these odd results.
\n\nIt would be interesting if the organizers could at some point reveal the test labels and the Public/Private data split to help explain the shakeup.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "496156": "First, I would like to congratulate the winners and all the participants.\nThe Private LeaderBoard is a lot different from the public one, that was happening also in the last competition \"Microsoft Malware detection\"\n\nIs this normal ??", "496157": "Definitely not", "496163": "This is not normal, but happens sometimes :)", "496164": "This was bigger than the last shake. But had a good experience using RNN's", "496172": "This is not normal, huge shake-ups making kaggle competitions more interesting. Next time I will enter a competition, trying to do something in first month then I will just wait. Maybe with that way I can win a gold medal.", "496280": "I thought it's not normal but it's 2 consecutive shake down for me in MS and VSB... Now I realize why so many kagglers suggest \"Trust your CV\".", "496287": "Trust your cv is what all you should do.", "496299": "To my friends that say “trust your CV”, would you mind sharing your thought here : \nhttps://www.kaggle.com/c/vsb-power-line-fault-detection/discussion/85167\n\n(issue 2, on the CV vs. LB where LB number of data is actually higher)", "496381": "I've been in a lot of Kaggle competitions (this is my 36th!), and several have had big shakeups. I've had the good fortune of mostly being \"shaken up\" as opposed to \"shaken down\". Based on this contest's preliminary standings I went from rank 1010 on the Public LB to 50 on the Private. I didn't expect this result, since I'm not an expert on this kind of signal processing, and for most of the competition I felt like I was floundering, with little idea of what methods would work or how to improve my results. But I did stick to a general principle that has served me well in previous competitions: don't rely on the published kernels except for interesting ideas worth incorporating into my own algorithms. Competitions with big shakeups have usually featured one or more popular kernels with good Public LB scores that were due at least partly to overfitting. Those kernels have often dropped way down on the Private LB, taking their adherents with them. This appears to have happened to some extent in this contest as well. For example, there were a lot of participants who scored 0.694 (the score of a popular kernel) on the Public LB that dropped to 0.609 on the Private LB.\nBut the above only partially explains my final ranking. At the end of a competition I often try to do a \"post-mortem\" exam to figure out what went wrong. In this case I am faced with having to figure out what went right, which is still largely a mystery.", "496719": "Having read some of the other posts about the shakeup, I'm not so sure that the drop in score from 0.694 on Public LB to 0.609 on Private LB of the \"5-fold LSTM Attention (fully commented)\" kernel was just due to overfitting as I suggested above. The differences between Public and Private scores for many participants, including myself, are rather puzzling, and perhaps there are factors that involve the differences between the Public and Private test datasets, together with the characteristics of the MCC metric, that combined to produce these odd results.\n\nIt would be interesting if the organizers could at some point reveal the test labels and the Public/Private data split to help explain the shakeup." }, "source": "meta" }