Health and other concerns sometimes keep me away from participation and following Kaggle challenges. Not logged in for about a week, but when I do I see a couple new leaders. Incredible score 0.00000 don't remember seeing them in top 10 before last weekend. So, in 2 days, i.e max of 10 submissions, their score improved by factor of 10000; unless I missed something in my absence, has to be at least 1000 times. That's some crazy skills. Wondering if a model change and/or training changed is reason. Either way it's a huge accomplishment. Props to 2nd place as well, seems nearly same improvement in same time, at same time as 1st. Nice.
", "messages": [ { "id": "233581", "postDate": "10/20/2017 15:47:40", "content": "Health and other concerns sometimes keep me away from participation and following Kaggle challenges. Not logged in for about a week, but when I do I see a couple new leaders. Incredible score 0.00000 don't remember seeing them in top 10 before last weekend. So, in 2 days, i.e max of 10 submissions, their score improved by factor of 10000; unless I missed something in my absence, has to be at least 1000 times. That's some crazy skills. Wondering if a model change and/or training changed is reason. Either way it's a huge accomplishment. Props to 2nd place as well, seems nearly same improvement in same time, at same time as 1st. Nice.
", "rawMarkdown": "Health and other concerns sometimes keep me away from participation and following Kaggle challenges. Not logged in for about a week, but when I do I see a couple new leaders. Incredible score 0.00000 don't remember seeing them in top 10 before last weekend. So, in 2 days, i.e max of 10 submissions, their score improved by factor of 10000; unless I missed something in my absence, has to be at least 1000 times. That's some crazy skills. Wondering if a model change and/or training changed is reason. Either way it's a huge accomplishment. Props to 2nd place as well, seems nearly same improvement in same time, at same time as 1st. Nice.", "votes": null }, { "id": "233586", "postDate": "10/20/2017 15:56:57", "content": "There are only 100 samples in the stage 1 test set, with the right amount of motivation and hand labeling anyone can get a perfect score in stage 1!!! if they get a perfect score in stage 2! then you are on to something here...
", "rawMarkdown": "There are only 100 samples in the stage 1 test set, with the right amount of motivation and hand labeling anyone can get a perfect score in stage 1!!! if they get a perfect score in stage 2! then you are on to something here...", "votes": null }, { "id": "233640", "postDate": "10/20/2017 18:45:37", "content": "It's a person (or 2 people) using 4 accounts to probe the leaderboard. It's pretty clear when you look at the submission patterns of the 4 accounts in the submission data.
", "rawMarkdown": "It's a person (or 2 people) using 4 accounts to probe the leaderboard. It's pretty clear when you look at the submission patterns of the 4 accounts in the submission data.", "votes": null }, { "id": "233653", "postDate": "10/20/2017 19:19:59", "content": "Ha, yeah I know. Was being a smart ass; but really sound like I wasn't. :) Was just messing with cheaters.
", "rawMarkdown": "Ha, yeah I know. Was being a smart ass; but really sound like I wasn't. :) Was just messing with cheaters.", "votes": null }, { "id": "233655", "postDate": "10/20/2017 19:23:47", "content": "I think its actually 5 accounts...
", "rawMarkdown": "I think its actually 5 accounts...", "votes": null }, { "id": "233671", "postDate": "10/20/2017 19:54:17", "content": "Yeah it is, missed that 5th one.
", "rawMarkdown": "Yeah it is, missed that 5th one.", "votes": null }, { "id": "237201", "postDate": "10/29/2017 20:03:06", "content": "I am kind of new here trying to learn machine learning. So pardon my noobie question. But, what's preventing some one from hand labeling the test set in the 2nd stage and simply submit them? Is it because in the 2nd stage, we only submit the model code but not the model generated labels ourselves, and Kaggle will read through each model code and run it to generate the labels to evaluate?
", "rawMarkdown": "I am kind of new here trying to learn machine learning. So pardon my noobie question. But, what's preventing some one from hand labeling the test set in the 2nd stage and simply submit them? Is it because in the 2nd stage, we only submit the model code but not the model generated labels ourselves, and Kaggle will read through each model code and run it to generate the labels to evaluate?", "votes": null }, { "id": "238584", "postDate": "11/01/2017 16:42:46", "content": "@DeltoiX: The short answer is: it would disqualify that submission, according to the rules. We still submit labels in the second stage, and the private leaderboard will be based on those labels. The competition sponsors will run the code for top-scoring submissions to make sure the code actually generates the labels that were submitted. (To be eligible for a prize, teams must upload their models by the first stage deadline. I doubt, however, that the models will be run unless they are for top-scoring submissions. It would take enormous effort to install/adapt/run everyone's submitted code.)
", "rawMarkdown": "DeltoiX: The short answer is: it would disqualify that submission, according to the rules. We still submit labels in the second stage, and the private leaderboard will be based on those labels. The competition sponsors will run the code for top-scoring submissions to make sure the code actually generates the labels that were submitted. (To be eligible for a prize, teams must upload their models by the first stage deadline. I doubt, however, that the models will be run unless they are for top-scoring submissions. It would take enormous effort to install/adapt/run everyone's submitted code.)", "votes": null }, { "id": "240661", "postDate": "11/07/2017 05:22:04", "content": "For what it's worth, I'm having an awful time beating the statistical baseline. If anybody is scoring under 0.1 fairly -- meaning their model(s) are generalizing -- my hat's off to your feature engineering skills :)
", "rawMarkdown": "For what it's worth, I'm having an awful time beating the statistical baseline. If anybody is scoring under 0.1 fairly -- meaning their model(s) are generalizing -- my hat's off to your feature engineering skills :)", "votes": null }, { "id": "242917", "postDate": "11/13/2017 01:00:34", "content": "Can you please list all of them?
", "rawMarkdown": "Can you please list all of them?", "votes": null }, { "id": "242919", "postDate": "11/13/2017 01:04:55", "content": "Do they also check model training? Or do they only predict results using the model on the pre-trained weights? (Retraining gives different results each time and it's impossible to get the same results if the model is retrained.)
", "rawMarkdown": "Do they also check model training? Or do they only predict results using the model on the pre-trained weights? (Retraining gives different results each time and it's impossible to get the same results if the model is retrained.)", "votes": null }, { "id": "242925", "postDate": "11/13/2017 01:22:20", "content": "Dmitry,
\n\nWinning algorithms must be able to reproduce the leaderboard score.
", "rawMarkdown": "Dmitry,\n\nWinning algorithms must be able to reproduce the leaderboard score.", "votes": null }, { "id": "242927", "postDate": "11/13/2017 01:25:46", "content": "Of course. Winning algorithm should read pre-trained weights and output predictions with the same leaderboard score. But do they check (and how) model training?
", "rawMarkdown": "Of course. Winning algorithm should read pre-trained weights and output predictions with the same leaderboard score. But do they check (and how) model training?", "votes": null } ], "comments": [ { "id": 233586, "author_name": "godaibo", "author_url": "", "post_date": "10/20/2017 15:56:57", "content": "There are only 100 samples in the stage 1 test set, with the right amount of motivation and hand labeling anyone can get a perfect score in stage 1!!! if they get a perfect score in stage 2! then you are on to something here...
", "votes": null, "replies": [] }, { "id": 233640, "author_name": "brandenkmurray", "author_url": "", "post_date": "10/20/2017 18:45:37", "content": "It's a person (or 2 people) using 4 accounts to probe the leaderboard. It's pretty clear when you look at the submission patterns of the 4 accounts in the submission data.
", "votes": null, "replies": [ { "id": 233655, "author_name": "jamesrequa", "author_url": "", "post_date": "10/20/2017 19:23:47", "content": "I think its actually 5 accounts...
", "votes": null, "replies": [] }, { "id": 233671, "author_name": "brandenkmurray", "author_url": "", "post_date": "10/20/2017 19:54:17", "content": "Yeah it is, missed that 5th one.
", "votes": null, "replies": [] }, { "id": 242917, "author_name": "dmitrykovba", "author_url": "", "post_date": "11/13/2017 01:00:34", "content": "Can you please list all of them?
", "votes": null, "replies": [] } ] }, { "id": 233653, "author_name": "srdhaeayautnon", "author_url": "", "post_date": "10/20/2017 19:19:59", "content": "Ha, yeah I know. Was being a smart ass; but really sound like I wasn't. :) Was just messing with cheaters.
", "votes": null, "replies": [] }, { "id": 237201, "author_name": "deltoix", "author_url": "", "post_date": "10/29/2017 20:03:06", "content": "I am kind of new here trying to learn machine learning. So pardon my noobie question. But, what's preventing some one from hand labeling the test set in the 2nd stage and simply submit them? Is it because in the 2nd stage, we only submit the model code but not the model generated labels ourselves, and Kaggle will read through each model code and run it to generate the labels to evaluate?
", "votes": null, "replies": [ { "id": 238584, "author_name": "nathanrm", "author_url": "", "post_date": "11/01/2017 16:42:46", "content": "@DeltoiX: The short answer is: it would disqualify that submission, according to the rules. We still submit labels in the second stage, and the private leaderboard will be based on those labels. The competition sponsors will run the code for top-scoring submissions to make sure the code actually generates the labels that were submitted. (To be eligible for a prize, teams must upload their models by the first stage deadline. I doubt, however, that the models will be run unless they are for top-scoring submissions. It would take enormous effort to install/adapt/run everyone's submitted code.)
", "votes": null, "replies": [] }, { "id": 242919, "author_name": "dmitrykovba", "author_url": "", "post_date": "11/13/2017 01:04:55", "content": "Do they also check model training? Or do they only predict results using the model on the pre-trained weights? (Retraining gives different results each time and it's impossible to get the same results if the model is retrained.)
", "votes": null, "replies": [] }, { "id": 242925, "author_name": "addisonhoward", "author_url": "", "post_date": "11/13/2017 01:22:20", "content": "Dmitry,
\n\nWinning algorithms must be able to reproduce the leaderboard score.
", "votes": null, "replies": [] }, { "id": 242927, "author_name": "dmitrykovba", "author_url": "", "post_date": "11/13/2017 01:25:46", "content": "Of course. Winning algorithm should read pre-trained weights and output predictions with the same leaderboard score. But do they check (and how) model training?
", "votes": null, "replies": [] } ] }, { "id": 240661, "author_name": "mmiron", "author_url": "", "post_date": "11/07/2017 05:22:04", "content": "For what it's worth, I'm having an awful time beating the statistical baseline. If anybody is scoring under 0.1 fairly -- meaning their model(s) are generalizing -- my hat's off to your feature engineering skills :)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "233581": "Health and other concerns sometimes keep me away from participation and following Kaggle challenges. Not logged in for about a week, but when I do I see a couple new leaders. Incredible score 0.00000 don't remember seeing them in top 10 before last weekend. So, in 2 days, i.e max of 10 submissions, their score improved by factor of 10000; unless I missed something in my absence, has to be at least 1000 times. That's some crazy skills. Wondering if a model change and/or training changed is reason. Either way it's a huge accomplishment. Props to 2nd place as well, seems nearly same improvement in same time, at same time as 1st. Nice.", "233586": "There are only 100 samples in the stage 1 test set, with the right amount of motivation and hand labeling anyone can get a perfect score in stage 1!!! if they get a perfect score in stage 2! then you are on to something here...", "233640": "It's a person (or 2 people) using 4 accounts to probe the leaderboard. It's pretty clear when you look at the submission patterns of the 4 accounts in the submission data.", "233653": "Ha, yeah I know. Was being a smart ass; but really sound like I wasn't. :) Was just messing with cheaters.", "233655": "I think its actually 5 accounts...", "233671": "Yeah it is, missed that 5th one.", "237201": "I am kind of new here trying to learn machine learning. So pardon my noobie question. But, what's preventing some one from hand labeling the test set in the 2nd stage and simply submit them? Is it because in the 2nd stage, we only submit the model code but not the model generated labels ourselves, and Kaggle will read through each model code and run it to generate the labels to evaluate?", "238584": "DeltoiX: The short answer is: it would disqualify that submission, according to the rules. We still submit labels in the second stage, and the private leaderboard will be based on those labels. The competition sponsors will run the code for top-scoring submissions to make sure the code actually generates the labels that were submitted. (To be eligible for a prize, teams must upload their models by the first stage deadline. I doubt, however, that the models will be run unless they are for top-scoring submissions. It would take enormous effort to install/adapt/run everyone's submitted code.)", "240661": "For what it's worth, I'm having an awful time beating the statistical baseline. If anybody is scoring under 0.1 fairly -- meaning their model(s) are generalizing -- my hat's off to your feature engineering skills :)", "242917": "Can you please list all of them?", "242919": "Do they also check model training? Or do they only predict results using the model on the pre-trained weights? (Retraining gives different results each time and it's impossible to get the same results if the model is retrained.)", "242925": "Dmitry,\n\nWinning algorithms must be able to reproduce the leaderboard score.", "242927": "Of course. Winning algorithm should read pre-trained weights and output predictions with the same leaderboard score. But do they check (and how) model training?" }, "source": "meta" }