Should we all quit or is something wrong with top 1? Is results mining allowed? \nAnyways, congrats to Carlos Grajeda if with only 3 submissions he was able to reach the top!
", "messages": [ { "id": "178546", "postDate": "04/28/2017 06:37:50", "content": "Should we all quit or is something wrong with top 1? Is results mining allowed? \nAnyways, congrats to Carlos Grajeda if with only 3 submissions he was able to reach the top!
", "rawMarkdown": "Should we all quit or is something wrong with top 1? Is results mining allowed? \nAnyways, congrats to Carlos Grajeda if with only 3 submissions he was able to reach the top!", "votes": null }, { "id": "178588", "postDate": "04/28/2017 09:01:26", "content": "Andrei,\nPossibly it is a case of leader board mining. See here: https://www.kaggle.com/olegtrott/data-science-bowl-2017/the-perfect-score-script
", "rawMarkdown": "Andrei,\nPossibly it is a case of leader board mining. See here: https://www.kaggle.com/olegtrott/data-science-bowl-2017/the-perfect-score-script", "votes": null }, { "id": "178600", "postDate": "04/28/2017 10:12:43", "content": "But 3 submissions are too few for this
", "rawMarkdown": "But 3 submissions are too few for this", "votes": null }, { "id": "178627", "postDate": "04/28/2017 13:00:45", "content": "If someone brute force and get perfect score, will he/she will be eligible for proce?
", "rawMarkdown": "If someone brute force and get perfect score, will he/she will be eligible for proce?", "votes": null }, { "id": "178659", "postDate": "04/28/2017 14:58:59", "content": "haha, Have you heard? If ever someone gets a perfect loss. They have overfitted.
", "rawMarkdown": "haha, Have you heard? If ever someone gets a perfect loss. They have overfitted.", "votes": null }, { "id": "178660", "postDate": "04/28/2017 14:59:10", "content": "of course not
", "rawMarkdown": "of course not", "votes": null }, { "id": "178685", "postDate": "04/28/2017 16:45:10", "content": "Actually he took 2 submission to probe LB and the 3th to submit the best score. 2 Subs to probe 512 records of 3 classes labels and taking into account Kaggle round scores to 5 digits?!? Impressive. I would say impossible before saw it.
", "rawMarkdown": "Actually he took 2 submission to probe LB and the 3th to submit the best score. 2 Subs to probe 512 records of 3 classes labels and taking into account Kaggle round scores to 5 digits?!? Impressive. I would say impossible before saw it.", "votes": null }, { "id": "178686", "postDate": "04/28/2017 16:46:28", "content": "It's impressive, but balanced out by this being a 2-stage competition.
", "rawMarkdown": "It's impressive, but balanced out by this being a 2-stage competition.", "votes": null }, { "id": "178691", "postDate": "04/28/2017 16:55:58", "content": "Please tell me you are kidding XD
\n\nIt's not possible to extract more than 7 truth values per submission. There's a few people who have gone through the trouble of doing so (me included), and this is just the first account that was created just to post that 0 LB entry.
\n\nIf the competition maintainers don't want the LB to become completely unusable, they should delete this account (breaking site rules, multiple accounts or sharing data outside of the forum).
\n\nIf they don't care, then I'll post the full truth values in a couple days so everyone can benefit from them :-)
", "rawMarkdown": "Please tell me you are kidding XD\n\nIt's not possible to extract more than 7 truth values per submission. There's a few people who have gone through the trouble of doing so (me included), and this is just the first account that was created just to post that 0 LB entry.\n\nIf the competition maintainers don't want the LB to become completely unusable, they should delete this account (breaking site rules, multiple accounts or sharing data outside of the forum).\n\nIf they don't care, then I'll post the full truth values in a couple days so everyone can benefit from them :-)", "votes": null }, { "id": "178712", "postDate": "04/28/2017 18:17:39", "content": "Thanks r4m0n for taking the time to explain all this. I found one of your other posts here: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/30150#167944 talking about this same thing.
\n\nJust to clarify, is this a 2-stage competition? If so, this won't be a problem in second stage, but it clearly demotivates the rest of us Machine Learning mortals.
\n\nThe account is new, that's correct.
", "rawMarkdown": "Thanks r4m0n for taking the time to explain all this. I found one of your other posts here: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/30150#167944 talking about this same thing.\n\nJust to clarify, is this a 2-stage competition? If so, this won't be a problem in second stage, but it clearly demotivates the rest of us Machine Learning mortals.\n \nThe account is new, that's correct.", "votes": null }, { "id": "178717", "postDate": "04/28/2017 18:23:10", "content": "I'd like to understand why this is not braking the site rules? and how the site admins are going to react to this kind of cheats? (Only if its one, of course).
", "rawMarkdown": "I'd like to understand why this is not braking the site rules? and how the site admins are going to react to this kind of cheats? (Only if its one, of course).", "votes": null }, { "id": "178718", "postDate": "04/28/2017 18:25:14", "content": "Yes, this is a 2-stage competition and yes, people having the stage 1 truth and 0 LB scores won't cause any problems for the second stage. All that's going to happen is that the stage 1 LB is going to become full of meaningless scores.
\n\nFor anyone competing, don't worry about this, and just don't take any unreasonably good LB scores seriously, they aren't. And just to make it clear, once you have the stage 1 truth values, you can make ANY score on the LB. Right now only the 0 LB score is unreasonable, but don't be discouraged when 0.4's and 0.3's start popping up.
", "rawMarkdown": "Yes, this is a 2-stage competition and yes, people having the stage 1 truth and 0 LB scores won't cause any problems for the second stage. All that's going to happen is that the stage 1 LB is going to become full of meaningless scores.\n\nFor anyone competing, don't worry about this, and just don't take any unreasonably good LB scores seriously, they aren't. And just to make it clear, once you have the stage 1 truth values, you can make ANY score on the LB. Right now only the 0 LB score is unreasonable, but don't be discouraged when 0.4's and 0.3's start popping up.", "votes": null }, { "id": "178719", "postDate": "04/28/2017 18:29:31", "content": "Mining the stage 1 truth values doesn't break any rules, creating a second account or sending the truth values to someone else does. On the DSB the organizers let the original poster of the 0 LB score stay (Oleg's) and mostly kept the LB clean by deleting any other submissions. On this one the first 0 LB poster is obviously breaking the rules, and I expect the organizers to be harsher.
", "rawMarkdown": "Mining the stage 1 truth values doesn't break any rules, creating a second account or sending the truth values to someone else does. On the DSB the organizers let the original poster of the 0 LB score stay (Oleg's) and mostly kept the LB clean by deleting any other submissions. On this one the first 0 LB poster is obviously breaking the rules, and I expect the organizers to be harsher.", "votes": null }, { "id": "178735", "postDate": "04/28/2017 19:04:38", "content": "Actually, I find this leaderboard mining very wrong and it should be banned by Kaggle as it effectively increases train set size for those who mined the leaderboard, giving them an unfair advantage in tuning model parameters with full dataset. There's no need to explain how you got your parameters and having an extra 30% dataset can be very beneficial. 2-stage competition model does not eliminate this advantage in any way.
", "rawMarkdown": "Actually, I find this leaderboard mining very wrong and it should be banned by Kaggle as it effectively increases train set size for those who mined the leaderboard, giving them an unfair advantage in tuning model parameters with full dataset. There's no need to explain how you got your parameters and having an extra 30% dataset can be very beneficial. 2-stage competition model does not eliminate this advantage in any way.", "votes": null }, { "id": "178738", "postDate": "04/28/2017 19:09:37", "content": "In the DSB the stage 1 truth was released in the stage 2 to be used to train your model and keep things fair, and I fully expect them to do the same here. I haven't yet heard from any of the organizers here about this and I sure hope they make their position clear here in the next few days.
\n\nIf they don't make their position clear until, say, Tuesday, I'll make the point moot by just releasing the full stage 1 truth here on the forum. I don't think that would be against any of the rules, and it would remove any disadvantage for people who don't have it (and make the LB a complete mess, but I can't do anything about that).
", "rawMarkdown": "In the DSB the stage 1 truth was released in the stage 2 to be used to train your model and keep things fair, and I fully expect them to do the same here. I haven't yet heard from any of the organizers here about this and I sure hope they make their position clear here in the next few days.\n\nIf they don't make their position clear until, say, Tuesday, I'll make the point moot by just releasing the full stage 1 truth here on the forum. I don't think that would be against any of the rules, and it would remove any disadvantage for people who don't have it (and make the LB a complete mess, but I can't do anything about that).", "votes": null }, { "id": "178739", "postDate": "04/28/2017 19:09:41", "content": "If you have multiple accounts you can score 0 even with a single submission.
", "rawMarkdown": "If you have multiple accounts you can score 0 even with a single submission.", "votes": null }, { "id": "179534", "postDate": "05/01/2017 22:31:06", "content": "It's Tuesday...feel free to release the LB ground truths haha
", "rawMarkdown": "It's Tuesday...feel free to release the LB ground truths haha", "votes": null }, { "id": "179587", "postDate": "05/02/2017 06:15:47", "content": "Don't do this man! It's enough data for training. Don't need huge mess in LB.
", "rawMarkdown": "Don't do this man! It's enough data for training. Don't need huge mess in LB.", "votes": null }, { "id": "182179", "postDate": "05/12/2017 16:13:24", "content": "I am wondering why Kaggle admins are not taking any action aginst leader board mining. This happened in the DSB competition as well and it really reduces the excitement and fun part of compitition.
", "rawMarkdown": "I am wondering why Kaggle admins are not taking any action aginst leader board mining. This happened in the DSB competition as well and it really reduces the excitement and fun part of compitition.", "votes": null } ], "comments": [ { "id": 178588, "author_name": "skhemka", "author_url": "", "post_date": "04/28/2017 09:01:26", "content": "Andrei,\nPossibly it is a case of leader board mining. See here: https://www.kaggle.com/olegtrott/data-science-bowl-2017/the-perfect-score-script
", "votes": null, "replies": [ { "id": 178600, "author_name": "mikhailtrofimov", "author_url": "", "post_date": "04/28/2017 10:12:43", "content": "But 3 submissions are too few for this
", "votes": null, "replies": [] }, { "id": 178685, "author_name": "titericz", "author_url": "", "post_date": "04/28/2017 16:45:10", "content": "Actually he took 2 submission to probe LB and the 3th to submit the best score. 2 Subs to probe 512 records of 3 classes labels and taking into account Kaggle round scores to 5 digits?!? Impressive. I would say impossible before saw it.
", "votes": null, "replies": [] }, { "id": 178691, "author_name": "mumech", "author_url": "", "post_date": "04/28/2017 16:55:58", "content": "Please tell me you are kidding XD
\n\nIt's not possible to extract more than 7 truth values per submission. There's a few people who have gone through the trouble of doing so (me included), and this is just the first account that was created just to post that 0 LB entry.
\n\nIf the competition maintainers don't want the LB to become completely unusable, they should delete this account (breaking site rules, multiple accounts or sharing data outside of the forum).
\n\nIf they don't care, then I'll post the full truth values in a couple days so everyone can benefit from them :-)
", "votes": null, "replies": [] }, { "id": 178712, "author_name": "arukavina", "author_url": "", "post_date": "04/28/2017 18:17:39", "content": "Thanks r4m0n for taking the time to explain all this. I found one of your other posts here: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/30150#167944 talking about this same thing.
\n\nJust to clarify, is this a 2-stage competition? If so, this won't be a problem in second stage, but it clearly demotivates the rest of us Machine Learning mortals.
\n\nThe account is new, that's correct.
", "votes": null, "replies": [] }, { "id": 178718, "author_name": "mumech", "author_url": "", "post_date": "04/28/2017 18:25:14", "content": "Yes, this is a 2-stage competition and yes, people having the stage 1 truth and 0 LB scores won't cause any problems for the second stage. All that's going to happen is that the stage 1 LB is going to become full of meaningless scores.
\n\nFor anyone competing, don't worry about this, and just don't take any unreasonably good LB scores seriously, they aren't. And just to make it clear, once you have the stage 1 truth values, you can make ANY score on the LB. Right now only the 0 LB score is unreasonable, but don't be discouraged when 0.4's and 0.3's start popping up.
", "votes": null, "replies": [] }, { "id": 178739, "author_name": "sakvaua", "author_url": "", "post_date": "04/28/2017 19:09:41", "content": "If you have multiple accounts you can score 0 even with a single submission.
", "votes": null, "replies": [] } ] }, { "id": 178627, "author_name": "gotohell1", "author_url": "", "post_date": "04/28/2017 13:00:45", "content": "If someone brute force and get perfect score, will he/she will be eligible for proce?
", "votes": null, "replies": [ { "id": 178660, "author_name": "kingburrito666", "author_url": "", "post_date": "04/28/2017 14:59:10", "content": "of course not
", "votes": null, "replies": [] }, { "id": 178717, "author_name": "arukavina", "author_url": "", "post_date": "04/28/2017 18:23:10", "content": "I'd like to understand why this is not braking the site rules? and how the site admins are going to react to this kind of cheats? (Only if its one, of course).
", "votes": null, "replies": [] }, { "id": 178719, "author_name": "mumech", "author_url": "", "post_date": "04/28/2017 18:29:31", "content": "Mining the stage 1 truth values doesn't break any rules, creating a second account or sending the truth values to someone else does. On the DSB the organizers let the original poster of the 0 LB score stay (Oleg's) and mostly kept the LB clean by deleting any other submissions. On this one the first 0 LB poster is obviously breaking the rules, and I expect the organizers to be harsher.
", "votes": null, "replies": [] } ] }, { "id": 178659, "author_name": "kingburrito666", "author_url": "", "post_date": "04/28/2017 14:58:59", "content": "haha, Have you heard? If ever someone gets a perfect loss. They have overfitted.
", "votes": null, "replies": [] }, { "id": 178686, "author_name": "happycube", "author_url": "", "post_date": "04/28/2017 16:46:28", "content": "It's impressive, but balanced out by this being a 2-stage competition.
", "votes": null, "replies": [] }, { "id": 178735, "author_name": "sakvaua", "author_url": "", "post_date": "04/28/2017 19:04:38", "content": "Actually, I find this leaderboard mining very wrong and it should be banned by Kaggle as it effectively increases train set size for those who mined the leaderboard, giving them an unfair advantage in tuning model parameters with full dataset. There's no need to explain how you got your parameters and having an extra 30% dataset can be very beneficial. 2-stage competition model does not eliminate this advantage in any way.
", "votes": null, "replies": [ { "id": 178738, "author_name": "mumech", "author_url": "", "post_date": "04/28/2017 19:09:37", "content": "In the DSB the stage 1 truth was released in the stage 2 to be used to train your model and keep things fair, and I fully expect them to do the same here. I haven't yet heard from any of the organizers here about this and I sure hope they make their position clear here in the next few days.
\n\nIf they don't make their position clear until, say, Tuesday, I'll make the point moot by just releasing the full stage 1 truth here on the forum. I don't think that would be against any of the rules, and it would remove any disadvantage for people who don't have it (and make the LB a complete mess, but I can't do anything about that).
", "votes": null, "replies": [] }, { "id": 179534, "author_name": "kubilai", "author_url": "", "post_date": "05/01/2017 22:31:06", "content": "It's Tuesday...feel free to release the LB ground truths haha
", "votes": null, "replies": [] }, { "id": 179587, "author_name": "victorsd", "author_url": "", "post_date": "05/02/2017 06:15:47", "content": "Don't do this man! It's enough data for training. Don't need huge mess in LB.
", "votes": null, "replies": [] } ] }, { "id": 182179, "author_name": "svesal", "author_url": "", "post_date": "05/12/2017 16:13:24", "content": "I am wondering why Kaggle admins are not taking any action aginst leader board mining. This happened in the DSB competition as well and it really reduces the excitement and fun part of compitition.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "178546": "Should we all quit or is something wrong with top 1? Is results mining allowed? \nAnyways, congrats to Carlos Grajeda if with only 3 submissions he was able to reach the top!", "178588": "Andrei,\nPossibly it is a case of leader board mining. See here: https://www.kaggle.com/olegtrott/data-science-bowl-2017/the-perfect-score-script", "178600": "But 3 submissions are too few for this", "178627": "If someone brute force and get perfect score, will he/she will be eligible for proce?", "178659": "haha, Have you heard? If ever someone gets a perfect loss. They have overfitted.", "178660": "of course not", "178685": "Actually he took 2 submission to probe LB and the 3th to submit the best score. 2 Subs to probe 512 records of 3 classes labels and taking into account Kaggle round scores to 5 digits?!? Impressive. I would say impossible before saw it.", "178686": "It's impressive, but balanced out by this being a 2-stage competition.", "178691": "Please tell me you are kidding XD\n\nIt's not possible to extract more than 7 truth values per submission. There's a few people who have gone through the trouble of doing so (me included), and this is just the first account that was created just to post that 0 LB entry.\n\nIf the competition maintainers don't want the LB to become completely unusable, they should delete this account (breaking site rules, multiple accounts or sharing data outside of the forum).\n\nIf they don't care, then I'll post the full truth values in a couple days so everyone can benefit from them :-)", "178712": "Thanks r4m0n for taking the time to explain all this. I found one of your other posts here: https://www.kaggle.com/c/intel-mobileodt-cervical-cancer-screening/discussion/30150#167944 talking about this same thing.\n\nJust to clarify, is this a 2-stage competition? If so, this won't be a problem in second stage, but it clearly demotivates the rest of us Machine Learning mortals.\n \nThe account is new, that's correct.", "178717": "I'd like to understand why this is not braking the site rules? and how the site admins are going to react to this kind of cheats? (Only if its one, of course).", "178718": "Yes, this is a 2-stage competition and yes, people having the stage 1 truth and 0 LB scores won't cause any problems for the second stage. All that's going to happen is that the stage 1 LB is going to become full of meaningless scores.\n\nFor anyone competing, don't worry about this, and just don't take any unreasonably good LB scores seriously, they aren't. And just to make it clear, once you have the stage 1 truth values, you can make ANY score on the LB. Right now only the 0 LB score is unreasonable, but don't be discouraged when 0.4's and 0.3's start popping up.", "178719": "Mining the stage 1 truth values doesn't break any rules, creating a second account or sending the truth values to someone else does. On the DSB the organizers let the original poster of the 0 LB score stay (Oleg's) and mostly kept the LB clean by deleting any other submissions. On this one the first 0 LB poster is obviously breaking the rules, and I expect the organizers to be harsher.", "178735": "Actually, I find this leaderboard mining very wrong and it should be banned by Kaggle as it effectively increases train set size for those who mined the leaderboard, giving them an unfair advantage in tuning model parameters with full dataset. There's no need to explain how you got your parameters and having an extra 30% dataset can be very beneficial. 2-stage competition model does not eliminate this advantage in any way.", "178738": "In the DSB the stage 1 truth was released in the stage 2 to be used to train your model and keep things fair, and I fully expect them to do the same here. I haven't yet heard from any of the organizers here about this and I sure hope they make their position clear here in the next few days.\n\nIf they don't make their position clear until, say, Tuesday, I'll make the point moot by just releasing the full stage 1 truth here on the forum. I don't think that would be against any of the rules, and it would remove any disadvantage for people who don't have it (and make the LB a complete mess, but I can't do anything about that).", "178739": "If you have multiple accounts you can score 0 even with a single submission.", "179534": "It's Tuesday...feel free to release the LB ground truths haha", "179587": "Don't do this man! It's enough data for training. Don't need huge mess in LB.", "182179": "I am wondering why Kaggle admins are not taking any action aginst leader board mining. This happened in the DSB competition as well and it really reduces the excitement and fun part of compitition." }, "source": "meta" }