![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1025985%2F64e3ed2b7ddef59dce0ee1ad8fab4333%2F2019-09-26%201.02.47.png?generation=1569430996030723&alt=media\")
I love Kaggle messages, always so optimistic! It must know something, I will keep trying.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2243477%2F9195cbf731db7384b9a6f1f12b91a95a%2F1000.png?generation=1569438445331930&alt=media\")
As @devvindan correctly noted, it is an LB probing. Our true score is 0.986, we will share more details as the competition ends. Good luck!
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2243477%2Fadc7a9685a4b7b88f929d71964dd5a88%2Fpuzzle-says-3-5-years-only-took-1-hour-memeful-com-3929326.png?generation=1569434066347309&alt=media\")
Thanks for the info! Looking forward to your sharing :)
", "rawMarkdown": "Thanks for the info! Looking forward to your sharing :)", "votes": 3 }, { "id": 634055, "postDate": "2019-09-25T19:11:54.363Z", "content": "And I am really looking forward for your solution, in fact, all top solutions must be super interesting. What are the details that make the difference between top10?
", "rawMarkdown": "And I am really looking forward for your solution, in fact, all top solutions must be super interesting. What are the details that make the difference between top10?" }, { "id": 634061, "postDate": "2019-09-25T19:25:25.113Z", "content": "You were playing a different game of getting #1 public in the earthquake competition, too 🤣
", "rawMarkdown": "You were playing a different game of getting #1 public in the earthquake competition, too 🤣", "votes": 2 }, { "id": 634069, "postDate": "2019-09-25T19:47:23.033Z", "content": "Yes, @junkoda , there is a post that I wrote about it, take a look if interesting.
", "rawMarkdown": "Yes, @junkoda , [there is a post](https://towardsdatascience.com/how-to-lb-probe-on-kaggle-c0aa21458bfe) that I wrote about it, take a look if interesting.", "votes": 1 }, { "id": 634148, "postDate": "2019-09-25T23:51:08.513Z", "content": "Interesting. Good luck.
", "rawMarkdown": "Interesting. Good luck.", "votes": 1 } ] }, { "id": 633998, "postDate": "2019-09-25T17:15:44.667Z", "content": "how can 1.0 be a leak and 0.99 not? 👀
", "rawMarkdown": "how can 1.0 be a leak and 0.99 not? 👀 ", "votes": 9 }, { "id": 633999, "postDate": "2019-09-25T17:22:43.203Z", "content": "322 submits is more like probing the LB than a leak
", "rawMarkdown": "322 submits is more like probing the LB than a leak", "votes": 4 }, { "id": 634075, "postDate": "2019-09-25T19:57:39.783Z", "content": "Wow. This is madness. Can’t wait to see what you guys did!
", "rawMarkdown": "Wow. This is madness. Can’t wait to see what you guys did!", "votes": 1 }, { "id": 634018, "postDate": "2019-09-25T17:39:51.317Z", "content": "maybe they are successful in finding the random state that shuffles the wells :D
", "rawMarkdown": "maybe they are successful in finding the random state that shuffles the wells :D", "votes": 1 }, { "id": 634005, "postDate": "2019-09-25T17:31:14.650Z", "content": "They officially have 4 more experiments with 100% confidence in the labels in the training set. Plus the HUVEC leak, its 5 experiments. While I still don't know why my model got U2OS-04 so wrong.
", "rawMarkdown": "They officially have 4 more experiments with 100% confidence in the labels in the training set. Plus the HUVEC leak, its 5 experiments. While I still don't know why my model got U2OS-04 so wrong.", "votes": 1, "replies": [ { "id": 634026, "postDate": "2019-09-25T17:50:42.527Z", "content": "Well U2OS-04 is the difficult one, the most difficult of all U2OS according to our estimation, and I think it is second in difficulty only to HUVEC-05. Good luck!
", "rawMarkdown": "Well U2OS-04 is the difficult one, the most difficult of all U2OS according to our estimation, and I think it is second in difficulty only to HUVEC-05. Good luck!", "votes": 1 } ] }, { "id": 634374, "postDate": "2019-09-26T07:46:07.583Z", "content": "and then Artificial General Intelligence was made. 😂
", "rawMarkdown": "and then Artificial General Intelligence was made. 😂 ", "votes": 2 }, { "id": 634059, "postDate": "2019-09-25T19:20:25.217Z", "content": "Does LB probing against the rule? \nDoes 1) prob a/some samples labels, 2) use these labels in next submissions mean “hand labelling”\n@sohier
", "rawMarkdown": "Does LB probing against the rule? \nDoes 1) prob a/some samples labels, 2) use these labels in next submissions mean “hand labelling”\n@sohier ", "votes": 2, "replies": [ { "id": 634097, "postDate": "2019-09-25T20:56:19.550Z", "content": "Interesting topic, a note here - everyone uses public LB as an additional validation set that guides model development. Some competition formats are better at preventing LB probing, specifically ones where one needs to submit a model (read a Kaggle kernel) which does not use external data.
", "rawMarkdown": "Interesting topic, a note here - everyone uses public LB as an additional validation set that guides model development. Some competition formats are better at preventing LB probing, specifically ones where one needs to submit a model (read a Kaggle kernel) which does not use external data.", "votes": 1 }, { "id": 634134, "postDate": "2019-09-25T23:01:26.110Z", "content": "Hi, thanks for answering. No offense, I just remember one competition rule that we cannot do hand labelling, thus post my questions here.
", "rawMarkdown": "Hi, thanks for answering. No offense, I just remember one competition rule that we cannot do hand labelling, thus post my questions here. ", "votes": 1 }, { "id": 634140, "postDate": "2019-09-25T23:19:41.547Z", "content": "With the model score in your guys' territory, pseudo labels on pb set are almost as good as ground truth...
", "rawMarkdown": "With the model score in your guys' territory, pseudo labels on pb set are almost as good as ground truth... " }, { "id": 634169, "postDate": "2019-09-26T01:20:27.163Z", "content": "Pseudo Labelling is quite different and safe, you don’t have to know the whole performance of the test set you predicted. Whereas, probing means you “take a look at” the ground truth. \nLet’s wait for the official response : )
", "rawMarkdown": "Pseudo Labelling is quite different and safe, you don’t have to know the whole performance of the test set you predicted. Whereas, probing means you “take a look at” the ground truth. \nLet’s wait for the official response : )" }, { "id": 634449, "postDate": "2019-09-26T10:05:04.743Z", "content": "LB probing is not hand labeling. Non of us can hand label these images, as we don't know how to look at the picture and decide which SiRNA it is. The LB probing was a kind of optimization problem (Which @zaharch is an expert on). The only issue with it is that you can't use these methods when you don't have feedback like the LB score.
", "rawMarkdown": "LB probing is not hand labeling. Non of us can hand label these images, as we don't know how to look at the picture and decide which SiRNA it is. The LB probing was a kind of optimization problem (Which @zaharch is an expert on). The only issue with it is that you can't use these methods when you don't have feedback like the LB score. " } ] }, { "id": 634525, "postDate": "2019-09-26T12:06:02.907Z", "content": "I really support the public board thing to go away. Sure, it will be far less engaging, but much less problematic. And it will force people to really learn CV setups, instead of talking about it while the same time sucking out all the info from public board. It will probably also eliminate all the blenders too.
", "rawMarkdown": "I really support the public board thing to go away. Sure, it will be far less engaging, but much less problematic. And it will force people to really learn CV setups, instead of talking about it while the same time sucking out all the info from public board. It will probably also eliminate all the blenders too.", "votes": -2, "replies": [ { "id": 634641, "postDate": "2019-09-26T14:51:10.667Z", "content": "It is not about CV. It is about being able to estimate how good is your model compared to the best possible model.\nI think LB is very motivating. When you think you built the best possible model and scored 0.8, but then somebody comes and scores 0.95, you know you missed something.
\n\n(For this reason @zaharch , submitted this 1.0 only Yesterday, although he had this solution for some time - Not to confuse anybody)
", "rawMarkdown": "It is not about CV. It is about being able to estimate how good is your model compared to the best possible model.\nI think LB is very motivating. When you think you built the best possible model and scored 0.8, but then somebody comes and scores 0.95, you know you missed something.\n\n(For this reason @zaharch , submitted this 1.0 only Yesterday, although he had this solution for some time - Not to confuse anybody)", "votes": 1 }, { "id": 634696, "postDate": "2019-09-26T16:22:24.997Z", "content": "In that case, LB is still not a wonderful solution. There are many competitions where the public board is flat out useless.
\n\nMaybe its because I am not that competitive in these games, the longer I am on Kaggle, the less I feel motivated by the LB.
", "rawMarkdown": "In that case, LB is still not a wonderful solution. There are many competitions where the public board is flat out useless. \n\nMaybe its because I am not that competitive in these games, the longer I am on Kaggle, the less I feel motivated by the LB. " }, { "id": 634717, "postDate": "2019-09-26T16:57:39.227Z", "content": "\n\n\n(For this reason @zaharch , submitted this 1.0 only Yesterday, although he had this solution for some time - Not to confuse anybody)
\n
Tbh, I expected this coming. Still looking forward to learning from both the actual solution and how to probe LB like a pro! :)
", "rawMarkdown": "> (For this reason @zaharch , submitted this 1.0 only Yesterday, although he had this solution for some time - Not to confuse anybody)\n\nTbh, I expected this coming. Still looking forward to learning from both the actual solution and how to probe LB like a pro! :) " } ] }, { "id": 634295, "postDate": "2019-09-26T06:08:44.037Z", "content": "Great work!
", "rawMarkdown": "Great work!" }, { "id": 634241, "postDate": "2019-09-26T04:35:05.483Z", "content": "You guys are awesome!
", "rawMarkdown": "You guys are awesome!" }, { "id": 634085, "postDate": "2019-09-25T20:12:05.490Z", "content": "1.0 is really exciting!
\n\nLooking forward to see the private LB and approaches that were used for 1.0 score.
\n\nInteresting what was used - data leak or something smart with data!
\n\nThis biological competition seems to be really interesting:)
", "rawMarkdown": "**1.0 is really exciting!** \n\nLooking forward to see the private LB and approaches that were used for **1.0** score.\n\nInteresting what was used - data leak or something smart with data!\n\n**This biological competition seems to be really interesting:)**" }, { "id": 634011, "postDate": "2019-09-25T17:36:45.810Z", "content": "It's public LB, still not sure for private LB.
", "rawMarkdown": "It's public LB, still not sure for private LB.", "replies": [ { "id": 634013, "postDate": "2019-09-25T17:38:02.043Z", "content": "I am not expecting much a shakeup really.
", "rawMarkdown": "I am not expecting much a shakeup really. ", "votes": 1 }, { "id": 634030, "postDate": "2019-09-25T17:57:57.887Z", "content": "Agree, I don't expect much shakeup too. Besides us falling 7-8 places :)
", "rawMarkdown": "Agree, I don't expect much shakeup too. Besides us falling 7-8 places :)", "votes": 1 }, { "id": 634033, "postDate": "2019-09-25T18:05:29.437Z", "content": "I estimate 0.5-1.0% on private :)
", "rawMarkdown": "I estimate 0.5-1.0% on private :) ", "votes": 1 } ] }, { "id": 634095, "postDate": "2019-09-25T20:38:47.133Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 634052, "author_name": "nosound", "author_url": "", "post_date": "2019-09-25T19:07:51.840000", "content": "I love Kaggle messages, always so optimistic! It must know something, I will keep trying.
\n\n
As @devvindan correctly noted, it is an LB probing. Our true score is 0.986, we will share more details as the competition ends. Good luck!
\n\n
Thanks for the info! Looking forward to your sharing :)
", "votes": 3, "replies": [] }, { "id": 634055, "author_name": "nosound", "author_url": "", "post_date": "2019-09-25T19:11:54.363000", "content": "And I am really looking forward for your solution, in fact, all top solutions must be super interesting. What are the details that make the difference between top10?
", "votes": 0, "replies": [] }, { "id": 634061, "author_name": "🐢 Jun Koda", "author_url": "", "post_date": "2019-09-25T19:25:25.113000", "content": "You were playing a different game of getting #1 public in the earthquake competition, too 🤣
", "votes": 2, "replies": [] }, { "id": 634069, "author_name": "nosound", "author_url": "", "post_date": "2019-09-25T19:47:23.033000", "content": "Yes, @junkoda , there is a post that I wrote about it, take a look if interesting.
", "votes": 1, "replies": [] }, { "id": 634148, "author_name": "tascj", "author_url": "", "post_date": "2019-09-25T23:51:08.513000", "content": "Interesting. Good luck.
", "votes": 1, "replies": [] } ] }, { "id": 633998, "author_name": "Abhishek Thakur", "author_url": "", "post_date": "2019-09-25T17:15:44.667000", "content": "how can 1.0 be a leak and 0.99 not? 👀
", "votes": 9, "replies": [] }, { "id": 633999, "author_name": "Daniil Barysevich", "author_url": "", "post_date": "2019-09-25T17:22:43.203000", "content": "322 submits is more like probing the LB than a leak
", "votes": 4, "replies": [] }, { "id": 634075, "author_name": "interneuron", "author_url": "", "post_date": "2019-09-25T19:57:39.783000", "content": "Wow. This is madness. Can’t wait to see what you guys did!
", "votes": 1, "replies": [] }, { "id": 634018, "author_name": "cab", "author_url": "", "post_date": "2019-09-25T17:39:51.317000", "content": "maybe they are successful in finding the random state that shuffles the wells :D
", "votes": 1, "replies": [] }, { "id": 634005, "author_name": "Ren", "author_url": "", "post_date": "2019-09-25T17:31:14.650000", "content": "They officially have 4 more experiments with 100% confidence in the labels in the training set. Plus the HUVEC leak, its 5 experiments. While I still don't know why my model got U2OS-04 so wrong.
", "votes": 1, "replies": [ { "id": 634026, "author_name": "nosound", "author_url": "", "post_date": "2019-09-25T17:50:42.527000", "content": "Well U2OS-04 is the difficult one, the most difficult of all U2OS according to our estimation, and I think it is second in difficulty only to HUVEC-05. Good luck!
", "votes": 1, "replies": [] } ] }, { "id": 634374, "author_name": "torch", "author_url": "", "post_date": "2019-09-26T07:46:07.583000", "content": "and then Artificial General Intelligence was made. 😂
", "votes": 2, "replies": [] }, { "id": 634059, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-09-25T19:20:25.217000", "content": "Does LB probing against the rule? \nDoes 1) prob a/some samples labels, 2) use these labels in next submissions mean “hand labelling”\n@sohier
", "votes": 2, "replies": [ { "id": 634097, "author_name": "nosound", "author_url": "", "post_date": "2019-09-25T20:56:19.550000", "content": "Interesting topic, a note here - everyone uses public LB as an additional validation set that guides model development. Some competition formats are better at preventing LB probing, specifically ones where one needs to submit a model (read a Kaggle kernel) which does not use external data.
", "votes": 1, "replies": [] }, { "id": 634134, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-09-25T23:01:26.110000", "content": "Hi, thanks for answering. No offense, I just remember one competition rule that we cannot do hand labelling, thus post my questions here.
", "votes": 1, "replies": [] }, { "id": 634140, "author_name": "Ren", "author_url": "", "post_date": "2019-09-25T23:19:41.547000", "content": "With the model score in your guys' territory, pseudo labels on pb set are almost as good as ground truth...
", "votes": 0, "replies": [] }, { "id": 634169, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-09-26T01:20:27.163000", "content": "Pseudo Labelling is quite different and safe, you don’t have to know the whole performance of the test set you predicted. Whereas, probing means you “take a look at” the ground truth. \nLet’s wait for the official response : )
", "votes": 0, "replies": [] }, { "id": 634449, "author_name": "yuval reina", "author_url": "", "post_date": "2019-09-26T10:05:04.743000", "content": "LB probing is not hand labeling. Non of us can hand label these images, as we don't know how to look at the picture and decide which SiRNA it is. The LB probing was a kind of optimization problem (Which @zaharch is an expert on). The only issue with it is that you can't use these methods when you don't have feedback like the LB score.
", "votes": 0, "replies": [] } ] }, { "id": 634525, "author_name": "Ren", "author_url": "", "post_date": "2019-09-26T12:06:02.907000", "content": "I really support the public board thing to go away. Sure, it will be far less engaging, but much less problematic. And it will force people to really learn CV setups, instead of talking about it while the same time sucking out all the info from public board. It will probably also eliminate all the blenders too.
", "votes": -2, "replies": [ { "id": 634641, "author_name": "yuval reina", "author_url": "", "post_date": "2019-09-26T14:51:10.667000", "content": "It is not about CV. It is about being able to estimate how good is your model compared to the best possible model.\nI think LB is very motivating. When you think you built the best possible model and scored 0.8, but then somebody comes and scores 0.95, you know you missed something.
\n\n(For this reason @zaharch , submitted this 1.0 only Yesterday, although he had this solution for some time - Not to confuse anybody)
", "votes": 1, "replies": [] }, { "id": 634696, "author_name": "Ren", "author_url": "", "post_date": "2019-09-26T16:22:24.997000", "content": "In that case, LB is still not a wonderful solution. There are many competitions where the public board is flat out useless.
\n\nMaybe its because I am not that competitive in these games, the longer I am on Kaggle, the less I feel motivated by the LB.
", "votes": 0, "replies": [] }, { "id": 634717, "author_name": "wjsheng", "author_url": "", "post_date": "2019-09-26T16:57:39.227000", "content": "\n\n\n(For this reason @zaharch , submitted this 1.0 only Yesterday, although he had this solution for some time - Not to confuse anybody)
\n
Tbh, I expected this coming. Still looking forward to learning from both the actual solution and how to probe LB like a pro! :)
", "votes": 0, "replies": [] } ] }, { "id": 634295, "author_name": "Manoj Chandrashekar", "author_url": "", "post_date": "2019-09-26T06:08:44.037000", "content": "Great work!
", "votes": 0, "replies": [] }, { "id": 634241, "author_name": "Felipe Solares", "author_url": "", "post_date": "2019-09-26T04:35:05.483000", "content": "You guys are awesome!
", "votes": 0, "replies": [] }, { "id": 634085, "author_name": "grib0ed0v", "author_url": "", "post_date": "2019-09-25T20:12:05.490000", "content": "1.0 is really exciting!
\n\nLooking forward to see the private LB and approaches that were used for 1.0 score.
\n\nInteresting what was used - data leak or something smart with data!
\n\nThis biological competition seems to be really interesting:)
", "votes": 0, "replies": [] }, { "id": 634011, "author_name": "0xFunky", "author_url": "", "post_date": "2019-09-25T17:36:45.810000", "content": "It's public LB, still not sure for private LB.
", "votes": 0, "replies": [ { "id": 634013, "author_name": "Ren", "author_url": "", "post_date": "2019-09-25T17:38:02.043000", "content": "I am not expecting much a shakeup really.
", "votes": 1, "replies": [] }, { "id": 634030, "author_name": "nosound", "author_url": "", "post_date": "2019-09-25T17:57:57.887000", "content": "Agree, I don't expect much shakeup too. Besides us falling 7-8 places :)
", "votes": 1, "replies": [] }, { "id": 634033, "author_name": "cab", "author_url": "", "post_date": "2019-09-25T18:05:29.437000", "content": "I estimate 0.5-1.0% on private :)
", "votes": 1, "replies": [] } ] }, { "id": 634095, "author_name": "", "author_url": "", "post_date": "2019-09-25T20:38:47.133000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "634052": "I love Kaggle messages, always so optimistic! It must know something, I will keep trying.\n\n", "633994": "\n", "634027": "As @devvindan correctly noted, it is an LB probing. Our true score is 0.986, we will share more details as the competition ends. Good luck!\n\n\n", "633998": "how can 1.0 be a leak and 0.99 not? 👀 ", "633999": "322 submits is more like probing the LB than a leak", "634075": "Wow. This is madness. Can’t wait to see what you guys did!", "634018": "maybe they are successful in finding the random state that shuffles the wells :D", "634005": "They officially have 4 more experiments with 100% confidence in the labels in the training set. Plus the HUVEC leak, its 5 experiments. While I still don't know why my model got U2OS-04 so wrong.", "634374": "and then Artificial General Intelligence was made. 😂 ", "634059": "Does LB probing against the rule? \nDoes 1) prob a/some samples labels, 2) use these labels in next submissions mean “hand labelling”\n@sohier ", "634525": "I really support the public board thing to go away. Sure, it will be far less engaging, but much less problematic. And it will force people to really learn CV setups, instead of talking about it while the same time sucking out all the info from public board. It will probably also eliminate all the blenders too.", "634295": "Great work!", "634241": "You guys are awesome!", "634085": "**1.0 is really exciting!** \n\nLooking forward to see the private LB and approaches that were used for **1.0** score.\n\nInteresting what was used - data leak or something smart with data!\n\n**This biological competition seems to be really interesting:)**", "634011": "It's public LB, still not sure for private LB.", "634095": "" } }