{ "id": 268351, "title": "Wow public LB 1.0!!! ", "url": "/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/268351", "author_name": "KhanhVD", "post_date": "2021-08-27T03:21:13.227000", "votes": 25, "comment_count": 33, "views": 0, "content": "

i see ONODERA got a 1.0 on public LB, curious to see if hand labeling or leaked data or something magically here :))

", "messages": [ { "id": 1492270, "postDate": "2021-08-27T03:21:13.227Z", "content": "

i see ONODERA got a 1.0 on public LB, curious to see if hand labeling or leaked data or something magically here :))

", "rawMarkdown": "i see ONODERA got a 1.0 on public LB, curious to see if hand labeling or leaked data or something magically here :))", "votes": 24 }, { "id": 1492357, "postDate": "2021-08-27T05:39:33.840Z", "content": "

This is result of LB probing. So my private LB is 0.5.

", "rawMarkdown": "This is result of LB probing. So my private LB is 0.5.", "votes": 16, "replies": [ { "id": 1492376, "postDate": "2021-08-27T06:09:27.250Z", "content": "

I asked a question regarding this matter in another thread, but that thread is quite old so let me ask again. From the rules it seems we are not allowed to probe the public lb

\n
\n

Imaging studies used in this Challenge have been contributed by several sources and are de-identified patient data, meaning that the contributing institutions have taken reasonable care to remove from them all personally identifiable information. As a participant in the Competition, you agree 1) not to make copies of, or in any way redistribute, any of the data made available to you, 2) not to attempt to re-identify any personal information based on the data, 3) not to attempt to probe the test set's labels, and 4) to notify the Competition organizers of any personally identifiable information you encounter in the data by posting a message to the Kaggle community discussion forums.

\n
\n

On the other hand, many teams had probed the lb and it seems okay to do it. So… are we allowed to probe the lb or not?

", "rawMarkdown": "I asked a question regarding this matter in [another thread](https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/256706), but that thread is quite old so let me ask again. From the rules it seems we are not allowed to probe the public lb\n> Imaging studies used in this Challenge have been contributed by several sources and are de-identified patient data, meaning that the contributing institutions have taken reasonable care to remove from them all personally identifiable information. As a participant in the Competition, you agree 1) not to make copies of, or in any way redistribute, any of the data made available to you, 2) not to attempt to re-identify any personal information based on the data, **3) not to attempt to probe the test set's labels,** and 4) to notify the Competition organizers of any personally identifiable information you encounter in the data by posting a message to the Kaggle community discussion forums.\n\nOn the other hand, many teams had probed the lb and it seems okay to do it. So... are we allowed to probe the lb or not?", "votes": 2 }, { "id": 1492385, "postDate": "2021-08-27T06:21:36.073Z", "content": "

thx for your reply. random metric ranking of competition is ridiculous.

", "rawMarkdown": "thx for your reply. random metric ranking of competition is ridiculous.", "votes": 1 }, { "id": 1492407, "postDate": "2021-08-27T06:41:12.817Z", "content": "
\n

3) not to attempt to probe the test set's labels

\n
\n

I overlooked this part. But this is still ambiguous. If we cant use LB feedback, the host shouldn't have distributed 87 public test samples. Actually my best public score was 0.523 until yesterday. I'm just wondering how to check those who are hiding score.

", "rawMarkdown": "> 3) not to attempt to probe the test set's labels\n\nI overlooked this part. But this is still ambiguous. If we cant use LB feedback, the host shouldn't have distributed 87 public test samples. Actually my best public score was 0.523 until yesterday. I'm just wondering how to check those who are hiding score.", "votes": 3 }, { "id": 1492843, "postDate": "2021-08-27T13:11:19.120Z", "content": "

Please add [HL] to the beginning of your team name so that others on the LB know that you hand labelled. This will help preserve the integrity of the LB to some extent.

\n

@onodera

", "rawMarkdown": "Please add [HL] to the beginning of your team name so that others on the LB know that you hand labelled. This will help preserve the integrity of the LB to some extent.\n\n@onodera ", "votes": 8 }, { "id": 1495503, "postDate": "2021-08-29T15:43:59.167Z", "content": "

Will there be an official position of the Kaggle? @juliaelliott

", "rawMarkdown": "Will there be an official position of the Kaggle? @juliaelliott " } ] }, { "id": 1492913, "postDate": "2021-08-27T14:13:30.813Z", "content": "

I guess it's just a matter of time before the public test targets are available to everyone, @onodera are you willing to be the first to disclose them?

", "rawMarkdown": "I guess it's just a matter of time before the public test targets are available to everyone, @onodera are you willing to be the first to disclose them?", "votes": 7 }, { "id": 1494679, "postDate": "2021-08-29T00:45:31.270Z", "content": "

The rules also says

\n
\n

Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

\n
\n

So now there are no benefit to probe public test. You just don't need to submit anymore.

", "rawMarkdown": "The rules also says\n> Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.\n\nSo now there are no benefit to probe public test. You just don't need to submit anymore.", "votes": 3 }, { "id": 1492535, "postDate": "2021-08-27T08:37:44.663Z", "content": "

\"yNlQWRM\"
\n@onodera

", "rawMarkdown": "\"yNlQWRM\"\n@onodera ", "votes": 3 }, { "id": 1492788, "postDate": "2021-08-27T12:31:56.250Z", "content": "

See here for the full discussion on this topic.

\n

https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/256706

", "rawMarkdown": "See here for the full discussion on this topic.\n\nhttps://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/256706", "votes": 4, "replies": [ { "id": 1492833, "postDate": "2021-08-27T13:04:46.580Z", "content": "

I had not noticed that the top ten rankers in the public LB must present their methods and results. I am not sure what the participants of MICCAI will take from the presentation of LB hacking technique😅.

", "rawMarkdown": "I had not noticed that [the top ten rankers in the public LB must present their methods and results](https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/254252). I am not sure what the participants of MICCAI will take from the presentation of LB hacking technique😅.", "votes": 2 }, { "id": 1492842, "postDate": "2021-08-27T13:11:03.690Z", "content": "

Now I regret submitting perfect score

", "rawMarkdown": "Now I regret submitting perfect score", "votes": 4 }, { "id": 1494494, "postDate": "2021-08-28T17:50:23.247Z", "content": "

Hey guys, Darien, Onodera, @dschettler8845 @onodera

\n

This isn't working for me. For training, when I set all values to a constant, (0, 0.5, 1) and then change 1 row at a time to either 0 or 1, the AUC will change as mentioned to ~0.502, 0.498).

\n

But for the public leaderboard when I do this, many times the score stays at 0.5 when I set all values to 0 and change 1 row to 1.

\n

Any thoughts and did you guys or anyone else encounter this?

\n

Thank you so much for your help.

", "rawMarkdown": "Hey guys, Darien, Onodera, @dschettler8845 @onodera\n\nThis isn't working for me. For training, when I set all values to a constant, (0, 0.5, 1) and then change 1 row at a time to either 0 or 1, the AUC will change as mentioned to ~0.502, 0.498).\n\nBut for the public leaderboard when I do this, many times the score stays at 0.5 when I set all values to 0 and change 1 row to 1. \n\nAny thoughts and did you guys or anyone else encounter this?\n\nThank you so much for your help. " }, { "id": 1494503, "postDate": "2021-08-28T18:01:59.203Z", "content": "

@bopengiowa make your probing kernel public. I would find out the bug

", "rawMarkdown": "@bopengiowa make your probing kernel public. I would find out the bug", "votes": 2 }, { "id": 1494515, "postDate": "2021-08-28T18:21:55.637Z", "content": "

Onodera @onodera,

\n

Thanks for your great help. Here is the notebook: https://www.kaggle.com/bopengiowa/public-lb-1-row-submission

\n

I just set all rows to 1, and changed row 3 to 0. And the AUC for this notebook was 0.5

\n

But for training dataset, when I set all rows to 1 and set one row to 0, the AUC always changes to ~0.498, or ~0.502.

\n

Thank you for your great help Onodera and congrats on your incredible Kaggle competitions record!

", "rawMarkdown": "Onodera @onodera,\n\nThanks for your great help. Here is the notebook: https://www.kaggle.com/bopengiowa/public-lb-1-row-submission\n\nI just set all rows to 1, and changed row 3 to 0. And the AUC for this notebook was 0.5 \n\nBut for training dataset, when I set all rows to 1 and set one row to 0, the AUC always changes to ~0.498, or ~0.502. \n\nThank you for your great help Onodera and congrats on your incredible Kaggle competitions record!", "votes": 1 }, { "id": 1494666, "postDate": "2021-08-28T23:38:45.573Z", "content": "

Use BraTS21ID instead of index. Because we don't know private sample_submission.csv order.

\n
\n

sample_submission['MGMT_value'][2] = 0

\n
\n

Do you think private sample_submission['MGMT_value'][2] means same BraTS21ID as public? I don't think so. That's why your probing are wrong.

", "rawMarkdown": "Use BraTS21ID instead of index. Because we don't know private sample_submission.csv order.\n> sample_submission['MGMT_value'][2] = 0\n\nDo you think private **sample_submission['MGMT_value'][2]** means same BraTS21ID as public? I don't think so. That's why your probing are wrong.", "votes": 3 }, { "id": 1495604, "postDate": "2021-08-29T16:45:42.290Z", "content": "

Hey Bo Peng @bopengiowa,

\n

I have a similar issue and don't quite follow onodera's last reply

\n

I get 0.488 auc on public submission when I change the first(BraTS21ID=1) MGMT_value to 1, but thereafter I get 0.500 auc for individual changes to the 2nd and 3rd row with ID13 and ID15 respectively(even after resetting the entire df mgmt values to 0 prior to the individual change).

\n

Thanks for your time and patience, sorry if I'm missing something obvious(super possible)

", "rawMarkdown": "Hey Bo Peng @bopengiowa,\n\nI have a similar issue and don't quite follow onodera's last reply\n\n I get 0.488 auc on public submission when I change the first(BraTS21ID=1) MGMT_value to 1, but thereafter I get 0.500 auc for individual changes to the 2nd and 3rd row with ID13 and ID15 respectively(even after resetting the entire df mgmt values to 0 prior to the individual change).\n\nThanks for your time and patience, sorry if I'm missing something obvious(super possible)" }, { "id": 1498299, "postDate": "2021-08-31T20:55:10.513Z", "content": "

Hi Onodera @onodera,

\n

Thanks my friend. That makes sense and solved it. Appreciate your kindness and help.

\n

@clarksaben you can use this line:

\n
sample_submission['MGMT_value'] = 1\nsample_submission['MGMT_value'][sample_submission['BraTS21ID'] == 1] = 0\n
", "rawMarkdown": "Hi Onodera @onodera,\n\nThanks my friend. That makes sense and solved it. Appreciate your kindness and help.\n\n@clarksaben you can use this line: \n\n```\nsample_submission['MGMT_value'] = 1\nsample_submission['MGMT_value'][sample_submission['BraTS21ID'] == 1] = 0\n```\n", "votes": 1 }, { "id": 1498319, "postDate": "2021-08-31T21:32:23.093Z", "content": "

Hi @bopengiowa ,

\n

This did the trick, you are a legend -> Thank you !

", "rawMarkdown": "Hi @bopengiowa ,\n\nThis did the trick, you are a legend -> Thank you !", "votes": 1 } ] }, { "id": 1493394, "postDate": "2021-08-27T20:51:09.700Z", "content": "

I got a mini-heart attack :D

", "rawMarkdown": "I got a mini-heart attack :D", "votes": 2 }, { "id": 1492290, "postDate": "2021-08-27T04:19:15.427Z", "content": "

maybe hand labelling:v

", "rawMarkdown": "maybe hand labelling:v", "votes": 2, "replies": [ { "id": 1492300, "postDate": "2021-08-27T04:35:35.053Z", "content": "

or he is a magician 🤔

", "rawMarkdown": "or he is a magician 🤔" }, { "id": 1492313, "postDate": "2021-08-27T04:48:38.050Z", "content": "

No, ROC-AUC is very difficult to equal 1.0

", "rawMarkdown": "No, ROC-AUC is very difficult to equal 1.0", "votes": 1 }, { "id": 1492347, "postDate": "2021-08-27T05:22:12.903Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1492350, "postDate": "2021-08-27T05:31:06.970Z", "content": "

Seems like a leak
\nhttps://twitter.com/0verfit/status/1431116580679675904?s=20

", "rawMarkdown": "Seems like a leak\nhttps://twitter.com/0verfit/status/1431116580679675904?s=20", "votes": 1, "replies": [ { "id": 1492785, "postDate": "2021-08-27T12:30:26.080Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1493178, "postDate": "2021-08-27T17:14:09.713Z", "content": "

Hi, @narek1110 ,could you please explain actually what is happening, I am getting confused and losing motivation to work for this competetion.

", "rawMarkdown": "Hi, @narek1110 ,could you please explain actually what is happening, I am getting confused and losing motivation to work for this competetion." }, { "id": 1494251, "postDate": "2021-08-28T14:22:59.880Z", "content": "

It's hand labelling. Using a probing technique you can discover the labels for the Public LB. If you so choose, at this point, you can submit those labels directly and you will score 100%. However, this in no way means that you will get 100% on the private LB.

", "rawMarkdown": "It's hand labelling. Using a probing technique you can discover the labels for the Public LB. If you so choose, at this point, you can submit those labels directly and you will score 100%. However, this in no way means that you will get 100% on the private LB.", "votes": 1 } ] }, { "id": 1494944, "postDate": "2021-08-29T07:30:20.553Z", "content": "

@onodera Good work. 👍

", "rawMarkdown": "@onodera Good work. 👍", "votes": -5 }, { "id": 1493980, "postDate": "2021-08-28T10:07:54.307Z", "content": "

Good One, Nice Work

", "rawMarkdown": "Good One, Nice Work", "votes": -1 }, { "id": 1494390, "postDate": "2021-08-28T16:22:52.180Z", "content": "

Onodera, do you know which test observations make up the public LB? Can you add these to the training dataset now? The problem with this competition seems to be the small training data size, maybe this could help.

", "rawMarkdown": "Onodera, do you know which test observations make up the public LB? Can you add these to the training dataset now? The problem with this competition seems to be the small training data size, maybe this could help.", "replies": [ { "id": 1494697, "postDate": "2021-08-29T01:47:02.853Z", "content": "

All samples in sample_submission.csv are public

", "rawMarkdown": "All samples in sample_submission.csv are public", "votes": 1 }, { "id": 1494710, "postDate": "2021-08-29T02:25:10.043Z", "content": "

Thanks, Onodera. To my second question, it looks like you clarified that hand labeled test data cannot be used in the submission. I think this would include labeling by probing. It seems the only safe way is to not include these 87 test observations in the training data for any final submissions to avoid running into trouble. Just thinking out loud. Interesting thread.

", "rawMarkdown": "Thanks, Onodera. To my second question, it looks like you clarified that hand labeled test data cannot be used in the submission. I think this would include labeling by probing. It seems the only safe way is to not include these 87 test observations in the training data for any final submissions to avoid running into trouble. Just thinking out loud. Interesting thread." } ] } ], "comments": [ { "id": 1492357, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-27T05:39:33.840000", "content": "

This is result of LB probing. So my private LB is 0.5.

", "votes": 16, "replies": [ { "id": 1492376, "author_name": "NaN", "author_url": "", "post_date": "2021-08-27T06:09:27.250000", "content": "

I asked a question regarding this matter in another thread, but that thread is quite old so let me ask again. From the rules it seems we are not allowed to probe the public lb

\n
\n

Imaging studies used in this Challenge have been contributed by several sources and are de-identified patient data, meaning that the contributing institutions have taken reasonable care to remove from them all personally identifiable information. As a participant in the Competition, you agree 1) not to make copies of, or in any way redistribute, any of the data made available to you, 2) not to attempt to re-identify any personal information based on the data, 3) not to attempt to probe the test set's labels, and 4) to notify the Competition organizers of any personally identifiable information you encounter in the data by posting a message to the Kaggle community discussion forums.

\n
\n

On the other hand, many teams had probed the lb and it seems okay to do it. So… are we allowed to probe the lb or not?

", "votes": 2, "replies": [] }, { "id": 1492385, "author_name": "dragon zhang", "author_url": "", "post_date": "2021-08-27T06:21:36.073000", "content": "

thx for your reply. random metric ranking of competition is ridiculous.

", "votes": 1, "replies": [] }, { "id": 1492407, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-27T06:41:12.817000", "content": "
\n

3) not to attempt to probe the test set's labels

\n
\n

I overlooked this part. But this is still ambiguous. If we cant use LB feedback, the host shouldn't have distributed 87 public test samples. Actually my best public score was 0.523 until yesterday. I'm just wondering how to check those who are hiding score.

", "votes": 3, "replies": [] }, { "id": 1492843, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-08-27T13:11:19.120000", "content": "

Please add [HL] to the beginning of your team name so that others on the LB know that you hand labelled. This will help preserve the integrity of the LB to some extent.

\n

@onodera

", "votes": 8, "replies": [] }, { "id": 1495503, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2021-08-29T15:43:59.167000", "content": "

Will there be an official position of the Kaggle? @juliaelliott

", "votes": 0, "replies": [] } ] }, { "id": 1492913, "author_name": "Optimo", "author_url": "", "post_date": "2021-08-27T14:13:30.813000", "content": "

I guess it's just a matter of time before the public test targets are available to everyone, @onodera are you willing to be the first to disclose them?

", "votes": 7, "replies": [] }, { "id": 1494679, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-29T00:45:31.270000", "content": "

The rules also says

\n
\n

Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

\n
\n

So now there are no benefit to probe public test. You just don't need to submit anymore.

", "votes": 3, "replies": [] }, { "id": 1492535, "author_name": "Theo Viel", "author_url": "", "post_date": "2021-08-27T08:37:44.663000", "content": "

\"yNlQWRM\"
\n@onodera

", "votes": 3, "replies": [] }, { "id": 1492788, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-08-27T12:31:56.250000", "content": "

See here for the full discussion on this topic.

\n

https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/256706

", "votes": 4, "replies": [ { "id": 1492833, "author_name": "tomoo inubushi", "author_url": "", "post_date": "2021-08-27T13:04:46.580000", "content": "

I had not noticed that the top ten rankers in the public LB must present their methods and results. I am not sure what the participants of MICCAI will take from the presentation of LB hacking technique😅.

", "votes": 2, "replies": [] }, { "id": 1492842, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-27T13:11:03.690000", "content": "

Now I regret submitting perfect score

", "votes": 4, "replies": [] }, { "id": 1494494, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-08-28T17:50:23.247000", "content": "

Hey guys, Darien, Onodera, @dschettler8845 @onodera

\n

This isn't working for me. For training, when I set all values to a constant, (0, 0.5, 1) and then change 1 row at a time to either 0 or 1, the AUC will change as mentioned to ~0.502, 0.498).

\n

But for the public leaderboard when I do this, many times the score stays at 0.5 when I set all values to 0 and change 1 row to 1.

\n

Any thoughts and did you guys or anyone else encounter this?

\n

Thank you so much for your help.

", "votes": 0, "replies": [] }, { "id": 1494503, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-28T18:01:59.203000", "content": "

@bopengiowa make your probing kernel public. I would find out the bug

", "votes": 2, "replies": [] }, { "id": 1494515, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-08-28T18:21:55.637000", "content": "

Onodera @onodera,

\n

Thanks for your great help. Here is the notebook: https://www.kaggle.com/bopengiowa/public-lb-1-row-submission

\n

I just set all rows to 1, and changed row 3 to 0. And the AUC for this notebook was 0.5

\n

But for training dataset, when I set all rows to 1 and set one row to 0, the AUC always changes to ~0.498, or ~0.502.

\n

Thank you for your great help Onodera and congrats on your incredible Kaggle competitions record!

", "votes": 1, "replies": [] }, { "id": 1494666, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-28T23:38:45.573000", "content": "

Use BraTS21ID instead of index. Because we don't know private sample_submission.csv order.

\n
\n

sample_submission['MGMT_value'][2] = 0

\n
\n

Do you think private sample_submission['MGMT_value'][2] means same BraTS21ID as public? I don't think so. That's why your probing are wrong.

", "votes": 3, "replies": [] }, { "id": 1495604, "author_name": "arelius", "author_url": "", "post_date": "2021-08-29T16:45:42.290000", "content": "

Hey Bo Peng @bopengiowa,

\n

I have a similar issue and don't quite follow onodera's last reply

\n

I get 0.488 auc on public submission when I change the first(BraTS21ID=1) MGMT_value to 1, but thereafter I get 0.500 auc for individual changes to the 2nd and 3rd row with ID13 and ID15 respectively(even after resetting the entire df mgmt values to 0 prior to the individual change).

\n

Thanks for your time and patience, sorry if I'm missing something obvious(super possible)

", "votes": 0, "replies": [] }, { "id": 1498299, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-08-31T20:55:10.513000", "content": "

Hi Onodera @onodera,

\n

Thanks my friend. That makes sense and solved it. Appreciate your kindness and help.

\n

@clarksaben you can use this line:

\n
sample_submission['MGMT_value'] = 1\nsample_submission['MGMT_value'][sample_submission['BraTS21ID'] == 1] = 0\n
", "votes": 1, "replies": [] }, { "id": 1498319, "author_name": "arelius", "author_url": "", "post_date": "2021-08-31T21:32:23.093000", "content": "

Hi @bopengiowa ,

\n

This did the trick, you are a legend -> Thank you !

", "votes": 1, "replies": [] } ] }, { "id": 1493394, "author_name": "Ayush Thakur", "author_url": "", "post_date": "2021-08-27T20:51:09.700000", "content": "

I got a mini-heart attack :D

", "votes": 2, "replies": [] }, { "id": 1492290, "author_name": "ThuyenNH", "author_url": "", "post_date": "2021-08-27T04:19:15.427000", "content": "

maybe hand labelling:v

", "votes": 2, "replies": [ { "id": 1492300, "author_name": "Georgii Sirotenko", "author_url": "", "post_date": "2021-08-27T04:35:35.053000", "content": "

or he is a magician 🤔

", "votes": 0, "replies": [] }, { "id": 1492313, "author_name": "ThuyenNH", "author_url": "", "post_date": "2021-08-27T04:48:38.050000", "content": "

No, ROC-AUC is very difficult to equal 1.0

", "votes": 1, "replies": [] }, { "id": 1492347, "author_name": "", "author_url": "", "post_date": "2021-08-27T05:22:12.903000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1492350, "author_name": "Narek Maloyan", "author_url": "", "post_date": "2021-08-27T05:31:06.970000", "content": "

Seems like a leak
\nhttps://twitter.com/0verfit/status/1431116580679675904?s=20

", "votes": 1, "replies": [ { "id": 1492785, "author_name": "", "author_url": "", "post_date": "2021-08-27T12:30:26.080000", "content": "", "votes": 0, "replies": [] }, { "id": 1493178, "author_name": "Wfarzana", "author_url": "", "post_date": "2021-08-27T17:14:09.713000", "content": "

Hi, @narek1110 ,could you please explain actually what is happening, I am getting confused and losing motivation to work for this competetion.

", "votes": 0, "replies": [] }, { "id": 1494251, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-08-28T14:22:59.880000", "content": "

It's hand labelling. Using a probing technique you can discover the labels for the Public LB. If you so choose, at this point, you can submit those labels directly and you will score 100%. However, this in no way means that you will get 100% on the private LB.

", "votes": 1, "replies": [] } ] }, { "id": 1494944, "author_name": "Susant_Achary🎲", "author_url": "", "post_date": "2021-08-29T07:30:20.553000", "content": "

@onodera Good work. 👍

", "votes": -5, "replies": [] }, { "id": 1493980, "author_name": "Tushar Sharma", "author_url": "", "post_date": "2021-08-28T10:07:54.307000", "content": "

Good One, Nice Work

", "votes": -1, "replies": [] }, { "id": 1494390, "author_name": "Andy Atkinson", "author_url": "", "post_date": "2021-08-28T16:22:52.180000", "content": "

Onodera, do you know which test observations make up the public LB? Can you add these to the training dataset now? The problem with this competition seems to be the small training data size, maybe this could help.

", "votes": 0, "replies": [ { "id": 1494697, "author_name": "ONODERA", "author_url": "", "post_date": "2021-08-29T01:47:02.853000", "content": "

All samples in sample_submission.csv are public

", "votes": 1, "replies": [] }, { "id": 1494710, "author_name": "Andy Atkinson", "author_url": "", "post_date": "2021-08-29T02:25:10.043000", "content": "

Thanks, Onodera. To my second question, it looks like you clarified that hand labeled test data cannot be used in the submission. I think this would include labeling by probing. It seems the only safe way is to not include these 87 test observations in the training data for any final submissions to avoid running into trouble. Just thinking out loud. Interesting thread.

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1492270": "i see ONODERA got a 1.0 on public LB, curious to see if hand labeling or leaked data or something magically here :))", "1492357": "This is result of LB probing. So my private LB is 0.5.", "1492913": "I guess it's just a matter of time before the public test targets are available to everyone, @onodera are you willing to be the first to disclose them?", "1494679": "The rules also says\n> Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.\n\nSo now there are no benefit to probe public test. You just don't need to submit anymore.", "1492535": "\"yNlQWRM\"\n@onodera ", "1492788": "See here for the full discussion on this topic.\n\nhttps://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/256706", "1493394": "I got a mini-heart attack :D", "1492290": "maybe hand labelling:v", "1492350": "Seems like a leak\nhttps://twitter.com/0verfit/status/1431116580679675904?s=20", "1494944": "@onodera Good work. 👍", "1493980": "Good One, Nice Work", "1494390": "Onodera, do you know which test observations make up the public LB? Can you add these to the training dataset now? The problem with this competition seems to be the small training data size, maybe this could help." } }