\n
![\"image\"](\"https://i.imgur.com/sSBHouy.png\")
@philippsinger @ilu000 @christofhenkel Please 🙏, tell us its not another leak 😬.
", "messages": [ { "id": 1405155, "postDate": "2021-07-30T14:31:49.423Z", "content": "@philippsinger @ilu000 @christofhenkel Please 🙏, tell us its not another leak 😬.
", "rawMarkdown": "@philippsinger @ilu000 @christofhenkel Please 🙏, tell us its not another leak 😬.", "votes": 48 }, { "id": 1405581, "postDate": "2021-07-31T04:16:30.480Z", "content": "Well, we may think some possibility to get higher score.
\nModel Ensemble training,
\nTTA at inference,
\nPostprocessing,
\nChange brightness level at the inference,
\nVision transform,
\nFinding additional signal pattern using classical feature like Hog,
\nChange stride in conv_stem from (2,2) to (1,1) (it doesn't work for me)
… There are many possibility.
\nI am trying to check these things but it need large time to explore.
0.80 is not strict upper bound. I think test image is very simple,
\nand they just continuously have unseen pattern.
\nNueral network can predict other 0.20 part easily, but It reduce entire score.
So I think find a class of test_images and
\nfind ensemble constant for each class (like stacking) is also important.
@woosungyoon I am learning quite a lot from your comments. Do you mind elaborating on the last point, finding a new class of test images?
", "rawMarkdown": "@woosungyoon I am learning quite a lot from your comments. Do you mind elaborating on the last point, finding a new class of test images?", "votes": 2 } ] }, { "id": 1405195, "postDate": "2021-07-30T15:15:32.783Z", "content": "Oops, lag others huge behind, looks the big 3 found the secret of CV/LB gap, hope it`s not a leak.
", "rawMarkdown": "Oops, lag others huge behind, looks the big 3 found the secret of CV/LB gap, hope it`s not a leak. ", "votes": 4, "replies": [ { "id": 1405282, "postDate": "2021-07-30T17:03:05.393Z", "content": "They might have just found the extra class we snuck in test. I'm not worried.
", "rawMarkdown": "They might have just found the extra class we snuck in test. I'm not worried.", "votes": 20 }, { "id": 1405330, "postDate": "2021-07-30T18:03:20.690Z", "content": "Great, now it`s becoming very interesting 😃
", "rawMarkdown": "Great, now it`s becoming very interesting 😃" }, { "id": 1405357, "postDate": "2021-07-30T18:43:14.107Z", "content": "@yuhongc would you please clarify what do you mean by the extra class?
", "rawMarkdown": "@yuhongc would you please clarify what do you mean by the extra class?" }, { "id": 1405367, "postDate": "2021-07-30T18:55:29.540Z", "content": "There are different alien patterns in the data. Test set probably has unseen patterns. I was guessing that but now that the organizers told it, it is no more fun.
", "rawMarkdown": "There are different alien patterns in the data. Test set probably has unseen patterns. I was guessing that but now that the organizers told it, it is no more fun.", "votes": 10 }, { "id": 1405732, "postDate": "2021-07-31T06:50:16.900Z", "content": "Sorry I spoiled it for you, didn't want people worrying about another reset
", "rawMarkdown": "Sorry I spoiled it for you, didn't want people worrying about another reset ", "votes": 2 }, { "id": 1406088, "postDate": "2021-07-31T13:38:31.890Z", "content": "It was the most probable explanation of the large CV LB gap.
", "rawMarkdown": "It was the most probable explanation of the large CV LB gap.", "votes": 4 }, { "id": 1406095, "postDate": "2021-07-31T13:51:11.807Z", "content": "\n\nIt was the most probable explanation of the large CV LB gap.
\n
Most probable explanation was the test set being more noisy for me since people were reporting 99% adversarial validation AUC.
", "rawMarkdown": "> It was the most probable explanation of the large CV LB gap.\n\nMost probable explanation was the test set being more noisy for me since people were reporting 99% adversarial validation AUC.", "votes": -1 }, { "id": 1406100, "postDate": "2021-07-31T13:58:30.033Z", "content": "This is also true.
", "rawMarkdown": "This is also true.", "votes": 4 }, { "id": 1406130, "postDate": "2021-07-31T14:31:15.980Z", "content": "I am fairly sure there is more to the train/test difference than a new type of signal. Below are randomly chosen samples from train and test, respectively. Both visualized with exactly the same parameters. Bottom line: test looks qualitatively different from train. Less \"wild\" and more \"gaussian\".
\n
I think there is result of using different hyperparameters of generation algorithm for test and train.
", "rawMarkdown": "I think there is result of using different hyperparameters of generation algorithm for test and train.", "votes": 9 } ] }, { "id": 1405238, "postDate": "2021-07-30T16:05:06.667Z", "content": "They probably already regret submitting. Unless they had to submit:)
", "rawMarkdown": "They probably already regret submitting. Unless they had to submit:)", "votes": 1, "replies": [ { "id": 1405253, "postDate": "2021-07-30T16:29:52.613Z", "content": "If it`s another leak I believe they will reveal soon, if not maybe some special kind of preprocess/augmentation(or other magic recipe) is the key for the new data, hope the latter is correct and make the competition more interesting.
", "rawMarkdown": "If it`s another leak I believe they will reveal soon, if not maybe some special kind of preprocess/augmentation(or other magic recipe) is the key for the new data, hope the latter is correct and make the competition more interesting.", "votes": 1 }, { "id": 1480370, "postDate": "2021-08-19T02:03:29.227Z", "content": "\n\nThey probably already regret submitting.
\n
This was actually true :D
", "rawMarkdown": "> They probably already regret submitting.\n\nThis was actually true :D", "votes": 1 } ] } ], "comments": [ { "id": 1405581, "author_name": "WOOSUNG YOON", "author_url": "", "post_date": "2021-07-31T04:16:30.480000", "content": "Well, we may think some possibility to get higher score.
\nModel Ensemble training,
\nTTA at inference,
\nPostprocessing,
\nChange brightness level at the inference,
\nVision transform,
\nFinding additional signal pattern using classical feature like Hog,
\nChange stride in conv_stem from (2,2) to (1,1) (it doesn't work for me)
… There are many possibility.
\nI am trying to check these things but it need large time to explore.
0.80 is not strict upper bound. I think test image is very simple,
\nand they just continuously have unseen pattern.
\nNueral network can predict other 0.20 part easily, but It reduce entire score.
So I think find a class of test_images and
\nfind ensemble constant for each class (like stacking) is also important.
@woosungyoon I am learning quite a lot from your comments. Do you mind elaborating on the last point, finding a new class of test images?
", "votes": 2, "replies": [] } ] }, { "id": 1405195, "author_name": "Hao", "author_url": "", "post_date": "2021-07-30T15:15:32.783000", "content": "Oops, lag others huge behind, looks the big 3 found the secret of CV/LB gap, hope it`s not a leak.
", "votes": 4, "replies": [ { "id": 1405282, "author_name": "Yuhong Chen", "author_url": "", "post_date": "2021-07-30T17:03:05.393000", "content": "They might have just found the extra class we snuck in test. I'm not worried.
", "votes": 20, "replies": [] }, { "id": 1405330, "author_name": "Hao", "author_url": "", "post_date": "2021-07-30T18:03:20.690000", "content": "Great, now it`s becoming very interesting 😃
", "votes": 0, "replies": [] }, { "id": 1405357, "author_name": "Mohammed Sabry", "author_url": "", "post_date": "2021-07-30T18:43:14.107000", "content": "@yuhongc would you please clarify what do you mean by the extra class?
", "votes": 0, "replies": [] }, { "id": 1405367, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2021-07-30T18:55:29.540000", "content": "There are different alien patterns in the data. Test set probably has unseen patterns. I was guessing that but now that the organizers told it, it is no more fun.
", "votes": 10, "replies": [] }, { "id": 1405732, "author_name": "Yuhong Chen", "author_url": "", "post_date": "2021-07-31T06:50:16.900000", "content": "Sorry I spoiled it for you, didn't want people worrying about another reset
", "votes": 2, "replies": [] }, { "id": 1406088, "author_name": "CPMP", "author_url": "", "post_date": "2021-07-31T13:38:31.890000", "content": "It was the most probable explanation of the large CV LB gap.
", "votes": 4, "replies": [] }, { "id": 1406095, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2021-07-31T13:51:11.807000", "content": "\n\nIt was the most probable explanation of the large CV LB gap.
\n
Most probable explanation was the test set being more noisy for me since people were reporting 99% adversarial validation AUC.
", "votes": -1, "replies": [] }, { "id": 1406100, "author_name": "CPMP", "author_url": "", "post_date": "2021-07-31T13:58:30.033000", "content": "This is also true.
", "votes": 4, "replies": [] }, { "id": 1406130, "author_name": "Markus Frank", "author_url": "", "post_date": "2021-07-31T14:31:15.980000", "content": "I am fairly sure there is more to the train/test difference than a new type of signal. Below are randomly chosen samples from train and test, respectively. Both visualized with exactly the same parameters. Bottom line: test looks qualitatively different from train. Less \"wild\" and more \"gaussian\".
\n
I think there is result of using different hyperparameters of generation algorithm for test and train.
", "votes": 9, "replies": [] } ] }, { "id": 1405238, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2021-07-30T16:05:06.667000", "content": "They probably already regret submitting. Unless they had to submit:)
", "votes": 1, "replies": [ { "id": 1405253, "author_name": "Hao", "author_url": "", "post_date": "2021-07-30T16:29:52.613000", "content": "If it`s another leak I believe they will reveal soon, if not maybe some special kind of preprocess/augmentation(or other magic recipe) is the key for the new data, hope the latter is correct and make the competition more interesting.
", "votes": 1, "replies": [] }, { "id": 1480370, "author_name": "Dieter", "author_url": "", "post_date": "2021-08-19T02:03:29.227000", "content": "\n\nThey probably already regret submitting.
\n
This was actually true :D
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1405155": "@philippsinger @ilu000 @christofhenkel Please 🙏, tell us its not another leak 😬.", "1405581": "Well, we may think some possibility to get higher score.\nModel Ensemble training, \nTTA at inference, \nPostprocessing,\nChange brightness level at the inference, \nVision transform,\nFinding additional signal pattern using classical feature like Hog,\nChange stride in conv_stem from (2,2) to (1,1) (it doesn't work for me)\n\n... There are many possibility.\nI am trying to check these things but it need large time to explore.\n\n0.80 is not strict upper bound. I think test image is very simple,\nand they just continuously have unseen pattern.\nNueral network can predict other 0.20 part easily, but It reduce entire score.\n\nSo I think find a class of test_images and\nfind ensemble constant for each class (like stacking) is also important.\n", "1405195": "Oops, lag others huge behind, looks the big 3 found the secret of CV/LB gap, hope it`s not a leak. ", "1405238": "They probably already regret submitting. Unless they had to submit:)" } }