According to this really cool kernel, 72.770% of the public test images has the size of 640x480. Is this ratio the same in the private test set? The answer to this question would give us some clues about how much difference there is between the public test set and the private test set. In this kernel, inspired by this discussion and this kernel, the image size distribution of public + private test set will be probed.
\n\nApproach:\nLet's assume that we have N-(submission.csv, public LB score) pairs from already submitted kernels. By changing the submission file according to the desired information about the private test set, we can get log(N) bits information from one submission. Here \"change submission\" means to copy the targets of the public test set from the known submissions and insert arbitrary dummy targets to the remaining private test set. By doing so, we can control the public LB score according to what we want to know.
\n\nAnswer:\nThe answer is no. While the ratio of 640x480 images in the public test set is 72.770%, the ratio of 640x480 images in the public + private test set is 30-40%. There are some differences between the public test set and the private test set. I hope this information would be useful in choosing the final submission(s).
\n\nPlease refer to this kernel for details.
", "messages": [ { "id": "602034", "postDate": "08/18/2019 13:32:22", "content": "According to this really cool kernel, 72.770% of the public test images has the size of 640x480. Is this ratio the same in the private test set? The answer to this question would give us some clues about how much difference there is between the public test set and the private test set. In this kernel, inspired by this discussion and this kernel, the image size distribution of public + private test set will be probed.
\n\nApproach:\nLet's assume that we have N-(submission.csv, public LB score) pairs from already submitted kernels. By changing the submission file according to the desired information about the private test set, we can get log(N) bits information from one submission. Here \"change submission\" means to copy the targets of the public test set from the known submissions and insert arbitrary dummy targets to the remaining private test set. By doing so, we can control the public LB score according to what we want to know.
\n\nAnswer:\nThe answer is no. While the ratio of 640x480 images in the public test set is 72.770%, the ratio of 640x480 images in the public + private test set is 30-40%. There are some differences between the public test set and the private test set. I hope this information would be useful in choosing the final submission(s).
\n\nPlease refer to this kernel for details.
", "rawMarkdown": "According to [this really cool kernel](https://www.kaggle.com/fhopfmueller/removing-unwanted-correlations-in-training-public), 72.770% of the public test images has the size of 640x480. Is this ratio the same in the private test set? The answer to this question would give us some clues about how much difference there is between the public test set and the private test set. In this kernel, inspired by [this discussion](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97652#latest-596913) and [this kernel](https://www.kaggle.com/cdeotte/private-lb-probing-0-950), the image size distribution of public + private test set will be probed.\n\nApproach:\nLet's assume that we have N-(submission.csv, public LB score) pairs from already submitted kernels. By changing the submission file according to the desired information about the private test set, we can get log(N) bits information from one submission. Here \"change submission\" means to copy the targets of the public test set from the known submissions and insert arbitrary dummy targets to the remaining private test set. By doing so, we can control the public LB score according to what we want to know.\n\nAnswer:\nThe answer is no. While the ratio of 640x480 images in the public test set is 72.770%, the ratio of 640x480 images in the public + private test set is 30-40%. There are some differences between the public test set and the private test set. I hope this information would be useful in choosing the final submission(s).\n\nPlease refer to [this kernel](https://www.kaggle.com/ren4yu/aptos-2019-probing-private-test-image-sizes) for details.", "votes": null }, { "id": "602187", "postDate": "08/18/2019 18:17:52", "content": "That's really useful! Thank you.
", "rawMarkdown": "That's really useful! Thank you.", "votes": null }, { "id": "602697", "postDate": "08/19/2019 11:55:44", "content": "Interesting kernel! Thanks for sharing! I have seen a thread where the test images are indeed very different from the training set. You can even spot the difference by eye. This has been a weird competition so far. Curious to see how the top 10 at the end of the competition approached this!
", "rawMarkdown": "Interesting kernel! Thanks for sharing! I have seen a thread where the test images are indeed very different from the training set. You can even spot the difference by eye. This has been a weird competition so far. Curious to see how the top 10 at the end of the competition approached this!", "votes": null }, { "id": "603412", "postDate": "08/20/2019 08:26:55", "content": "Thanks for sharing these good kernels! Really helps me.
", "rawMarkdown": "Thanks for sharing these good kernels! Really helps me.", "votes": null }, { "id": "603729", "postDate": "08/20/2019 15:48:19", "content": "Thanks for sharing
", "rawMarkdown": "Thanks for sharing", "votes": null } ], "comments": [ { "id": 602187, "author_name": "seesee", "author_url": "", "post_date": "08/18/2019 18:17:52", "content": "That's really useful! Thank you.
", "votes": null, "replies": [] }, { "id": 602697, "author_name": "carlolepelaars", "author_url": "", "post_date": "08/19/2019 11:55:44", "content": "Interesting kernel! Thanks for sharing! I have seen a thread where the test images are indeed very different from the training set. You can even spot the difference by eye. This has been a weird competition so far. Curious to see how the top 10 at the end of the competition approached this!
", "votes": null, "replies": [] }, { "id": 603412, "author_name": "creacyhuang", "author_url": "", "post_date": "08/20/2019 08:26:55", "content": "Thanks for sharing these good kernels! Really helps me.
", "votes": null, "replies": [] }, { "id": 603729, "author_name": "", "author_url": "", "post_date": "08/20/2019 15:48:19", "content": "Thanks for sharing
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "602034": "According to [this really cool kernel](https://www.kaggle.com/fhopfmueller/removing-unwanted-correlations-in-training-public), 72.770% of the public test images has the size of 640x480. Is this ratio the same in the private test set? The answer to this question would give us some clues about how much difference there is between the public test set and the private test set. In this kernel, inspired by [this discussion](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97652#latest-596913) and [this kernel](https://www.kaggle.com/cdeotte/private-lb-probing-0-950), the image size distribution of public + private test set will be probed.\n\nApproach:\nLet's assume that we have N-(submission.csv, public LB score) pairs from already submitted kernels. By changing the submission file according to the desired information about the private test set, we can get log(N) bits information from one submission. Here \"change submission\" means to copy the targets of the public test set from the known submissions and insert arbitrary dummy targets to the remaining private test set. By doing so, we can control the public LB score according to what we want to know.\n\nAnswer:\nThe answer is no. While the ratio of 640x480 images in the public test set is 72.770%, the ratio of 640x480 images in the public + private test set is 30-40%. There are some differences between the public test set and the private test set. I hope this information would be useful in choosing the final submission(s).\n\nPlease refer to [this kernel](https://www.kaggle.com/ren4yu/aptos-2019-probing-private-test-image-sizes) for details.", "602187": "That's really useful! Thank you.", "602697": "Interesting kernel! Thanks for sharing! I have seen a thread where the test images are indeed very different from the training set. You can even spot the difference by eye. This has been a weird competition so far. Curious to see how the top 10 at the end of the competition approached this!", "603412": "Thanks for sharing these good kernels! Really helps me.", "603729": "Thanks for sharing" }, "source": "meta" }