I'm relying on the Retrieval track leaderboard to test out my models, and to ensure that my local validation / training pipeline is correct. However, today the retrieval track submission got disabled. Is it a technical glitch by Kaggle, or the organizers of this competition has disabled it?
\nFor many folks who haven't participated in the Retrieval track or joined it late (I joined only 8 days prior to the competition deadline and didn't have a proper training pipeline), relying on the Retrieval LB is the most feasible way to compete with other teams who have the advantage of having a good global descriptor from the previous competition. I believe that disabling it makes the gap between those who joined late and those who participated in the previous competition larger.
\nUpdate: you can use this colab notebook to download the test set to your GCS bucket.
", "messages": [ { "id": "997192", "postDate": "09/03/2020 20:03:28", "content": "I'm relying on the Retrieval track leaderboard to test out my models, and to ensure that my local validation / training pipeline is correct. However, today the retrieval track submission got disabled. Is it a technical glitch by Kaggle, or the organizers of this competition has disabled it?
\nFor many folks who haven't participated in the Retrieval track or joined it late (I joined only 8 days prior to the competition deadline and didn't have a proper training pipeline), relying on the Retrieval LB is the most feasible way to compete with other teams who have the advantage of having a good global descriptor from the previous competition. I believe that disabling it makes the gap between those who joined late and those who participated in the previous competition larger.
\nUpdate: you can use this colab notebook to download the test set to your GCS bucket.
", "rawMarkdown": "I'm relying on the Retrieval track leaderboard to test out my models, and to ensure that my local validation / training pipeline is correct. However, today the retrieval track submission got disabled. Is it a technical glitch by Kaggle, or the organizers of this competition has disabled it?\n\nFor many folks who haven't participated in the Retrieval track or joined it late (I joined only 8 days prior to the competition deadline and didn't have a proper training pipeline), relying on the Retrieval LB is the most feasible way to compete with other teams who have the advantage of having a good global descriptor from the previous competition. I believe that disabling it makes the gap between those who joined late and those who participated in the previous competition larger.\n\n--------------------------------\n\n**Update:** you can use [this colab notebook](https://www.kaggle.com/chankhavu/export-gldv2-full-test-set-to-your-own-gcs-bucket) to download the test set to your GCS bucket.", "votes": null }, { "id": "997291", "postDate": "09/03/2020 22:57:58", "content": "Yes, we did disable late submissions to the retrieval track. We don't want people to continue overfitting on that dataset because we may want to reuse it later.
\nOur recommendation is to use the public GLDv2 in order to validate your models. All labels and metric computation code are public.
", "rawMarkdown": "Yes, we did disable late submissions to the retrieval track. We don't want people to continue overfitting on that dataset because we may want to reuse it later.\n\nOur recommendation is to use the public [GLDv2](https://github.com/cvdfoundation/google-landmark) in order to validate your models. All labels and metric computation code are public.", "votes": null }, { "id": "997296", "postDate": "09/03/2020 23:08:00", "content": "Thanks for the quick confirmation!
", "rawMarkdown": "Thanks for the quick confirmation!", "votes": null }, { "id": "997667", "postDate": "09/04/2020 06:51:40", "content": "@andrefaraujo Are the test images here taken from the test set you linked as well?
", "rawMarkdown": "andrefaraujo Are the test images here taken from the test set you linked as well?", "votes": null }, { "id": "998160", "postDate": "09/04/2020 14:35:49", "content": "Is it possible to enable the late submission with only showing the public LB score and hide the private one? Local validation is hard for many participants who have lack of GPU/TPU resources as the training actually requires a long time for such a big dataset. It would be good if we can use the submission system to validate our models.
", "rawMarkdown": "Is it possible to enable the late submission with only showing the public LB score and hide the private one? Local validation is hard for many participants who have lack of GPU/TPU resources as the training actually requires a long time for such a big dataset. It would be good if we can use the submission system to validate our models.", "votes": null }, { "id": "998632", "postDate": "09/04/2020 22:24:02", "content": "Yeah, would be really nice if they can make late submission only on public LB. The recognition track is much harder than the retrieval one, and with the late submission for Retrieval track available for global descriptor validation, participants in Recognition track can save time on validation and be more creative in their re-ranking approaches.
", "rawMarkdown": "Yeah, would be really nice if they can make late submission only on public LB. The recognition track is much harder than the retrieval one, and with the late submission for Retrieval track available for global descriptor validation, participants in Recognition track can save time on validation and be more creative in their re-ranking approaches.", "votes": null }, { "id": "1029946", "postDate": "09/28/2020 09:41:40", "content": "@andrefaraujo Can you please open the late submission for the Retrieval Track after the competition ends (at least for a few days)? I really want to test out the global model that I trained for the Recognition track, as I did not had a chance to properly participate in the Retrieval competition.
", "rawMarkdown": "andrefaraujo Can you please open the late submission for the Retrieval Track after the competition ends (at least for a few days)? I really want to test out the global model that I trained for the Recognition track, as I did not had a chance to properly participate in the Retrieval competition.", "votes": null }, { "id": "1031283", "postDate": "09/29/2020 11:09:00", "content": "@css919, you could use the Roxford and the Paris dataset for validation. It took my RTX2070 to finish eval on the Roxford in like 3min (single scale effcientnetb6).
", "rawMarkdown": "css919, you could use the Roxford and the Paris dataset for validation. It took my RTX2070 to finish eval on the Roxford in like 3min (single scale effcientnetb6).", "votes": null } ], "comments": [ { "id": 997291, "author_name": "andrefaraujo", "author_url": "", "post_date": "09/03/2020 22:57:58", "content": "Yes, we did disable late submissions to the retrieval track. We don't want people to continue overfitting on that dataset because we may want to reuse it later.
\nOur recommendation is to use the public GLDv2 in order to validate your models. All labels and metric computation code are public.
", "votes": null, "replies": [ { "id": 997296, "author_name": "chankhavu", "author_url": "", "post_date": "09/03/2020 23:08:00", "content": "Thanks for the quick confirmation!
", "votes": null, "replies": [] }, { "id": 997667, "author_name": "usmannkhan", "author_url": "", "post_date": "09/04/2020 06:51:40", "content": "@andrefaraujo Are the test images here taken from the test set you linked as well?
", "votes": null, "replies": [] }, { "id": 998160, "author_name": "css919", "author_url": "", "post_date": "09/04/2020 14:35:49", "content": "Is it possible to enable the late submission with only showing the public LB score and hide the private one? Local validation is hard for many participants who have lack of GPU/TPU resources as the training actually requires a long time for such a big dataset. It would be good if we can use the submission system to validate our models.
", "votes": null, "replies": [] }, { "id": 998632, "author_name": "chankhavu", "author_url": "", "post_date": "09/04/2020 22:24:02", "content": "Yeah, would be really nice if they can make late submission only on public LB. The recognition track is much harder than the retrieval one, and with the late submission for Retrieval track available for global descriptor validation, participants in Recognition track can save time on validation and be more creative in their re-ranking approaches.
", "votes": null, "replies": [] }, { "id": 1029946, "author_name": "chankhavu", "author_url": "", "post_date": "09/28/2020 09:41:40", "content": "@andrefaraujo Can you please open the late submission for the Retrieval Track after the competition ends (at least for a few days)? I really want to test out the global model that I trained for the Recognition track, as I did not had a chance to properly participate in the Retrieval competition.
", "votes": null, "replies": [] }, { "id": 1031283, "author_name": "doncalculator", "author_url": "", "post_date": "09/29/2020 11:09:00", "content": "@css919, you could use the Roxford and the Paris dataset for validation. It took my RTX2070 to finish eval on the Roxford in like 3min (single scale effcientnetb6).
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "997192": "I'm relying on the Retrieval track leaderboard to test out my models, and to ensure that my local validation / training pipeline is correct. However, today the retrieval track submission got disabled. Is it a technical glitch by Kaggle, or the organizers of this competition has disabled it?\n\nFor many folks who haven't participated in the Retrieval track or joined it late (I joined only 8 days prior to the competition deadline and didn't have a proper training pipeline), relying on the Retrieval LB is the most feasible way to compete with other teams who have the advantage of having a good global descriptor from the previous competition. I believe that disabling it makes the gap between those who joined late and those who participated in the previous competition larger.\n\n--------------------------------\n\n**Update:** you can use [this colab notebook](https://www.kaggle.com/chankhavu/export-gldv2-full-test-set-to-your-own-gcs-bucket) to download the test set to your GCS bucket.", "997291": "Yes, we did disable late submissions to the retrieval track. We don't want people to continue overfitting on that dataset because we may want to reuse it later.\n\nOur recommendation is to use the public [GLDv2](https://github.com/cvdfoundation/google-landmark) in order to validate your models. All labels and metric computation code are public.", "997296": "Thanks for the quick confirmation!", "997667": "andrefaraujo Are the test images here taken from the test set you linked as well?", "998160": "Is it possible to enable the late submission with only showing the public LB score and hide the private one? Local validation is hard for many participants who have lack of GPU/TPU resources as the training actually requires a long time for such a big dataset. It would be good if we can use the submission system to validate our models.", "998632": "Yeah, would be really nice if they can make late submission only on public LB. The recognition track is much harder than the retrieval one, and with the late submission for Retrieval track available for global descriptor validation, participants in Recognition track can save time on validation and be more creative in their re-ranking approaches.", "1029946": "andrefaraujo Can you please open the late submission for the Retrieval Track after the competition ends (at least for a few days)? I really want to test out the global model that I trained for the Recognition track, as I did not had a chance to properly participate in the Retrieval competition.", "1031283": "css919, you could use the Roxford and the Paris dataset for validation. It took my RTX2070 to finish eval on the Roxford in like 3min (single scale effcientnetb6)." }, "source": "meta" }