Welcome to the third edition of the Landmark Recognition Challenge! This year, the challenge is a code competition, meaning that you will submit a Kaggle kernel that is used to compute your predictions instead of submitting a file with pre-computed predictions. You can develop and test this kernel using the public test and train data we provide. When you submit the kernel, it is evaluated on private test and train data and the resulting scores appear on the leaderboard. Note that the runtime limit for the kernels differs between development time and submission time: During development, kernels can run for 9 hours, while the submission kernel can run for 12 hours. This is meant to give you extra time for methods like spatial verification with local features.
\n\nWe hope you enjoy the challenge and are looking forward to see what solutions you come up with!
\n\nHave fun!
", "messages": [ { "id": 952100, "postDate": "2020-07-30T16:12:17.870Z", "content": "Welcome to the third edition of the Landmark Recognition Challenge! This year, the challenge is a code competition, meaning that you will submit a Kaggle kernel that is used to compute your predictions instead of submitting a file with pre-computed predictions. You can develop and test this kernel using the public test and train data we provide. When you submit the kernel, it is evaluated on private test and train data and the resulting scores appear on the leaderboard. Note that the runtime limit for the kernels differs between development time and submission time: During development, kernels can run for 9 hours, while the submission kernel can run for 12 hours. This is meant to give you extra time for methods like spatial verification with local features.
\n\nWe hope you enjoy the challenge and are looking forward to see what solutions you come up with!
\n\nHave fun!
", "rawMarkdown": "Welcome to the third edition of the Landmark Recognition Challenge! This year, the challenge is a code competition, meaning that you will submit a Kaggle kernel that is used to compute your predictions instead of submitting a file with pre-computed predictions. You can develop and test this kernel using the public test and train data we provide. When you submit the kernel, it is evaluated on private test and train data and the resulting scores appear on the leaderboard. Note that the runtime limit for the kernels differs between development time and submission time: During development, kernels can run for 9 hours, while the submission kernel can run for 12 hours. This is meant to give you extra time for methods like spatial verification with local features.\n\nWe hope you enjoy the challenge and are looking forward to see what solutions you come up with!\n\nHave fun!", "votes": 33 }, { "id": 975017, "postDate": "2020-08-18T05:46:12.953Z", "content": "Like retrieval competition, could you please release the scoring script for this competition as well? It'll help many of us to evaluate our models locally and save time. @andrefaraujo @camaskew @tobwey
", "rawMarkdown": "Like retrieval competition, could you please release the scoring script for this competition as well? It'll help many of us to evaluate our models locally and save time. @andrefaraujo @camaskew @tobwey ", "votes": 3, "replies": [ { "id": 976054, "postDate": "2020-08-18T15:55:44.283Z", "content": "We open sourced a tool to compute the metrics, which you can find here.
", "rawMarkdown": "We open sourced a tool to compute the metrics, which you can find [here](https://github.com/tensorflow/models/blob/master/research/delf/delf/python/google_landmarks_dataset/compute_recognition_metrics.py).", "votes": 3 } ] }, { "id": 952124, "postDate": "2020-07-30T16:29:51.297Z", "content": "welcome everyone!
", "rawMarkdown": "welcome everyone!", "votes": 4 }, { "id": 993281, "postDate": "2020-08-31T19:51:05.160Z", "content": "Hi @tobwey ,
\n\n\nYou may still attach the full training set as an external data set if you wish.
\n
Is it possible to provide the full training set as a kaggle dataset? Uploading the 100GB dataset again ourselves seems a bit counter intuitive and might not be possible given all the restrictions…
\nPS: Also unable to add data from other competitions (eg: Retrieval2020 has the same data which could be used :P )
", "rawMarkdown": "Hi @tobwey , \n\n> You may still attach the full training set as an external data set if you wish.\n\nIs it possible to provide the full training set as a kaggle dataset? Uploading the 100GB dataset again ourselves seems a bit counter intuitive and might not be possible given all the restrictions...\n\nPS: Also unable to add data from other competitions (eg: Retrieval2020 has the same data which could be used :P )", "votes": 1 }, { "id": 983167, "postDate": "2020-08-24T05:00:35.880Z", "content": "The Data section stated that \"The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. \". I suppose the \"private dataset\" here is the private test set which is used for the private LB scoring. May I know is (entire or part of or none of) public LB score based on the downloadable test set (i.e. the representative set of files)?
", "rawMarkdown": "The Data section stated that \"The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. \". I suppose the \"private dataset\" here is the private test set which is used for the private LB scoring. May I know is (entire or part of or none of) public LB score based on the downloadable test set (i.e. the representative set of files)?", "votes": 1, "replies": [ { "id": 983898, "postDate": "2020-08-24T17:08:37.347Z", "content": "I think the terminology is a bit confusing - sorry about that! When you create a submission, your kernel is run on the private test set, let's call it evaluation set. And the evaluation set itself is split into 33% \"public\" and 66% \"private\" sets. Your kernel's performance on the \"public\" part of the evaluation set is what appears on the public leaderboard.
", "rawMarkdown": "I think the terminology is a bit confusing - sorry about that! When you create a submission, your kernel is run on the private test set, let's call it evaluation set. And the evaluation set itself is split into 33% \"public\" and 66% \"private\" sets. Your kernel's performance on the \"public\" part of the evaluation set is what appears on the public leaderboard.", "votes": 3 } ] }, { "id": 953888, "postDate": "2020-08-01T08:09:12.057Z", "content": "Thanks for organising this competition. I enjoyed the 2018 version, so I'm looking forward to another go (sadly too busy in 2019).
\n\nAre you going to tell us anything about the \"Organizers Baseline\" that is currently top of the leaderboard?
", "rawMarkdown": "Thanks for organising this competition. I enjoyed the 2018 version, so I'm looking forward to another go (sadly too busy in 2019).\n\nAre you going to tell us anything about the \"Organizers Baseline\" that is currently top of the leaderboard?", "votes": 1, "replies": [ { "id": 954507, "postDate": "2020-08-01T19:32:22.480Z", "content": "Great to hear you're interested in participating again this year!
\n\nYes, we are working on publishing the kernel used in the baseline.
", "rawMarkdown": "Great to hear you're interested in participating again this year!\n\nYes, we are working on publishing the kernel used in the baseline.", "votes": 3 }, { "id": 958817, "postDate": "2020-08-05T06:43:53.510Z", "content": "Are you releasing the model weight as well? Given the awkward situation at the retrieval challenge, it may end up with early baseline submitters getting most of the points.
", "rawMarkdown": "Are you releasing the model weight as well? Given the awkward situation at the retrieval challenge, it may end up with early baseline submitters getting most of the points.", "votes": 1 }, { "id": 959459, "postDate": "2020-08-05T15:46:22.947Z", "content": "Yes, code and model weights will be released.
", "rawMarkdown": "Yes, code and model weights will be released.", "votes": -1 }, { "id": 959507, "postDate": "2020-08-05T16:25:20.293Z", "content": "We just released the kernel here.
", "rawMarkdown": "We just released the kernel [here](https://www.kaggle.com/c/landmark-recognition-2020/discussion/172576).", "votes": 3 } ] }, { "id": 985527, "postDate": "2020-08-25T19:31:04.467Z", "content": "Code Requirements says that
\nSo, I'm not sure whether one can use their own pre-computed embeddings if it's not publicly available. Could you clarify it please?
", "rawMarkdown": "Code Requirements says that\n\n- Freely & publicly available external data is allowed, including pre-trained models\n\nSo, I'm not sure whether one can use their own pre-computed embeddings if it's not publicly available. Could you clarify it please?", "votes": 2 }, { "id": 960838, "postDate": "2020-08-06T18:01:02.340Z", "content": "Thanks for the welcome message! Just a quick question: can you clarify this sentence please:
\n\n\n\n\nWhen you submit the kernel, it is evaluated on private test and train data
\n
What does evaluation on train data mean?
", "rawMarkdown": "Thanks for the welcome message! Just a quick question: can you clarify this sentence please: \n\n> When you submit the kernel, it is evaluated on private test and train data\n\nWhat does evaluation on train data mean? ", "votes": 2 }, { "id": 1009910, "postDate": "2020-09-14T11:08:57.217Z", "content": "Hi @tobwey,
\nCan you allow us to upload the features file? (embedding)
\nhost already wrote about the image file in the data section, but do not write about features.
Hi tereka! It's allowed to upload embeddings (see this post).
", "rawMarkdown": "Hi tereka! It's allowed to upload embeddings (see [this post](https://www.kaggle.com/c/landmark-recognition-2020/discussion/176697#993747)).", "votes": 1 }, { "id": 1010236, "postDate": "2020-09-14T15:46:49.630Z", "content": "Thank you!
", "rawMarkdown": "Thank you!" } ] }, { "id": 985651, "postDate": "2020-08-25T22:51:02.570Z", "content": "Nice and great!
", "rawMarkdown": "Nice and great!" }, { "id": 980961, "postDate": "2020-08-22T03:53:51.573Z", "content": "Hi @tobwey, is the private test equivalent to public test, in terms of number samples? (ie I'm just curious if it wont be 2x, 3x or more, it matters due to the time limit)
", "rawMarkdown": "Hi @tobwey, is the private test equivalent to public test, in terms of number samples? (ie I'm just curious if it wont be 2x, 3x or more, it matters due to the time limit)", "replies": [ { "id": 983904, "postDate": "2020-08-24T17:14:33.143Z", "content": "Yes, it has a similar number of images, so for the sake of runtime estimation you can treat them as equivalent.
", "rawMarkdown": "Yes, it has a similar number of images, so for the sake of runtime estimation you can treat them as equivalent.", "votes": 1 } ] }, { "id": 978337, "postDate": "2020-08-20T05:39:46.750Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 961387, "postDate": "2020-08-07T06:22:51.310Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 975017, "author_name": "Suvronil", "author_url": "", "post_date": "2020-08-18T05:46:12.953000", "content": "Like retrieval competition, could you please release the scoring script for this competition as well? It'll help many of us to evaluate our models locally and save time. @andrefaraujo @camaskew @tobwey
", "votes": 3, "replies": [ { "id": 976054, "author_name": "Tobias Weyand", "author_url": "", "post_date": "2020-08-18T15:55:44.283000", "content": "We open sourced a tool to compute the metrics, which you can find here.
", "votes": 3, "replies": [] } ] }, { "id": 952124, "author_name": "Andre Araujo", "author_url": "", "post_date": "2020-07-30T16:29:51.297000", "content": "welcome everyone!
", "votes": 4, "replies": [] }, { "id": 993281, "author_name": "hirviö", "author_url": "", "post_date": "2020-08-31T19:51:05.160000", "content": "Hi @tobwey ,
\n\n\nYou may still attach the full training set as an external data set if you wish.
\n
Is it possible to provide the full training set as a kaggle dataset? Uploading the 100GB dataset again ourselves seems a bit counter intuitive and might not be possible given all the restrictions…
\nPS: Also unable to add data from other competitions (eg: Retrieval2020 has the same data which could be used :P )
", "votes": 1, "replies": [] }, { "id": 983167, "author_name": "ShinSiang", "author_url": "", "post_date": "2020-08-24T05:00:35.880000", "content": "The Data section stated that \"The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. \". I suppose the \"private dataset\" here is the private test set which is used for the private LB scoring. May I know is (entire or part of or none of) public LB score based on the downloadable test set (i.e. the representative set of files)?
", "votes": 1, "replies": [ { "id": 983898, "author_name": "Tobias Weyand", "author_url": "", "post_date": "2020-08-24T17:08:37.347000", "content": "I think the terminology is a bit confusing - sorry about that! When you create a submission, your kernel is run on the private test set, let's call it evaluation set. And the evaluation set itself is split into 33% \"public\" and 66% \"private\" sets. Your kernel's performance on the \"public\" part of the evaluation set is what appears on the public leaderboard.
", "votes": 3, "replies": [] } ] }, { "id": 953888, "author_name": "Andy Penrose", "author_url": "", "post_date": "2020-08-01T08:09:12.057000", "content": "Thanks for organising this competition. I enjoyed the 2018 version, so I'm looking forward to another go (sadly too busy in 2019).
\n\nAre you going to tell us anything about the \"Organizers Baseline\" that is currently top of the leaderboard?
", "votes": 1, "replies": [ { "id": 954507, "author_name": "Tobias Weyand", "author_url": "", "post_date": "2020-08-01T19:32:22.480000", "content": "Great to hear you're interested in participating again this year!
\n\nYes, we are working on publishing the kernel used in the baseline.
", "votes": 3, "replies": [] }, { "id": 958817, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2020-08-05T06:43:53.510000", "content": "Are you releasing the model weight as well? Given the awkward situation at the retrieval challenge, it may end up with early baseline submitters getting most of the points.
", "votes": 1, "replies": [] }, { "id": 959459, "author_name": "Tobias Weyand", "author_url": "", "post_date": "2020-08-05T15:46:22.947000", "content": "Yes, code and model weights will be released.
", "votes": -1, "replies": [] }, { "id": 959507, "author_name": "Tobias Weyand", "author_url": "", "post_date": "2020-08-05T16:25:20.293000", "content": "We just released the kernel here.
", "votes": 3, "replies": [] } ] }, { "id": 985527, "author_name": "Yusuf Büyükdağ", "author_url": "", "post_date": "2020-08-25T19:31:04.467000", "content": "Code Requirements says that
\nSo, I'm not sure whether one can use their own pre-computed embeddings if it's not publicly available. Could you clarify it please?
", "votes": 2, "replies": [] }, { "id": 960838, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2020-08-06T18:01:02.340000", "content": "Thanks for the welcome message! Just a quick question: can you clarify this sentence please:
\n\n\n\n\nWhen you submit the kernel, it is evaluated on private test and train data
\n
What does evaluation on train data mean?
", "votes": 2, "replies": [] }, { "id": 1009910, "author_name": "tereka", "author_url": "", "post_date": "2020-09-14T11:08:57.217000", "content": "Hi @tobwey,
\nCan you allow us to upload the features file? (embedding)
\nhost already wrote about the image file in the data section, but do not write about features.
Hi tereka! It's allowed to upload embeddings (see this post).
", "votes": 1, "replies": [] }, { "id": 1010236, "author_name": "tereka", "author_url": "", "post_date": "2020-09-14T15:46:49.630000", "content": "Thank you!
", "votes": 0, "replies": [] } ] }, { "id": 985651, "author_name": "YASSYN IDAR", "author_url": "", "post_date": "2020-08-25T22:51:02.570000", "content": "Nice and great!
", "votes": 0, "replies": [] }, { "id": 980961, "author_name": "hirviö", "author_url": "", "post_date": "2020-08-22T03:53:51.573000", "content": "Hi @tobwey, is the private test equivalent to public test, in terms of number samples? (ie I'm just curious if it wont be 2x, 3x or more, it matters due to the time limit)
", "votes": 0, "replies": [ { "id": 983904, "author_name": "Tobias Weyand", "author_url": "", "post_date": "2020-08-24T17:14:33.143000", "content": "Yes, it has a similar number of images, so for the sake of runtime estimation you can treat them as equivalent.
", "votes": 1, "replies": [] } ] }, { "id": 978337, "author_name": "", "author_url": "", "post_date": "2020-08-20T05:39:46.750000", "content": "", "votes": 1, "replies": [] }, { "id": 961387, "author_name": "", "author_url": "", "post_date": "2020-08-07T06:22:51.310000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "952100": "Welcome to the third edition of the Landmark Recognition Challenge! This year, the challenge is a code competition, meaning that you will submit a Kaggle kernel that is used to compute your predictions instead of submitting a file with pre-computed predictions. You can develop and test this kernel using the public test and train data we provide. When you submit the kernel, it is evaluated on private test and train data and the resulting scores appear on the leaderboard. Note that the runtime limit for the kernels differs between development time and submission time: During development, kernels can run for 9 hours, while the submission kernel can run for 12 hours. This is meant to give you extra time for methods like spatial verification with local features.\n\nWe hope you enjoy the challenge and are looking forward to see what solutions you come up with!\n\nHave fun!", "975017": "Like retrieval competition, could you please release the scoring script for this competition as well? It'll help many of us to evaluate our models locally and save time. @andrefaraujo @camaskew @tobwey ", "952124": "welcome everyone!", "993281": "Hi @tobwey , \n\n> You may still attach the full training set as an external data set if you wish.\n\nIs it possible to provide the full training set as a kaggle dataset? Uploading the 100GB dataset again ourselves seems a bit counter intuitive and might not be possible given all the restrictions...\n\nPS: Also unable to add data from other competitions (eg: Retrieval2020 has the same data which could be used :P )", "983167": "The Data section stated that \"The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. \". I suppose the \"private dataset\" here is the private test set which is used for the private LB scoring. May I know is (entire or part of or none of) public LB score based on the downloadable test set (i.e. the representative set of files)?", "953888": "Thanks for organising this competition. I enjoyed the 2018 version, so I'm looking forward to another go (sadly too busy in 2019).\n\nAre you going to tell us anything about the \"Organizers Baseline\" that is currently top of the leaderboard?", "985527": "Code Requirements says that\n\n- Freely & publicly available external data is allowed, including pre-trained models\n\nSo, I'm not sure whether one can use their own pre-computed embeddings if it's not publicly available. Could you clarify it please?", "960838": "Thanks for the welcome message! Just a quick question: can you clarify this sentence please: \n\n> When you submit the kernel, it is evaluated on private test and train data\n\nWhat does evaluation on train data mean? ", "1009910": "Hi @tobwey,\nCan you allow us to upload the features file? (embedding) \nhost already wrote about the image file in the data section, but do not write about features.", "985651": "Nice and great!", "980961": "Hi @tobwey, is the private test equivalent to public test, in terms of number samples? (ie I'm just curious if it wont be 2x, 3x or more, it matters due to the time limit)", "978337": "", "961387": "" } }