{
  "id": 164769,
  "title": "Summary of GLDv2 paper - 'A Large-scale benchmark for instance-level recognition and retrieval'",
  "url": "/competitions/landmark-retrieval-2020/discussion/164769",
  "author_name": "SkyLord",
  "post_date": "2020-07-07T13:30:17.632000",
  "votes": 1,
  "comment_count": 0,
  "views": 0,
  "content": "<p><strong>Heatmap of Landmarks</strong>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252874%2F8cb0345a521d828a60de19b8bed2bbc4%2FHeatmap_of_landmarks.JPG?generation=1594133511135051&amp;alt=media\" alt=\"Heatmap of Landmarks\"></p>\n\n<p>GLDv1 limitations:\n- 2.3mn images from 30k landmarks\n- but had copyright restrictions \n- hence dataset was not stable as owners removed their images </p>\n\n<p>GLDv2 has 5mn images &amp; 200k distinct instance labels \nThe new dataset is for large-scale, fine-grained instance recognition and image retrievals in the domain of human-made and natural landmarks. The dataset is a new benchmark for instance-level recognition and retrieval</p>\n\n<p>| Dataset | No of Images | No of Landmarks | Copyrights* | \n| ------- | ------------ | -------- | ----- |\n| Train set | 4,132,914 | 203k | CC-BY w/o ND licenses |\n| Train set (cleaned)| 1.2mn | 15k | CC-BY w/o ND |\n| Index set | 761,757 | 101k | CC-BY 4.0 |\n| Test  set | 117,577 | -  | CC-BY 4.0 |</p>\n\n<p><em>Note :dataset authors do not take any warranties for the image copyrights</em></p>\n\n<p>Images allow for reproductions and indefinite retention</p>\n\n<p>index &amp; training set have a large overlap with 92k common classes \nOnly 1% of the test images are within the target domain of landmarks while 99% are out-of-domain images\n| Landmark Type | %age |\n| ---- | ---- |\n| Natural Landmarks | 28% |\n| Human-made landmarks | 72% |</p>\n\n<p>Churches, parks and museaums are thethree largest classes </p>\n\n<p>The dataset has landmarks from 246 of 249 countries in the ISO 3166-1 country list \nbut it has a European bias , esp. Germany </p>\n\n<p>The Goals of the dataset- \n1. Large geographic scale, covering the entire world\n2. Intra-class variability - photos are taken under different lighting conditions &amp; from different views (including indoor, outdoor views) phots related to landmark but not including it directy like floor plans, potraits of architects or views from landmarks \n3. Long-tailed class distribution - more photos of famous landmarks than of lesser known ones\n4. Out-of domain queries - query can come from various applications like photo-album apps or visual search apps and may contain only a small fraction of the landmark </p>\n\n<p>What the dataset doesn't try to achieve - \n1. creating a clean query &amp; index dataset\n2. to measure generalization of embedding models to unseen data\n3. index &amp; training dataset do not have dis-joint class sets\n4. In the current iteration - does not aim to have image-level retrieval ground truths; instead they are at class-level</p>\n\n<p><strong>A note on dataset construction</strong></p>\n\n<ol>\n<li>The main source is <em>Wikimedia Commons</em></li>\n<li>Wiki organizes an annual challenge <em>Wiki Loves Monuments</em>, an annual world-wide contest for uploading high-quality freely licensed photos of landmarks </li>\n<li>In addition realistic query images were gathered via crowd-sourcing.</li>\n<li>Operators were sent out to take photos of selected landmarks of the world with smartphones</li>\n</ol>\n\n<p><strong>Metrics</strong> \n- For Retrieval \n<em>macro Average Precision</em> (mAP@100)\nDoesn't penalize if the system predicts a landmark for a non-domain queries, since such results are ignored</p>\n\n<ul>\n<li>For Recognition \n<em>Global Average Precision</em> (GAP)\nPenalizes if system predicts a landmark for a non-domain queries</li>\n</ul>\n\n<p><strong>Section 5: On experiments</strong>\n<em>This section contains results of comparison with comparable datasets and baseline benchmarks</em>\n*As such I think it would be better to read it from the <a href=\"https://arxiv.org/pdf/2004.01804.pdf\">original source!</a>*</p>\n\n<p><strong>Some useful links</strong>\n<a href=\"https://arxiv.org/pdf/2004.01804.pdf\">arxiv link to Paper</a>\n<a href=\"https://storage.googleapis.com/gld-v2/web/index.html\">Visual explorer for dataset</a>\n<a href=\"https://github.com/cvdfoundation/google-landmark\">Dataset URL</a>\n<a href=\"https://github.com/tensorflow/models/tree/master/research/delf/delf/python/google_landmarks_dataset\">Models from previous dataset are provided here</a></p>\n\n<p><code>\"Google Landmarks Dataset v2 - A Large-Scale Benchmark for Instance-Level Recognition and Retrieval\", T. Weyand, A. Araujo, B. Cao and J. Sim, Proc. CVPR'20\n</code></p>",
  "messages": [
    {
      "id": 918763,
      "postDate": "2020-07-07T13:30:17.633Z",
      "content": "<p><strong>Heatmap of Landmarks</strong>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252874%2F8cb0345a521d828a60de19b8bed2bbc4%2FHeatmap_of_landmarks.JPG?generation=1594133511135051&amp;alt=media\" alt=\"Heatmap of Landmarks\"></p>\n\n<p>GLDv1 limitations:\n- 2.3mn images from 30k landmarks\n- but had copyright restrictions \n- hence dataset was not stable as owners removed their images </p>\n\n<p>GLDv2 has 5mn images &amp; 200k distinct instance labels \nThe new dataset is for large-scale, fine-grained instance recognition and image retrievals in the domain of human-made and natural landmarks. The dataset is a new benchmark for instance-level recognition and retrieval</p>\n\n<p>| Dataset | No of Images | No of Landmarks | Copyrights* | \n| ------- | ------------ | -------- | ----- |\n| Train set | 4,132,914 | 203k | CC-BY w/o ND licenses |\n| Train set (cleaned)| 1.2mn | 15k | CC-BY w/o ND |\n| Index set | 761,757 | 101k | CC-BY 4.0 |\n| Test  set | 117,577 | -  | CC-BY 4.0 |</p>\n\n<p><em>Note :dataset authors do not take any warranties for the image copyrights</em></p>\n\n<p>Images allow for reproductions and indefinite retention</p>\n\n<p>index &amp; training set have a large overlap with 92k common classes \nOnly 1% of the test images are within the target domain of landmarks while 99% are out-of-domain images\n| Landmark Type | %age |\n| ---- | ---- |\n| Natural Landmarks | 28% |\n| Human-made landmarks | 72% |</p>\n\n<p>Churches, parks and museaums are thethree largest classes </p>\n\n<p>The dataset has landmarks from 246 of 249 countries in the ISO 3166-1 country list \nbut it has a European bias , esp. Germany </p>\n\n<p>The Goals of the dataset- \n1. Large geographic scale, covering the entire world\n2. Intra-class variability - photos are taken under different lighting conditions &amp; from different views (including indoor, outdoor views) phots related to landmark but not including it directy like floor plans, potraits of architects or views from landmarks \n3. Long-tailed class distribution - more photos of famous landmarks than of lesser known ones\n4. Out-of domain queries - query can come from various applications like photo-album apps or visual search apps and may contain only a small fraction of the landmark </p>\n\n<p>What the dataset doesn't try to achieve - \n1. creating a clean query &amp; index dataset\n2. to measure generalization of embedding models to unseen data\n3. index &amp; training dataset do not have dis-joint class sets\n4. In the current iteration - does not aim to have image-level retrieval ground truths; instead they are at class-level</p>\n\n<p><strong>A note on dataset construction</strong></p>\n\n<ol>\n<li>The main source is <em>Wikimedia Commons</em></li>\n<li>Wiki organizes an annual challenge <em>Wiki Loves Monuments</em>, an annual world-wide contest for uploading high-quality freely licensed photos of landmarks </li>\n<li>In addition realistic query images were gathered via crowd-sourcing.</li>\n<li>Operators were sent out to take photos of selected landmarks of the world with smartphones</li>\n</ol>\n\n<p><strong>Metrics</strong> \n- For Retrieval \n<em>macro Average Precision</em> (mAP@100)\nDoesn't penalize if the system predicts a landmark for a non-domain queries, since such results are ignored</p>\n\n<ul>\n<li>For Recognition \n<em>Global Average Precision</em> (GAP)\nPenalizes if system predicts a landmark for a non-domain queries</li>\n</ul>\n\n<p><strong>Section 5: On experiments</strong>\n<em>This section contains results of comparison with comparable datasets and baseline benchmarks</em>\n*As such I think it would be better to read it from the <a href=\"https://arxiv.org/pdf/2004.01804.pdf\">original source!</a>*</p>\n\n<p><strong>Some useful links</strong>\n<a href=\"https://arxiv.org/pdf/2004.01804.pdf\">arxiv link to Paper</a>\n<a href=\"https://storage.googleapis.com/gld-v2/web/index.html\">Visual explorer for dataset</a>\n<a href=\"https://github.com/cvdfoundation/google-landmark\">Dataset URL</a>\n<a href=\"https://github.com/tensorflow/models/tree/master/research/delf/delf/python/google_landmarks_dataset\">Models from previous dataset are provided here</a></p>\n\n<p><code>\"Google Landmarks Dataset v2 - A Large-Scale Benchmark for Instance-Level Recognition and Retrieval\", T. Weyand, A. Araujo, B. Cao and J. Sim, Proc. CVPR'20\n</code></p>",
      "rawMarkdown": "**Heatmap of Landmarks**\n![Heatmap of Landmarks](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252874%2F8cb0345a521d828a60de19b8bed2bbc4%2FHeatmap_of_landmarks.JPG?generation=1594133511135051&amp;alt=media)\n\n\nGLDv1 limitations:\n- 2.3mn images from 30k landmarks\n- but had copyright restrictions \n- hence dataset was not stable as owners removed their images \n\nGLDv2 has 5mn images &amp; 200k distinct instance labels \nThe new dataset is for large-scale, fine-grained instance recognition and image retrievals in the domain of human-made and natural landmarks. The dataset is a new benchmark for instance-level recognition and retrieval\n\n| Dataset | No of Images | No of Landmarks | Copyrights* | \n| ------- | ------------ | -------- | ----- |\n| Train set | 4,132,914 | 203k | CC-BY w/o ND licenses |\n| Train set (cleaned)| 1.2mn | 15k | CC-BY w/o ND |\n| Index set | 761,757 | 101k | CC-BY 4.0 |\n| Test  set | 117,577 | -  | CC-BY 4.0 |\n\n*Note :dataset authors do not take any warranties for the image copyrights*\n\nImages allow for reproductions and indefinite retention\n\nindex &amp; training set have a large overlap with 92k common classes \nOnly 1% of the test images are within the target domain of landmarks while 99% are out-of-domain images\n| Landmark Type | %age |\n| ---- | ---- |\n| Natural Landmarks | 28% |\n| Human-made landmarks | 72% |\n\nChurches, parks and museaums are thethree largest classes \n\nThe dataset has landmarks from 246 of 249 countries in the ISO 3166-1 country list \nbut it has a European bias , esp. Germany \n\nThe Goals of the dataset- \n1. Large geographic scale, covering the entire world\n2. Intra-class variability - photos are taken under different lighting conditions &amp; from different views (including indoor, outdoor views) phots related to landmark but not including it directy like floor plans, potraits of architects or views from landmarks \n3. Long-tailed class distribution - more photos of famous landmarks than of lesser known ones\n4. Out-of domain queries - query can come from various applications like photo-album apps or visual search apps and may contain only a small fraction of the landmark \n\nWhat the dataset doesn't try to achieve - \n1. creating a clean query &amp; index dataset\n2. to measure generalization of embedding models to unseen data\n3. index &amp; training dataset do not have dis-joint class sets\n4. In the current iteration - does not aim to have image-level retrieval ground truths; instead they are at class-level\n\n**A note on dataset construction**\n\n1. The main source is *Wikimedia Commons*\n2. Wiki organizes an annual challenge *Wiki Loves Monuments*, an annual world-wide contest for uploading high-quality freely licensed photos of landmarks \n3. In addition realistic query images were gathered via crowd-sourcing.\n4. Operators were sent out to take photos of selected landmarks of the world with smartphones\n\n\n\n**Metrics** \n- For Retrieval \n*macro Average Precision* (mAP@100)\nDoesn't penalize if the system predicts a landmark for a non-domain queries, since such results are ignored\n\n- For Recognition \n*Global Average Precision* (GAP)\nPenalizes if system predicts a landmark for a non-domain queries\n\n**Section 5: On experiments**\n*This section contains results of comparison with comparable datasets and baseline benchmarks*\n*As such I think it would be better to read it from the [original source!](https://arxiv.org/pdf/2004.01804.pdf)*\n\n**Some useful links**\n[arxiv link to Paper](https://arxiv.org/pdf/2004.01804.pdf)\n[Visual explorer for dataset](https://storage.googleapis.com/gld-v2/web/index.html)\n[Dataset URL](https://github.com/cvdfoundation/google-landmark)\n[Models from previous dataset are provided here](https://github.com/tensorflow/models/tree/master/research/delf/delf/python/google_landmarks_dataset)\n\n`\"Google Landmarks Dataset v2 - A Large-Scale Benchmark for Instance-Level Recognition and Retrieval\", T. Weyand, A. Araujo, B. Cao and J. Sim, Proc. CVPR'20\n`\n",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "918763": "**Heatmap of Landmarks**\n![Heatmap of Landmarks](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F252874%2F8cb0345a521d828a60de19b8bed2bbc4%2FHeatmap_of_landmarks.JPG?generation=1594133511135051&amp;alt=media)\n\n\nGLDv1 limitations:\n- 2.3mn images from 30k landmarks\n- but had copyright restrictions \n- hence dataset was not stable as owners removed their images \n\nGLDv2 has 5mn images &amp; 200k distinct instance labels \nThe new dataset is for large-scale, fine-grained instance recognition and image retrievals in the domain of human-made and natural landmarks. The dataset is a new benchmark for instance-level recognition and retrieval\n\n| Dataset | No of Images | No of Landmarks | Copyrights* | \n| ------- | ------------ | -------- | ----- |\n| Train set | 4,132,914 | 203k | CC-BY w/o ND licenses |\n| Train set (cleaned)| 1.2mn | 15k | CC-BY w/o ND |\n| Index set | 761,757 | 101k | CC-BY 4.0 |\n| Test  set | 117,577 | -  | CC-BY 4.0 |\n\n*Note :dataset authors do not take any warranties for the image copyrights*\n\nImages allow for reproductions and indefinite retention\n\nindex &amp; training set have a large overlap with 92k common classes \nOnly 1% of the test images are within the target domain of landmarks while 99% are out-of-domain images\n| Landmark Type | %age |\n| ---- | ---- |\n| Natural Landmarks | 28% |\n| Human-made landmarks | 72% |\n\nChurches, parks and museaums are thethree largest classes \n\nThe dataset has landmarks from 246 of 249 countries in the ISO 3166-1 country list \nbut it has a European bias , esp. Germany \n\nThe Goals of the dataset- \n1. Large geographic scale, covering the entire world\n2. Intra-class variability - photos are taken under different lighting conditions &amp; from different views (including indoor, outdoor views) phots related to landmark but not including it directy like floor plans, potraits of architects or views from landmarks \n3. Long-tailed class distribution - more photos of famous landmarks than of lesser known ones\n4. Out-of domain queries - query can come from various applications like photo-album apps or visual search apps and may contain only a small fraction of the landmark \n\nWhat the dataset doesn't try to achieve - \n1. creating a clean query &amp; index dataset\n2. to measure generalization of embedding models to unseen data\n3. index &amp; training dataset do not have dis-joint class sets\n4. In the current iteration - does not aim to have image-level retrieval ground truths; instead they are at class-level\n\n**A note on dataset construction**\n\n1. The main source is *Wikimedia Commons*\n2. Wiki organizes an annual challenge *Wiki Loves Monuments*, an annual world-wide contest for uploading high-quality freely licensed photos of landmarks \n3. In addition realistic query images were gathered via crowd-sourcing.\n4. Operators were sent out to take photos of selected landmarks of the world with smartphones\n\n\n\n**Metrics** \n- For Retrieval \n*macro Average Precision* (mAP@100)\nDoesn't penalize if the system predicts a landmark for a non-domain queries, since such results are ignored\n\n- For Recognition \n*Global Average Precision* (GAP)\nPenalizes if system predicts a landmark for a non-domain queries\n\n**Section 5: On experiments**\n*This section contains results of comparison with comparable datasets and baseline benchmarks*\n*As such I think it would be better to read it from the [original source!](https://arxiv.org/pdf/2004.01804.pdf)*\n\n**Some useful links**\n[arxiv link to Paper](https://arxiv.org/pdf/2004.01804.pdf)\n[Visual explorer for dataset](https://storage.googleapis.com/gld-v2/web/index.html)\n[Dataset URL](https://github.com/cvdfoundation/google-landmark)\n[Models from previous dataset are provided here](https://github.com/tensorflow/models/tree/master/research/delf/delf/python/google_landmarks_dataset)\n\n`\"Google Landmarks Dataset v2 - A Large-Scale Benchmark for Instance-Level Recognition and Retrieval\", T. Weyand, A. Araujo, B. Cao and J. Sim, Proc. CVPR'20\n`\n"
  }
}