Hi,
\nI was looking at the dataset and it seems that out of 260K images there are 121K images that are almost duplicates, most of them are pairs of images either in the train or in the test. I did find 40 images which are duplicates across train/ test.
\nHere are some examples: https://drive.google.com/file/d/1M0CpOMm-Gx89bKfm2ehi9XaGm2NKtYhm/view?usp=sharing
\nRealistically it seems that the data magniture can be reduced by half without any signal loss.
Hi,
\nI was looking at the dataset and it seems that out of 260K images there are 121K images that are almost duplicates, most of them are pairs of images either in the train or in the test. I did find 40 images which are duplicates across train/ test.
\nHere are some examples: https://drive.google.com/file/d/1M0CpOMm-Gx89bKfm2ehi9XaGm2NKtYhm/view?usp=sharing
\nRealistically it seems that the data magniture can be reduced by half without any signal loss.
The google drive link isn't working for me. Can you share a few image or sequence ids so I can check if there is an underlying issue.
", "rawMarkdown": "The google drive link isn't working for me. Can you share a few image or sequence ids so I can check if there is an underlying issue.", "votes": null }, { "id": "1785721", "postDate": "05/12/2022 10:57:02", "content": "Hi John, here are the most similar files between train/test:
\n,from,to,distance\n11,/test/972bfece-21bc-11ea-a13a-137349068a90.jpg,/train/97e34e8a-21bc-11ea-a13a-137349068a90.jpg,1.0\n23,/test/8e802c64-21bc-11ea-a13a-137349068a90.jpg,/train/96512bb4-21bc-11ea-a13a-137349068a90.jpg,1.0\n144,/test/93421820-21bc-11ea-a13a-137349068a90.jpg,/train/8b114860-21bc-11ea-a13a-137349068a90.jpg,1.0\n161,/test/89a7a690-21bc-11ea-a13a-137349068a90.jpg,/train/95d81cba-21bc-11ea-a13a-137349068a90.jpg,1.0\n392,/test/894d15fe-21bc-11ea-a13a-137349068a90.jpg,/train/8b5aee2a-21bc-11ea-a13a-137349068a90.jpg,1.0\n800,/test/8ea2a744-21bc-11ea-a13a-137349068a90.jpg,/train/8ac484d0-21bc-11ea-a13a-137349068a90.jpg,1.0\n1104,/test/969722e0-21bc-11ea-a13a-137349068a90.jpg,/train/8ad7b906-21bc-11ea-a13a-137349068a90.jpg,1.0\n1220,/test/934be1b6-21bc-11ea-a13a-137349068a90.jpg,/train/920d763e-21bc-11ea-a13a-137349068a90.jpg,1.0\n1363,/test/93bd03c8-21bc-11ea-a13a-137349068a90.jpg,/train/87aaf040-21bc-11ea-a13a-137349068a90.jpg,1.0\n1364,/test/93bd03c8-21bc-11ea-a13a-137349068a90.jpg,/train/8e0fee86-21bc-11ea-a13a-137349068a90.jpg,1.0\n1684,/test/9825b496-21bc-11ea-a13a-137349068a90.jpg,/train/976e7970-21bc-11ea-a13a-137349068a90.jpg,1.0\n1854,/test/89933cbe-21bc-11ea-a13a-137349068a90.jpg,/train/91be09e6-21bc-11ea-a13a-137349068a90.jpg,0.999999\n1861,/test/95eee94a-21bc-11ea-a13a-137349068a90.jpg,/train/923d0926-21bc-11ea-a13a-137349068a90.jpg,0.999999\n2671,/test/86a91d66-21bc-11ea-a13a-137349068a90.jpg,/train/8dc1c4cc-21bc-11ea-a13a-137349068a90.jpg,0.999888\n2674,/test/8aab99fc-21bc-11ea-a13a-137349068a90.jpg,/train/9577234c-21bc-11ea-a13a-137349068a90.jpg,0.999888\n2745,/test/88563892-21bc-11ea-a13a-137349068a90.jpg,/train/9184976a-21bc-11ea-a13a-137349068a90.jpg,0.999882\n2808,/test/990ba4d8-21bc-11ea-a13a-137349068a90.jpg,/train/9849e5c8-21bc-11ea-a13a-137349068a90.jpg,0.999878\n2888,/test/86a91d66-21bc-11ea-a13a-137349068a90.jpg,/train/87f24904-21bc-11ea-a13a-137349068a90.jpg,0.999872\n3005,/test/8f100c76-21bc-11ea-a13a-137349068a90.jpg,/train/96b01fd4-21bc-11ea-a13a-137349068a90.jpg,0.999864\n3054,/test/89a13da0-21bc-11ea-a13a-137349068a90.jpg,/train/96c287dc-21bc-11ea-a13a-137349068a90.jpg,0.999862\n3491,/test/8b7aa328-21bc-11ea-a13a-137349068a90.jpg,/train/962314f4-21bc-11ea-a13a-137349068a90.jpg,0.999839\n4173,/test/96d09890-21bc-11ea-a13a-137349068a90.jpg,/train/8b17246a-21bc-11ea-a13a-137349068a90.jpg,0.9998\n4234,/test/86f1375e-21bc-11ea-a13a-137349068a90.jpg,/train/978ff726-21bc-11ea-a13a-137349068a90.jpg,0.999796\n4523,/test/893fce1c-21bc-11ea-a13a-137349068a90.jpg,/train/8d994952-21bc-11ea-a13a-137349068a90.jpg,0.999769\n4678,/test/893fce1c-21bc-11ea-a13a-137349068a90.jpg,/train/93943baa-21bc-11ea-a13a-137349068a90.jpg,0.999751\n4898,/test/8f100c76-21bc-11ea-a13a-137349068a90.jpg,/train/8bb95bb8-21bc-11ea-a13a-137349068a90.jpg,0.99972\n7636,/test/8b008552-21bc-11ea-a13a-137349068a90.jpg,/train/91795710-21bc-11ea-a13a-137349068a90.jpg,0.999318\n9747,/test/9944097c-21bc-11ea-a13a-137349068a90.jpg,/train/96566958-21bc-11ea-a13a-137349068a90.jpg,0.999102\n43322,/test/87970210-21bc-11ea-a13a-137349068a90.jpg,/train/96b01fd4-21bc-11ea-a13a-137349068a90.jpg,0.996922\n48269,/test/98a6487c-21bc-11ea-a13a-137349068a90.jpg,/train/8872c390-21bc-11ea-a13a-137349068a90.jpg,0.996638\n49711,/test/99474a2e-21bc-11ea-a13a-137349068a90.jpg,/train/8813a9c8-21bc-11ea-a13a-137349068a90.jpg,0.996549\n63746,/test/8e42dc74-21bc-11ea-a13a-137349068a90.jpg,/train/93373432-21bc-11ea-a13a-137349068a90.jpg,0.995657\n80984,/test/90ddfeaa-21bc-11ea-a13a-137349068a90.jpg,/train/94046e8e-21bc-11ea-a13a-137349068a90.jpg,0.994336\n92743,/test/99474a2e-21bc-11ea-a13a-137349068a90.jpg,/train/90f7644e-21bc-11ea-a13a-137349068a90.jpg,0.993248\n96341,/test/953d53a6-21bc-11ea-a13a-137349068a90.jpg,/train/8c116894-21bc-11ea-a13a-137349068a90.jpg,0.992885\n113054,/test/8ea2a744-21bc-11ea-a13a-137349068a90.jpg,/train/8bbfb62a-21bc-11ea-a13a-137349068a90.jpg,0.991028\n115814,/test/9283ac46-21bc-11ea-a13a-137349068a90.jpg,/train/8cf0f626-21bc-11ea-a13a-137349068a90.jpg,0.990688\n119957,/test/93ee3902-21bc-11ea-a13a-137349068a90.jpg,/train/972ec6c2-21bc-11ea-a13a-137349068a90.jpg,0.990139\nHi! The iWildCam dataset is built on top of data and annotations collected by ecologists. Camera traps are static, so near-duplication is common (images taken one second apart from a static camera are often nearly identical). In the supplementary material of this paper we show that you can subsample the training data by half or even a quarter and get similar image classification results due to the low sample efficiency. However, that subsampling removes the temporal signal that should be valuable for counting. True duplicates in the data, particularly across train and test, would be due to an error in the metadata provided by the ecologists in the first place, as our splitting algorithm split the data based on unique location identifiers. We're looking into it, but don't plan to change the test set for this year as the duplication of 40 images shouldn't have a significant effect on the evaluation of a method over all 50K test images.
", "rawMarkdown": "Hi! The iWildCam dataset is built on top of data and annotations collected by ecologists. Camera traps are static, so near-duplication is common (images taken one second apart from a static camera are often nearly identical). In the supplementary material of [this paper](https://arxiv.org/abs/1807.04975) we show that you can subsample the training data by half or even a quarter and get similar image classification results due to the low sample efficiency. However, that subsampling removes the temporal signal that should be valuable for counting. True duplicates in the data, particularly across train and test, would be due to an error in the metadata provided by the ecologists in the first place, as our splitting algorithm split the data based on unique location identifiers. We're looking into it, but don't plan to change the test set for this year as the duplication of 40 images shouldn't have a significant effect on the evaluation of a method over all 50K test images.", "votes": null } ], "comments": [ { "id": 1785162, "author_name": "johnbeuving", "author_url": "", "post_date": "05/11/2022 21:30:04", "content": "The google drive link isn't working for me. Can you share a few image or sequence ids so I can check if there is an underlying issue.
", "votes": null, "replies": [] }, { "id": 1785721, "author_name": "graphlab", "author_url": "", "post_date": "05/12/2022 10:57:02", "content": "Hi John, here are the most similar files between train/test:
\n,from,to,distance\n11,/test/972bfece-21bc-11ea-a13a-137349068a90.jpg,/train/97e34e8a-21bc-11ea-a13a-137349068a90.jpg,1.0\n23,/test/8e802c64-21bc-11ea-a13a-137349068a90.jpg,/train/96512bb4-21bc-11ea-a13a-137349068a90.jpg,1.0\n144,/test/93421820-21bc-11ea-a13a-137349068a90.jpg,/train/8b114860-21bc-11ea-a13a-137349068a90.jpg,1.0\n161,/test/89a7a690-21bc-11ea-a13a-137349068a90.jpg,/train/95d81cba-21bc-11ea-a13a-137349068a90.jpg,1.0\n392,/test/894d15fe-21bc-11ea-a13a-137349068a90.jpg,/train/8b5aee2a-21bc-11ea-a13a-137349068a90.jpg,1.0\n800,/test/8ea2a744-21bc-11ea-a13a-137349068a90.jpg,/train/8ac484d0-21bc-11ea-a13a-137349068a90.jpg,1.0\n1104,/test/969722e0-21bc-11ea-a13a-137349068a90.jpg,/train/8ad7b906-21bc-11ea-a13a-137349068a90.jpg,1.0\n1220,/test/934be1b6-21bc-11ea-a13a-137349068a90.jpg,/train/920d763e-21bc-11ea-a13a-137349068a90.jpg,1.0\n1363,/test/93bd03c8-21bc-11ea-a13a-137349068a90.jpg,/train/87aaf040-21bc-11ea-a13a-137349068a90.jpg,1.0\n1364,/test/93bd03c8-21bc-11ea-a13a-137349068a90.jpg,/train/8e0fee86-21bc-11ea-a13a-137349068a90.jpg,1.0\n1684,/test/9825b496-21bc-11ea-a13a-137349068a90.jpg,/train/976e7970-21bc-11ea-a13a-137349068a90.jpg,1.0\n1854,/test/89933cbe-21bc-11ea-a13a-137349068a90.jpg,/train/91be09e6-21bc-11ea-a13a-137349068a90.jpg,0.999999\n1861,/test/95eee94a-21bc-11ea-a13a-137349068a90.jpg,/train/923d0926-21bc-11ea-a13a-137349068a90.jpg,0.999999\n2671,/test/86a91d66-21bc-11ea-a13a-137349068a90.jpg,/train/8dc1c4cc-21bc-11ea-a13a-137349068a90.jpg,0.999888\n2674,/test/8aab99fc-21bc-11ea-a13a-137349068a90.jpg,/train/9577234c-21bc-11ea-a13a-137349068a90.jpg,0.999888\n2745,/test/88563892-21bc-11ea-a13a-137349068a90.jpg,/train/9184976a-21bc-11ea-a13a-137349068a90.jpg,0.999882\n2808,/test/990ba4d8-21bc-11ea-a13a-137349068a90.jpg,/train/9849e5c8-21bc-11ea-a13a-137349068a90.jpg,0.999878\n2888,/test/86a91d66-21bc-11ea-a13a-137349068a90.jpg,/train/87f24904-21bc-11ea-a13a-137349068a90.jpg,0.999872\n3005,/test/8f100c76-21bc-11ea-a13a-137349068a90.jpg,/train/96b01fd4-21bc-11ea-a13a-137349068a90.jpg,0.999864\n3054,/test/89a13da0-21bc-11ea-a13a-137349068a90.jpg,/train/96c287dc-21bc-11ea-a13a-137349068a90.jpg,0.999862\n3491,/test/8b7aa328-21bc-11ea-a13a-137349068a90.jpg,/train/962314f4-21bc-11ea-a13a-137349068a90.jpg,0.999839\n4173,/test/96d09890-21bc-11ea-a13a-137349068a90.jpg,/train/8b17246a-21bc-11ea-a13a-137349068a90.jpg,0.9998\n4234,/test/86f1375e-21bc-11ea-a13a-137349068a90.jpg,/train/978ff726-21bc-11ea-a13a-137349068a90.jpg,0.999796\n4523,/test/893fce1c-21bc-11ea-a13a-137349068a90.jpg,/train/8d994952-21bc-11ea-a13a-137349068a90.jpg,0.999769\n4678,/test/893fce1c-21bc-11ea-a13a-137349068a90.jpg,/train/93943baa-21bc-11ea-a13a-137349068a90.jpg,0.999751\n4898,/test/8f100c76-21bc-11ea-a13a-137349068a90.jpg,/train/8bb95bb8-21bc-11ea-a13a-137349068a90.jpg,0.99972\n7636,/test/8b008552-21bc-11ea-a13a-137349068a90.jpg,/train/91795710-21bc-11ea-a13a-137349068a90.jpg,0.999318\n9747,/test/9944097c-21bc-11ea-a13a-137349068a90.jpg,/train/96566958-21bc-11ea-a13a-137349068a90.jpg,0.999102\n43322,/test/87970210-21bc-11ea-a13a-137349068a90.jpg,/train/96b01fd4-21bc-11ea-a13a-137349068a90.jpg,0.996922\n48269,/test/98a6487c-21bc-11ea-a13a-137349068a90.jpg,/train/8872c390-21bc-11ea-a13a-137349068a90.jpg,0.996638\n49711,/test/99474a2e-21bc-11ea-a13a-137349068a90.jpg,/train/8813a9c8-21bc-11ea-a13a-137349068a90.jpg,0.996549\n63746,/test/8e42dc74-21bc-11ea-a13a-137349068a90.jpg,/train/93373432-21bc-11ea-a13a-137349068a90.jpg,0.995657\n80984,/test/90ddfeaa-21bc-11ea-a13a-137349068a90.jpg,/train/94046e8e-21bc-11ea-a13a-137349068a90.jpg,0.994336\n92743,/test/99474a2e-21bc-11ea-a13a-137349068a90.jpg,/train/90f7644e-21bc-11ea-a13a-137349068a90.jpg,0.993248\n96341,/test/953d53a6-21bc-11ea-a13a-137349068a90.jpg,/train/8c116894-21bc-11ea-a13a-137349068a90.jpg,0.992885\n113054,/test/8ea2a744-21bc-11ea-a13a-137349068a90.jpg,/train/8bbfb62a-21bc-11ea-a13a-137349068a90.jpg,0.991028\n115814,/test/9283ac46-21bc-11ea-a13a-137349068a90.jpg,/train/8cf0f626-21bc-11ea-a13a-137349068a90.jpg,0.990688\n119957,/test/93ee3902-21bc-11ea-a13a-137349068a90.jpg,/train/972ec6c2-21bc-11ea-a13a-137349068a90.jpg,0.990139\nHi! The iWildCam dataset is built on top of data and annotations collected by ecologists. Camera traps are static, so near-duplication is common (images taken one second apart from a static camera are often nearly identical). In the supplementary material of this paper we show that you can subsample the training data by half or even a quarter and get similar image classification results due to the low sample efficiency. However, that subsampling removes the temporal signal that should be valuable for counting. True duplicates in the data, particularly across train and test, would be due to an error in the metadata provided by the ecologists in the first place, as our splitting algorithm split the data based on unique location identifiers. We're looking into it, but don't plan to change the test set for this year as the duplication of 40 images shouldn't have a significant effect on the evaluation of a method over all 50K test images.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1785051": "Hi, \nI was looking at the dataset and it seems that out of 260K images there are 121K images that are almost duplicates, most of them are pairs of images either in the train or in the test. I did find 40 images which are duplicates across train/ test. \nHere are some examples: https://drive.google.com/file/d/1M0CpOMm-Gx89bKfm2ehi9XaGm2NKtYhm/view?usp=sharing\nRealistically it seems that the data magniture can be reduced by half without any signal loss.", "1785162": "The google drive link isn't working for me. Can you share a few image or sequence ids so I can check if there is an underlying issue.", "1785721": "Hi John, here are the most similar files between train/test:\n\n```\n,from,to,distance\n11,/test/972bfece-21bc-11ea-a13a-137349068a90.jpg,/train/97e34e8a-21bc-11ea-a13a-137349068a90.jpg,1.0\n23,/test/8e802c64-21bc-11ea-a13a-137349068a90.jpg,/train/96512bb4-21bc-11ea-a13a-137349068a90.jpg,1.0\n144,/test/93421820-21bc-11ea-a13a-137349068a90.jpg,/train/8b114860-21bc-11ea-a13a-137349068a90.jpg,1.0\n161,/test/89a7a690-21bc-11ea-a13a-137349068a90.jpg,/train/95d81cba-21bc-11ea-a13a-137349068a90.jpg,1.0\n392,/test/894d15fe-21bc-11ea-a13a-137349068a90.jpg,/train/8b5aee2a-21bc-11ea-a13a-137349068a90.jpg,1.0\n800,/test/8ea2a744-21bc-11ea-a13a-137349068a90.jpg,/train/8ac484d0-21bc-11ea-a13a-137349068a90.jpg,1.0\n1104,/test/969722e0-21bc-11ea-a13a-137349068a90.jpg,/train/8ad7b906-21bc-11ea-a13a-137349068a90.jpg,1.0\n1220,/test/934be1b6-21bc-11ea-a13a-137349068a90.jpg,/train/920d763e-21bc-11ea-a13a-137349068a90.jpg,1.0\n1363,/test/93bd03c8-21bc-11ea-a13a-137349068a90.jpg,/train/87aaf040-21bc-11ea-a13a-137349068a90.jpg,1.0\n1364,/test/93bd03c8-21bc-11ea-a13a-137349068a90.jpg,/train/8e0fee86-21bc-11ea-a13a-137349068a90.jpg,1.0\n1684,/test/9825b496-21bc-11ea-a13a-137349068a90.jpg,/train/976e7970-21bc-11ea-a13a-137349068a90.jpg,1.0\n1854,/test/89933cbe-21bc-11ea-a13a-137349068a90.jpg,/train/91be09e6-21bc-11ea-a13a-137349068a90.jpg,0.999999\n1861,/test/95eee94a-21bc-11ea-a13a-137349068a90.jpg,/train/923d0926-21bc-11ea-a13a-137349068a90.jpg,0.999999\n2671,/test/86a91d66-21bc-11ea-a13a-137349068a90.jpg,/train/8dc1c4cc-21bc-11ea-a13a-137349068a90.jpg,0.999888\n2674,/test/8aab99fc-21bc-11ea-a13a-137349068a90.jpg,/train/9577234c-21bc-11ea-a13a-137349068a90.jpg,0.999888\n2745,/test/88563892-21bc-11ea-a13a-137349068a90.jpg,/train/9184976a-21bc-11ea-a13a-137349068a90.jpg,0.999882\n2808,/test/990ba4d8-21bc-11ea-a13a-137349068a90.jpg,/train/9849e5c8-21bc-11ea-a13a-137349068a90.jpg,0.999878\n2888,/test/86a91d66-21bc-11ea-a13a-137349068a90.jpg,/train/87f24904-21bc-11ea-a13a-137349068a90.jpg,0.999872\n3005,/test/8f100c76-21bc-11ea-a13a-137349068a90.jpg,/train/96b01fd4-21bc-11ea-a13a-137349068a90.jpg,0.999864\n3054,/test/89a13da0-21bc-11ea-a13a-137349068a90.jpg,/train/96c287dc-21bc-11ea-a13a-137349068a90.jpg,0.999862\n3491,/test/8b7aa328-21bc-11ea-a13a-137349068a90.jpg,/train/962314f4-21bc-11ea-a13a-137349068a90.jpg,0.999839\n4173,/test/96d09890-21bc-11ea-a13a-137349068a90.jpg,/train/8b17246a-21bc-11ea-a13a-137349068a90.jpg,0.9998\n4234,/test/86f1375e-21bc-11ea-a13a-137349068a90.jpg,/train/978ff726-21bc-11ea-a13a-137349068a90.jpg,0.999796\n4523,/test/893fce1c-21bc-11ea-a13a-137349068a90.jpg,/train/8d994952-21bc-11ea-a13a-137349068a90.jpg,0.999769\n4678,/test/893fce1c-21bc-11ea-a13a-137349068a90.jpg,/train/93943baa-21bc-11ea-a13a-137349068a90.jpg,0.999751\n4898,/test/8f100c76-21bc-11ea-a13a-137349068a90.jpg,/train/8bb95bb8-21bc-11ea-a13a-137349068a90.jpg,0.99972\n7636,/test/8b008552-21bc-11ea-a13a-137349068a90.jpg,/train/91795710-21bc-11ea-a13a-137349068a90.jpg,0.999318\n9747,/test/9944097c-21bc-11ea-a13a-137349068a90.jpg,/train/96566958-21bc-11ea-a13a-137349068a90.jpg,0.999102\n43322,/test/87970210-21bc-11ea-a13a-137349068a90.jpg,/train/96b01fd4-21bc-11ea-a13a-137349068a90.jpg,0.996922\n48269,/test/98a6487c-21bc-11ea-a13a-137349068a90.jpg,/train/8872c390-21bc-11ea-a13a-137349068a90.jpg,0.996638\n49711,/test/99474a2e-21bc-11ea-a13a-137349068a90.jpg,/train/8813a9c8-21bc-11ea-a13a-137349068a90.jpg,0.996549\n63746,/test/8e42dc74-21bc-11ea-a13a-137349068a90.jpg,/train/93373432-21bc-11ea-a13a-137349068a90.jpg,0.995657\n80984,/test/90ddfeaa-21bc-11ea-a13a-137349068a90.jpg,/train/94046e8e-21bc-11ea-a13a-137349068a90.jpg,0.994336\n92743,/test/99474a2e-21bc-11ea-a13a-137349068a90.jpg,/train/90f7644e-21bc-11ea-a13a-137349068a90.jpg,0.993248\n96341,/test/953d53a6-21bc-11ea-a13a-137349068a90.jpg,/train/8c116894-21bc-11ea-a13a-137349068a90.jpg,0.992885\n113054,/test/8ea2a744-21bc-11ea-a13a-137349068a90.jpg,/train/8bbfb62a-21bc-11ea-a13a-137349068a90.jpg,0.991028\n115814,/test/9283ac46-21bc-11ea-a13a-137349068a90.jpg,/train/8cf0f626-21bc-11ea-a13a-137349068a90.jpg,0.990688\n119957,/test/93ee3902-21bc-11ea-a13a-137349068a90.jpg,/train/972ec6c2-21bc-11ea-a13a-137349068a90.jpg,0.990139\n```", "1792014": "Hi! The iWildCam dataset is built on top of data and annotations collected by ecologists. Camera traps are static, so near-duplication is common (images taken one second apart from a static camera are often nearly identical). In the supplementary material of [this paper](https://arxiv.org/abs/1807.04975) we show that you can subsample the training data by half or even a quarter and get similar image classification results due to the low sample efficiency. However, that subsampling removes the temporal signal that should be valuable for counting. True duplicates in the data, particularly across train and test, would be due to an error in the metadata provided by the ecologists in the first place, as our splitting algorithm split the data based on unique location identifiers. We're looking into it, but don't plan to change the test set for this year as the duplication of 40 images shouldn't have a significant effect on the evaluation of a method over all 50K test images." }, "source": "meta" }