![\"\"](\"https://i.ibb.co/fFSwFMc/duplicate.png\")
Our annotation team updated the labels for the test datasets (private and public) but the train labels remain the same.
\nThe updated test labels correct the problem regarding duplicates or extra images on some of the studies, which previously did not have bounding box information. Bounding box information is now provided for duplicates or similar images that are part of the same study for the public and private test datasets.
\nHowever, regarding the train labels, due to time constraints, those labels were kept the same and not updated. As such, in situations where there are 2 or more images belonging to a study, we recommend only using the labels for the image with the bounding boxes and disregard the other duplicate/similar images. These other duplicate/similar images were likely not looked at by the annotators as we were unaware of them during the initial annotation process.
", "messages": [ { "id": 1351107, "postDate": "2021-06-16T04:37:32.980Z", "content": "Our annotation team updated the labels for the test datasets (private and public) but the train labels remain the same.
\nThe updated test labels correct the problem regarding duplicates or extra images on some of the studies, which previously did not have bounding box information. Bounding box information is now provided for duplicates or similar images that are part of the same study for the public and private test datasets.
\nHowever, regarding the train labels, due to time constraints, those labels were kept the same and not updated. As such, in situations where there are 2 or more images belonging to a study, we recommend only using the labels for the image with the bounding boxes and disregard the other duplicate/similar images. These other duplicate/similar images were likely not looked at by the annotators as we were unaware of them during the initial annotation process.
", "rawMarkdown": "Our annotation team updated the labels for the test datasets (private and public) but the train labels remain the same.\n\nThe updated test labels correct the problem regarding duplicates or extra images on some of the studies, which previously did not have bounding box information. Bounding box information is now provided for duplicates or similar images that are part of the same study for the public and private test datasets.\n\nHowever, regarding the train labels, due to time constraints, those labels were kept the same and not updated. As such, in situations where there are 2 or more images belonging to a study, we recommend only using the labels for the image with the bounding boxes and disregard the other duplicate/similar images. These other duplicate/similar images were likely not looked at by the annotators as we were unaware of them during the initial annotation process. ", "votes": 72 }, { "id": 1351173, "postDate": "2021-06-16T05:42:03.843Z", "content": "Thanks for the post.
\nYou mean…
\nFor train data, we have to use only BBoxed images if they are duplicates, but we can still use non-duplicated images without BBox as no-BBox control ones.
\nFor test data, true BBox info is corrected now, but the image itself is the same, so we don't have to download it again to make PNGs because it is just a labeling repairment (not images but true labels corrected).
\nIs my understanding correct?
it is up to you how to decide duplicate images and bbox label in train.
\nthe host suggest to use duplicate image with bbox label (but i think he has missed the fact that there may be some train duplicates with different bbox label? … i haven't checked the images and labels in detail yet)
you can treat the problem now as \"having training dataset with some label noise or uncertainty\".
\nit is up to you how to handle this noise.
but i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?
\nbut nevertheless, kaggler should take note of any difference of scores in local cross-validation and the public leaderboard to account for domain change or label noise
", "rawMarkdown": "it is up to you how to decide duplicate images and bbox label in train.\nthe host suggest to use duplicate image with bbox label (but i think he has missed the fact that there may be some train duplicates with different bbox label? ... i haven't checked the images and labels in detail yet)\n\nyou can treat the problem now as \"having training dataset with some label noise or uncertainty\".\nit is up to you how to handle this noise.\n\n\nbut i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?\n\nbut nevertheless, kaggler should take note of any difference of scores in local cross-validation and the public leaderboard to account for domain change or label noise", "votes": 8 }, { "id": 1353916, "postDate": "2021-06-17T09:41:46.877Z", "content": "\n\"We can consider the confidence score to be 1 always for every image at study-level because if you look at the train data, you will see that each image has only one label from 'negative', 'typical', 'indeterminate', 'atypical'.\"
\nis this true any more at study level?
", "rawMarkdown": "@hengck23 \n\n**\"We can consider the confidence score to be 1 always for every image at study-level because if you look at the train data, you will see that each image has only one label from 'negative', 'typical', 'indeterminate', 'atypical'.\"**\n\nis this true any more at study level?", "votes": 1 }, { "id": 1360361, "postDate": "2021-06-22T04:01:02.297Z", "content": "\nThere seems to be only one image with with different bounding boxes marked. This particular image is part of two different studies and is labeled as atypical appearance in one study and typical appearance in the other study.
Notebook with complete list of duplicate images here: siim-covid-19-duplicate-training-images
\n
Is this still the case or the dataset has been fixed ?
", "rawMarkdown": "Is this still the case or the dataset has been fixed ?" } ] }, { "id": 1459996, "postDate": "2021-08-08T16:35:08.873Z", "content": "Commen by Russian doesn`t count
", "rawMarkdown": "Commen by Russian doesn`t count" }, { "id": 1447013, "postDate": "2021-08-04T13:48:03.930Z", "content": "Hi I am new Here
", "rawMarkdown": "Hi I am new Here" }, { "id": 1404690, "postDate": "2021-07-30T07:04:16.827Z", "content": "i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?
", "rawMarkdown": "i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?" }, { "id": 1381047, "postDate": "2021-07-08T15:14:07.210Z", "content": "Does it mean that study level is also incorrect? Or are all the labels placed correctly at the study level?
", "rawMarkdown": "Does it mean that study level is also incorrect? Or are all the labels placed correctly at the study level?" }, { "id": 1355219, "postDate": "2021-06-18T06:52:26.080Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1464849, "postDate": "2021-08-10T18:27:58.620Z", "content": "Thanks for the post
", "rawMarkdown": "Thanks for the post" }, { "id": 1393003, "postDate": "2021-07-19T09:48:13.550Z", "content": "thank you for your generous
", "rawMarkdown": "thank you for your generous" } ], "comments": [ { "id": 1351173, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-06-16T05:42:03.843000", "content": "Thanks for the post.
\nYou mean…
\nFor train data, we have to use only BBoxed images if they are duplicates, but we can still use non-duplicated images without BBox as no-BBox control ones.
\nFor test data, true BBox info is corrected now, but the image itself is the same, so we don't have to download it again to make PNGs because it is just a labeling repairment (not images but true labels corrected).
\nIs my understanding correct?
it is up to you how to decide duplicate images and bbox label in train.
\nthe host suggest to use duplicate image with bbox label (but i think he has missed the fact that there may be some train duplicates with different bbox label? … i haven't checked the images and labels in detail yet)
you can treat the problem now as \"having training dataset with some label noise or uncertainty\".
\nit is up to you how to handle this noise.
but i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?
\nbut nevertheless, kaggler should take note of any difference of scores in local cross-validation and the public leaderboard to account for domain change or label noise
", "votes": 8, "replies": [] }, { "id": 1353916, "author_name": "Jaideep", "author_url": "", "post_date": "2021-06-17T09:41:46.877000", "content": "\n\"We can consider the confidence score to be 1 always for every image at study-level because if you look at the train data, you will see that each image has only one label from 'negative', 'typical', 'indeterminate', 'atypical'.\"
\nis this true any more at study level?
", "votes": 1, "replies": [] }, { "id": 1360361, "author_name": "KWK", "author_url": "", "post_date": "2021-06-22T04:01:02.297000", "content": "\nThere seems to be only one image with with different bounding boxes marked. This particular image is part of two different studies and is labeled as atypical appearance in one study and typical appearance in the other study.
Notebook with complete list of duplicate images here: siim-covid-19-duplicate-training-images
\n
Is this still the case or the dataset has been fixed ?
", "votes": 0, "replies": [] } ] }, { "id": 1459996, "author_name": "Ulymov Igor", "author_url": "", "post_date": "2021-08-08T16:35:08.873000", "content": "Commen by Russian doesn`t count
", "votes": 0, "replies": [] }, { "id": 1447013, "author_name": "tanmay walke", "author_url": "", "post_date": "2021-08-04T13:48:03.930000", "content": "Hi I am new Here
", "votes": 0, "replies": [] }, { "id": 1404690, "author_name": "Manjula Srihari", "author_url": "", "post_date": "2021-07-30T07:04:16.827000", "content": "i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?
", "votes": 0, "replies": [] }, { "id": 1381047, "author_name": "Aleksandr Emchinov", "author_url": "", "post_date": "2021-07-08T15:14:07.210000", "content": "Does it mean that study level is also incorrect? Or are all the labels placed correctly at the study level?
", "votes": 0, "replies": [] }, { "id": 1355219, "author_name": "", "author_url": "", "post_date": "2021-06-18T06:52:26.080000", "content": "", "votes": 1, "replies": [] }, { "id": 1464849, "author_name": "Anand Krishnan", "author_url": "", "post_date": "2021-08-10T18:27:58.620000", "content": "Thanks for the post
", "votes": 0, "replies": [] }, { "id": 1393003, "author_name": "jinpaigiegie", "author_url": "", "post_date": "2021-07-19T09:48:13.550000", "content": "thank you for your generous
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1351107": "Our annotation team updated the labels for the test datasets (private and public) but the train labels remain the same.\n\nThe updated test labels correct the problem regarding duplicates or extra images on some of the studies, which previously did not have bounding box information. Bounding box information is now provided for duplicates or similar images that are part of the same study for the public and private test datasets.\n\nHowever, regarding the train labels, due to time constraints, those labels were kept the same and not updated. As such, in situations where there are 2 or more images belonging to a study, we recommend only using the labels for the image with the bounding boxes and disregard the other duplicate/similar images. These other duplicate/similar images were likely not looked at by the annotators as we were unaware of them during the initial annotation process. ", "1351173": "Thanks for the post.\nYou mean...\nFor train data, we have to use only BBoxed images if they are duplicates, but we can still use non-duplicated images without BBox as no-BBox control ones.\nFor test data, true BBox info is corrected now, but the image itself is the same, so we don't have to download it again to make PNGs because it is just a labeling repairment (not images but true labels corrected).\nIs my understanding correct?", "1459996": "Commen by Russian doesn`t count", "1447013": "Hi I am new Here", "1404690": "i am worried that the test and train images are labelled \"differently\" since there are no details on \"updated the labels for the test datasets\" (e.g. did the annotator changed?, etc). will there be domain difference?", "1381047": "Does it mean that study level is also incorrect? Or are all the labels placed correctly at the study level?", "1355219": "", "1464849": "Thanks for the post", "1393003": "thank you for your generous" } }