{ "id": 239980, "title": "Same images but different annotations", "url": "/competitions/siim-covid19-detection/discussion/239980", "author_name": "cool_rabbit", "post_date": "2021-05-18T07:47:51.385000", "votes": 70, "comment_count": 20, "views": 0, "content": "

These are all study_id: 'a7335b2f9815' and seem to be the same images, but only one of them is annotated as opacity.
\nWhat does it mean?
\nDifferent annotators?
\nPlease describe the situation @paras42
\n\"Screen

", "messages": [ { "id": 1312715, "postDate": "2021-05-18T07:47:51.387Z", "content": "

These are all study_id: 'a7335b2f9815' and seem to be the same images, but only one of them is annotated as opacity.
\nWhat does it mean?
\nDifferent annotators?
\nPlease describe the situation @paras42
\n\"Screen

", "rawMarkdown": "These are all study_id: 'a7335b2f9815' and seem to be the same images, but only one of them is annotated as opacity.\nWhat does it mean?\nDifferent annotators?\nPlease describe the situation @paras42 \n\"Screen", "votes": 68 }, { "id": 1322944, "postDate": "2021-05-25T20:10:01.227Z", "content": "

We are now aware regarding some studies having multiple images (some are duplicates), some with bounding boxes, and others with not. We are investigating different approaches to remedy this issue. We will soon follow-up when we arrive at a solution and let you know.

", "rawMarkdown": "We are now aware regarding some studies having multiple images (some are duplicates), some with bounding boxes, and others with not. We are investigating different approaches to remedy this issue. We will soon follow-up when we arrive at a solution and let you know.", "votes": 14, "replies": [ { "id": 1323186, "postDate": "2021-05-26T04:29:35.407Z", "content": "

Thanks for the response.

", "rawMarkdown": "Thanks for the response." }, { "id": 1329077, "postDate": "2021-05-30T20:04:52.870Z", "content": "

Any update on this? It’s been weeks now since this issue was first noticed?

", "rawMarkdown": "Any update on this? It’s been weeks now since this issue was first noticed?", "votes": 1 }, { "id": 1329228, "postDate": "2021-05-31T01:31:38.597Z", "content": "

I guess it's under construction.

", "rawMarkdown": "I guess it's under construction." } ] }, { "id": 1312774, "postDate": "2021-05-18T08:34:14.980Z", "content": "

study_id: '8943d1d85097'
\nOne of them horizontal-flipped.
\n\"Screen

", "rawMarkdown": "study_id: '8943d1d85097'\nOne of them horizontal-flipped.\n\"Screen", "votes": 9, "replies": [ { "id": 1321063, "postDate": "2021-05-24T13:55:13.677Z", "content": "

Probably in case of positive studies for now its safer to only train with images with a bbox annotation and remove others.

", "rawMarkdown": "Probably in case of positive studies for now its safer to only train with images with a bbox annotation and remove others. ", "votes": 6 }, { "id": 1321474, "postDate": "2021-05-24T17:46:04.307Z", "content": "

Yes, I choose that strategy now.

", "rawMarkdown": "Yes, I choose that strategy now." } ] }, { "id": 1321060, "postDate": "2021-05-24T13:53:41.617Z", "content": "

How do you calculate LB score just for image level? Perhaps leaving study preds as an empty string “”?

", "rawMarkdown": "How do you calculate LB score just for image level? Perhaps leaving study preds as an empty string “”?", "votes": 1, "replies": [ { "id": 1321472, "postDate": "2021-05-24T17:45:22.520Z", "content": "

I compared LB scores with or without my detection model.
\nWith: my pred
\nWithout: all none 1 0 0 1 1

", "rawMarkdown": "I compared LB scores with or without my detection model.\nWith: my pred\nWithout: all none 1 0 0 1 1", "votes": 1 } ] }, { "id": 1318168, "postDate": "2021-05-22T04:45:26.047Z", "content": "

Hey @drtausamaru just a quick question, would it be safe to assume that the images which have NO annotations can be disregarded from the training set? I am also speculating that some of the annotated images might be lower res or blurry compared to the images which are not annotated (building on your observation), do you think that might be reasonable assumption? what are your thoughts based on your observations so far?

", "rawMarkdown": "Hey @drtausamaru just a quick question, would it be safe to assume that the images which have **NO** annotations can be disregarded from the training set? I am also speculating that some of the annotated images might be lower res or blurry compared to the images which are not annotated (building on your observation), do you think that might be reasonable assumption? what are your thoughts based on your observations so far?", "votes": 1 }, { "id": 1317171, "postDate": "2021-05-21T07:37:57.653Z", "content": "

I think all the images having the same study_id have same images

", "rawMarkdown": "I think all the images having the same study_id have same images", "votes": 1, "replies": [ { "id": 1317175, "postDate": "2021-05-21T07:42:08.053Z", "content": "

Check out for study id 970c96c9fa5d, there the image is same but one is very much blurry

", "rawMarkdown": "Check out for study id 970c96c9fa5d, there the image is same but one is very much blurry" }, { "id": 1317176, "postDate": "2021-05-21T07:42:45.957Z", "content": "

Yes, but the problem is that only one of them has the BBox although all the images seem to be the same.
\nWhat's the difference between them?
\nWe cannot decide whether the image actually has opacity or not.

", "rawMarkdown": "Yes, but the problem is that only one of them has the BBox although all the images seem to be the same. \nWhat's the difference between them?\nWe cannot decide whether the image actually has opacity or not.", "votes": 2 }, { "id": 1317184, "postDate": "2021-05-21T07:51:16.080Z", "content": "

Yes, I understand what you are trying to say.
\nOne thing I am not getting is, what do images of the same StudyInstanceUID even represent? They have the same images but some of them don't have boxes as you have said.
\n@paras42 could you please explain

", "rawMarkdown": "Yes, I understand what you are trying to say.\nOne thing I am not getting is, what do images of the same StudyInstanceUID even represent? They have the same images but some of them don't have boxes as you have said. \n@paras42 could you please explain", "votes": 1 }, { "id": 1317192, "postDate": "2021-05-21T07:57:59.523Z", "content": "

Now I'm using only images with BBox for training.
\nAnd another problem is that mAP score of object detection part is 0.001 lol.
\nOf course it is possible that my model doesn't work well, but I suspect LB scoring error because other people discuss similar situation.

", "rawMarkdown": "Now I'm using only images with BBox for training.\nAnd another problem is that mAP score of object detection part is 0.001 lol.\nOf course it is possible that my model doesn't work well, but I suspect LB scoring error because other people discuss similar situation.", "votes": 3 } ] }, { "id": 1351340, "postDate": "2021-06-16T08:25:10.967Z", "content": "

Updated.
\nhttps://www.kaggle.com/c/siim-covid19-detection/discussion/246597

", "rawMarkdown": "Updated.\nhttps://www.kaggle.com/c/siim-covid19-detection/discussion/246597", "votes": 2 }, { "id": 1326388, "postDate": "2021-05-28T12:49:00.260Z", "content": "

Is data correct now? I'm asking this question to see if it's worth downloading them locally or waiting for the annotations to be fixed.

", "rawMarkdown": "Is data correct now? I'm asking this question to see if it's worth downloading them locally or waiting for the annotations to be fixed.", "replies": [ { "id": 1326400, "postDate": "2021-05-28T13:02:03.917Z", "content": "

Hi. They are investigating like the comment below.

", "rawMarkdown": "Hi. They are investigating like the comment below." }, { "id": 1326953, "postDate": "2021-05-28T18:45:33.250Z", "content": "

How do you think the problem will be solved by correcting the annotations or adding information? Or will it be solved by partially replacing the dataset? I just can't decide whether to upload this large dataset to myself locally or not))

", "rawMarkdown": "How do you think the problem will be solved by correcting the annotations or adding information? Or will it be solved by partially replacing the dataset? I just can't decide whether to upload this large dataset to myself locally or not))" } ] }, { "id": 1317967, "postDate": "2021-05-21T20:39:46.317Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1322944, "author_name": "ParasLakhani", "author_url": "", "post_date": "2021-05-25T20:10:01.227000", "content": "

We are now aware regarding some studies having multiple images (some are duplicates), some with bounding boxes, and others with not. We are investigating different approaches to remedy this issue. We will soon follow-up when we arrive at a solution and let you know.

", "votes": 14, "replies": [ { "id": 1323186, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-26T04:29:35.407000", "content": "

Thanks for the response.

", "votes": 0, "replies": [] }, { "id": 1329077, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-05-30T20:04:52.870000", "content": "

Any update on this? It’s been weeks now since this issue was first noticed?

", "votes": 1, "replies": [] }, { "id": 1329228, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-31T01:31:38.597000", "content": "

I guess it's under construction.

", "votes": 0, "replies": [] } ] }, { "id": 1312774, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-18T08:34:14.980000", "content": "

study_id: '8943d1d85097'
\nOne of them horizontal-flipped.
\n\"Screen

", "votes": 9, "replies": [ { "id": 1321063, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2021-05-24T13:55:13.677000", "content": "

Probably in case of positive studies for now its safer to only train with images with a bbox annotation and remove others.

", "votes": 6, "replies": [] }, { "id": 1321474, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-24T17:46:04.307000", "content": "

Yes, I choose that strategy now.

", "votes": 0, "replies": [] } ] }, { "id": 1321060, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2021-05-24T13:53:41.617000", "content": "

How do you calculate LB score just for image level? Perhaps leaving study preds as an empty string “”?

", "votes": 1, "replies": [ { "id": 1321472, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-24T17:45:22.520000", "content": "

I compared LB scores with or without my detection model.
\nWith: my pred
\nWithout: all none 1 0 0 1 1

", "votes": 1, "replies": [] } ] }, { "id": 1318168, "author_name": "Dave Rattan", "author_url": "", "post_date": "2021-05-22T04:45:26.047000", "content": "

Hey @drtausamaru just a quick question, would it be safe to assume that the images which have NO annotations can be disregarded from the training set? I am also speculating that some of the annotated images might be lower res or blurry compared to the images which are not annotated (building on your observation), do you think that might be reasonable assumption? what are your thoughts based on your observations so far?

", "votes": 1, "replies": [] }, { "id": 1317171, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2021-05-21T07:37:57.653000", "content": "

I think all the images having the same study_id have same images

", "votes": 1, "replies": [ { "id": 1317175, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2021-05-21T07:42:08.053000", "content": "

Check out for study id 970c96c9fa5d, there the image is same but one is very much blurry

", "votes": 0, "replies": [] }, { "id": 1317176, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-21T07:42:45.957000", "content": "

Yes, but the problem is that only one of them has the BBox although all the images seem to be the same.
\nWhat's the difference between them?
\nWe cannot decide whether the image actually has opacity or not.

", "votes": 2, "replies": [] }, { "id": 1317184, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2021-05-21T07:51:16.080000", "content": "

Yes, I understand what you are trying to say.
\nOne thing I am not getting is, what do images of the same StudyInstanceUID even represent? They have the same images but some of them don't have boxes as you have said.
\n@paras42 could you please explain

", "votes": 1, "replies": [] }, { "id": 1317192, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-21T07:57:59.523000", "content": "

Now I'm using only images with BBox for training.
\nAnd another problem is that mAP score of object detection part is 0.001 lol.
\nOf course it is possible that my model doesn't work well, but I suspect LB scoring error because other people discuss similar situation.

", "votes": 3, "replies": [] } ] }, { "id": 1351340, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-06-16T08:25:10.967000", "content": "

Updated.
\nhttps://www.kaggle.com/c/siim-covid19-detection/discussion/246597

", "votes": 2, "replies": [] }, { "id": 1326388, "author_name": "Aleksandr Emchinov", "author_url": "", "post_date": "2021-05-28T12:49:00.260000", "content": "

Is data correct now? I'm asking this question to see if it's worth downloading them locally or waiting for the annotations to be fixed.

", "votes": 0, "replies": [ { "id": 1326400, "author_name": "cool_rabbit", "author_url": "", "post_date": "2021-05-28T13:02:03.917000", "content": "

Hi. They are investigating like the comment below.

", "votes": 0, "replies": [] }, { "id": 1326953, "author_name": "Aleksandr Emchinov", "author_url": "", "post_date": "2021-05-28T18:45:33.250000", "content": "

How do you think the problem will be solved by correcting the annotations or adding information? Or will it be solved by partially replacing the dataset? I just can't decide whether to upload this large dataset to myself locally or not))

", "votes": 0, "replies": [] } ] }, { "id": 1317967, "author_name": "", "author_url": "", "post_date": "2021-05-21T20:39:46.317000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1312715": "These are all study_id: 'a7335b2f9815' and seem to be the same images, but only one of them is annotated as opacity.\nWhat does it mean?\nDifferent annotators?\nPlease describe the situation @paras42 \n\"Screen", "1322944": "We are now aware regarding some studies having multiple images (some are duplicates), some with bounding boxes, and others with not. We are investigating different approaches to remedy this issue. We will soon follow-up when we arrive at a solution and let you know.", "1312774": "study_id: '8943d1d85097'\nOne of them horizontal-flipped.\n\"Screen", "1321060": "How do you calculate LB score just for image level? Perhaps leaving study preds as an empty string “”?", "1318168": "Hey @drtausamaru just a quick question, would it be safe to assume that the images which have **NO** annotations can be disregarded from the training set? I am also speculating that some of the annotated images might be lower res or blurry compared to the images which are not annotated (building on your observation), do you think that might be reasonable assumption? what are your thoughts based on your observations so far?", "1317171": "I think all the images having the same study_id have same images", "1351340": "Updated.\nhttps://www.kaggle.com/c/siim-covid19-detection/discussion/246597", "1326388": "Is data correct now? I'm asking this question to see if it's worth downloading them locally or waiting for the annotations to be fixed.", "1317967": "" } }