\nThis below image is my prediction of image 2-5752 along with GT provided
![\"Detected](\"https://imgur.com/a/HzdqSr2\")
\nAs u can see my model detected 1 extra COTS in upper left part, which I thought might be a False positive after looking at GT(although confidence was very high).
Bear with me because I am new to this.
\nI wanted to address one issue I just found regarding the ground truth provided in the competition.
\nThis below image is my prediction of image 2-5752 along with GT provided
\nAs u can see my model detected 1 extra COTS in upper left part, which I thought might be a False positive after looking at GT(although confidence was very high).
After that I looked at another result of image 2-5764 along with GT
\nAgain my model detected 3 more COTS in upper left area which were not present in GT, this made me suspicious because this time 2 of those detection actually looked like COTS to me so I went ahead and tried to find the next image in this video sequence.
Which lead me to image number 2-5774.
\nAs u can see, my model was not wrong in detecting those COTS which are clearly Positive in this image but were not annotated in the GT of those previous images.
While this was training data, this can also be present in hidden test set and can lead to lower our score in the end. Any idea/suggestion to mitigate this issue? or is this common?
", "messages": [ { "id": "1625753", "postDate": "12/22/2021 06:01:39", "content": "Bear with me because I am new to this.
\nI wanted to address one issue I just found regarding the ground truth provided in the competition.
\nThis below image is my prediction of image 2-5752 along with GT provided
\nAs u can see my model detected 1 extra COTS in upper left part, which I thought might be a False positive after looking at GT(although confidence was very high).
After that I looked at another result of image 2-5764 along with GT
\nAgain my model detected 3 more COTS in upper left area which were not present in GT, this made me suspicious because this time 2 of those detection actually looked like COTS to me so I went ahead and tried to find the next image in this video sequence.
Which lead me to image number 2-5774.
\nAs u can see, my model was not wrong in detecting those COTS which are clearly Positive in this image but were not annotated in the GT of those previous images.
While this was training data, this can also be present in hidden test set and can lead to lower our score in the end. Any idea/suggestion to mitigate this issue? or is this common?
", "rawMarkdown": "Bear with me because I am new to this.\n\nI wanted to address one issue I just found regarding the ground truth provided in the competition.\nThis below image is my prediction of image 2-5752 along with GT provided \nAs u can see my model detected 1 extra COTS in upper left part, which I thought might be a False positive after looking at GT(although confidence was very high).\n\nAfter that I looked at another result of image 2-5764 along with GT \nAgain my model detected 3 more COTS in upper left area which were not present in GT, this made me suspicious because this time 2 of those detection actually looked like COTS to me so I went ahead and tried to find the next image in this video sequence.\n\nWhich lead me to image number 2-5774. \nAs u can see, my model was not wrong in detecting those COTS which are clearly Positive in this image but were not annotated in the GT of those previous images.\n\nWhile this was training data, this can also be present in hidden test set and can lead to lower our score in the end. Any idea/suggestion to mitigate this issue? or is this common?", "votes": null }, { "id": "1625833", "postDate": "12/22/2021 07:16:06", "content": "Yes, we have the same situation. I generated videos from validation dataset and we found frames without labeling. Our detector is able to find it earlier then gt box appeares.
", "rawMarkdown": "Yes, we have the same situation. I generated videos from validation dataset and we found frames without labeling. Our detector is able to find it earlier then gt box appeares.", "votes": null }, { "id": "1625851", "postDate": "12/22/2021 07:46:38", "content": "I assume there is no way to go around it and just hope there are not much instance of this in test data, because otherwise its like punishing for being too accurate.
", "rawMarkdown": "I assume there is no way to go around it and just hope there are not much instance of this in test data, because otherwise its like punishing for being too accurate.", "votes": null }, { "id": "1625942", "postDate": "12/22/2021 10:05:48", "content": "Can kaggle staffs comment on these? please? More clarity would be helpful!!
", "rawMarkdown": "Can kaggle staffs comment on these? please? More clarity would be helpful!!", "votes": null }, { "id": "1625943", "postDate": "12/22/2021 10:08:05", "content": "Yes … wwe have no influence on the test set. Here it is evident that some starfishes are not labeled … which of course affects the f2 score.
", "rawMarkdown": "Yes ... wwe have no influence on the test set. Here it is evident that some starfishes are not labeled ... which of course affects the f2 score.", "votes": null }, { "id": "1628629", "postDate": "12/25/2021 05:37:12", "content": "I also observe it. The detector detects the COTS before gt box appears at the same location in video. Either they fix the data, or we try to \"overfit\" the dataset by filter dectection too far away?
", "rawMarkdown": "I also observe it. The detector detects the COTS before gt box appears at the same location in video. Either they fix the data, or we try to \"overfit\" the dataset by filter dectection too far away?", "votes": null } ], "comments": [ { "id": 1625833, "author_name": "remekkinas", "author_url": "", "post_date": "12/22/2021 07:16:06", "content": "Yes, we have the same situation. I generated videos from validation dataset and we found frames without labeling. Our detector is able to find it earlier then gt box appeares.
", "votes": null, "replies": [ { "id": 1625851, "author_name": "keshhere", "author_url": "", "post_date": "12/22/2021 07:46:38", "content": "I assume there is no way to go around it and just hope there are not much instance of this in test data, because otherwise its like punishing for being too accurate.
", "votes": null, "replies": [] }, { "id": 1625943, "author_name": "remekkinas", "author_url": "", "post_date": "12/22/2021 10:08:05", "content": "Yes … wwe have no influence on the test set. Here it is evident that some starfishes are not labeled … which of course affects the f2 score.
", "votes": null, "replies": [] } ] }, { "id": 1625942, "author_name": "darshangovindaraj", "author_url": "", "post_date": "12/22/2021 10:05:48", "content": "Can kaggle staffs comment on these? please? More clarity would be helpful!!
", "votes": null, "replies": [] }, { "id": 1628629, "author_name": "sunkuangyuan", "author_url": "", "post_date": "12/25/2021 05:37:12", "content": "I also observe it. The detector detects the COTS before gt box appears at the same location in video. Either they fix the data, or we try to \"overfit\" the dataset by filter dectection too far away?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1625753": "Bear with me because I am new to this.\n\nI wanted to address one issue I just found regarding the ground truth provided in the competition.\nThis below image is my prediction of image 2-5752 along with GT provided \nAs u can see my model detected 1 extra COTS in upper left part, which I thought might be a False positive after looking at GT(although confidence was very high).\n\nAfter that I looked at another result of image 2-5764 along with GT \nAgain my model detected 3 more COTS in upper left area which were not present in GT, this made me suspicious because this time 2 of those detection actually looked like COTS to me so I went ahead and tried to find the next image in this video sequence.\n\nWhich lead me to image number 2-5774. \nAs u can see, my model was not wrong in detecting those COTS which are clearly Positive in this image but were not annotated in the GT of those previous images.\n\nWhile this was training data, this can also be present in hidden test set and can lead to lower our score in the end. Any idea/suggestion to mitigate this issue? or is this common?", "1625833": "Yes, we have the same situation. I generated videos from validation dataset and we found frames without labeling. Our detector is able to find it earlier then gt box appeares.", "1625851": "I assume there is no way to go around it and just hope there are not much instance of this in test data, because otherwise its like punishing for being too accurate.", "1625942": "Can kaggle staffs comment on these? please? More clarity would be helpful!!", "1625943": "Yes ... wwe have no influence on the test set. Here it is evident that some starfishes are not labeled ... which of course affects the f2 score.", "1628629": "I also observe it. The detector detects the COTS before gt box appears at the same location in video. Either they fix the data, or we try to \"overfit\" the dataset by filter dectection too far away?" }, "source": "meta" }![\"Detected](\"https://imgur.com/a/q0DZwn8\")
![\"Detection](\"https://imgur.com/a/hk1zipu\")