I have a question for the host. In this competition, we need to put the study prediction and image prediction in submission.csv. How do you calculate the map score from this data? In particular, I have no idea if there are multiple images and multiple classes in one study. There are only two labels to put on the image level, none or opacity, right?
", "messages": [ { "id": "1314959", "postDate": "05/19/2021 13:20:47", "content": "I have a question for the host. In this competition, we need to put the study prediction and image prediction in submission.csv. How do you calculate the map score from this data? In particular, I have no idea if there are multiple images and multiple classes in one study. There are only two labels to put on the image level, none or opacity, right?
", "rawMarkdown": "I have a question for the host. In this competition, we need to put the study prediction and image prediction in submission.csv. How do you calculate the map score from this data? In particular, I have no idea if there are multiple images and multiple classes in one study. There are only two labels to put on the image level, none or opacity, right?", "votes": null }, { "id": "1343042", "postDate": "06/10/2021 01:44:54", "content": "Hello, @yujiariyasu Do you have an answer now?
", "rawMarkdown": "Hello, @yujiariyasu Do you have an answer now?", "votes": null }, { "id": "1343045", "postDate": "06/10/2021 01:47:20", "content": "It probably treats the six classes equally. But I won't be able to say for sure until the LB bug is fixed.
", "rawMarkdown": "It probably treats the six classes equally. But I won't be able to say for sure until the LB bug is fixed.", "votes": null }, { "id": "1343058", "postDate": "06/10/2021 02:05:30", "content": "What happened to LB? I submit two different submissions yesterday which only predict image-level samples and keep study-level predictions with these in original submission.csv the same. And these two submissions both only get LB 0.050 score which is equal to LB of original submission.csv. I don’t know if this phenomenon is related to the bug you mentioned.
", "rawMarkdown": "What happened to LB? I submit two different submissions yesterday which only predict image-level samples and keep study-level predictions with these in original submission.csv the same. And these two submissions both only get LB 0.050 score which is equal to LB of original submission.csv. I don’t know if this phenomenon is related to the bug you mentioned.", "votes": null }, { "id": "1343109", "postDate": "06/10/2021 03:37:51", "content": "The score of the image level is 0.001. To be precise, predicting only the none class will return a score of 0, and predicting the opacity class as well will return a score of 0.001.
", "rawMarkdown": "The score of the image level is 0.001. To be precise, predicting only the none class will return a score of 0, and predicting the opacity class as well will return a score of 0.001.", "votes": null }, { "id": "1343181", "postDate": "06/10/2021 05:09:35", "content": "Thank you for your reply!😊
", "rawMarkdown": "Thank you for your reply!😊", "votes": null }, { "id": "1351567", "postDate": "06/16/2021 12:11:15", "content": "Is it clear how the metric is calculated?
", "rawMarkdown": "Is it clear how the metric is calculated?", "votes": null }, { "id": "1353253", "postDate": "06/17/2021 02:36:39", "content": "It probably treats the six classes equally.
", "rawMarkdown": "It probably treats the six classes equally.", "votes": null }, { "id": "1353271", "postDate": "06/17/2021 02:49:29", "content": "Is there any mAP computation code in this competition released? I want to evaluate my training and validation process.
", "rawMarkdown": "Is there any mAP computation code in this competition released? I want to evaluate my training and validation process.", "votes": null }, { "id": "1353283", "postDate": "06/17/2021 03:02:50", "content": "No, but this competition uses the standard PASCAL VOC 2010. And the host probably won't publish the code, so it looks like you'll have to do the calculations yourself.
", "rawMarkdown": "No, but this competition uses the standard PASCAL VOC 2010. And the host probably won't publish the code, so it looks like you'll have to do the calculations yourself.", "votes": null } ], "comments": [ { "id": 1343042, "author_name": "shangweichen", "author_url": "", "post_date": "06/10/2021 01:44:54", "content": "Hello, @yujiariyasu Do you have an answer now?
", "votes": null, "replies": [ { "id": 1343045, "author_name": "yujiariyasu", "author_url": "", "post_date": "06/10/2021 01:47:20", "content": "It probably treats the six classes equally. But I won't be able to say for sure until the LB bug is fixed.
", "votes": null, "replies": [] }, { "id": 1343058, "author_name": "shangweichen", "author_url": "", "post_date": "06/10/2021 02:05:30", "content": "What happened to LB? I submit two different submissions yesterday which only predict image-level samples and keep study-level predictions with these in original submission.csv the same. And these two submissions both only get LB 0.050 score which is equal to LB of original submission.csv. I don’t know if this phenomenon is related to the bug you mentioned.
", "votes": null, "replies": [] }, { "id": 1343109, "author_name": "yujiariyasu", "author_url": "", "post_date": "06/10/2021 03:37:51", "content": "The score of the image level is 0.001. To be precise, predicting only the none class will return a score of 0, and predicting the opacity class as well will return a score of 0.001.
", "votes": null, "replies": [] }, { "id": 1343181, "author_name": "shangweichen", "author_url": "", "post_date": "06/10/2021 05:09:35", "content": "Thank you for your reply!😊
", "votes": null, "replies": [] } ] }, { "id": 1351567, "author_name": "greylord1996", "author_url": "", "post_date": "06/16/2021 12:11:15", "content": "Is it clear how the metric is calculated?
", "votes": null, "replies": [ { "id": 1353253, "author_name": "yujiariyasu", "author_url": "", "post_date": "06/17/2021 02:36:39", "content": "It probably treats the six classes equally.
", "votes": null, "replies": [] }, { "id": 1353271, "author_name": "shangweichen", "author_url": "", "post_date": "06/17/2021 02:49:29", "content": "Is there any mAP computation code in this competition released? I want to evaluate my training and validation process.
", "votes": null, "replies": [] }, { "id": 1353283, "author_name": "yujiariyasu", "author_url": "", "post_date": "06/17/2021 03:02:50", "content": "No, but this competition uses the standard PASCAL VOC 2010. And the host probably won't publish the code, so it looks like you'll have to do the calculations yourself.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1314959": "I have a question for the host. In this competition, we need to put the study prediction and image prediction in submission.csv. How do you calculate the map score from this data? In particular, I have no idea if there are multiple images and multiple classes in one study. There are only two labels to put on the image level, none or opacity, right?", "1343042": "Hello, @yujiariyasu Do you have an answer now?", "1343045": "It probably treats the six classes equally. But I won't be able to say for sure until the LB bug is fixed.", "1343058": "What happened to LB? I submit two different submissions yesterday which only predict image-level samples and keep study-level predictions with these in original submission.csv the same. And these two submissions both only get LB 0.050 score which is equal to LB of original submission.csv. I don’t know if this phenomenon is related to the bug you mentioned.", "1343109": "The score of the image level is 0.001. To be precise, predicting only the none class will return a score of 0, and predicting the opacity class as well will return a score of 0.001.", "1343181": "Thank you for your reply!😊", "1351567": "Is it clear how the metric is calculated?", "1353253": "It probably treats the six classes equally.", "1353271": "Is there any mAP computation code in this competition released? I want to evaluate my training and validation process.", "1353283": "No, but this competition uses the standard PASCAL VOC 2010. And the host probably won't publish the code, so it looks like you'll have to do the calculations yourself." }, "source": "meta" }