\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F349155%2F0cc9f9b13fb17cb0baf9c23a25ffa733%2FScreenshot%20from%202020-12-20%2011-46-28.png?generation=1608457721208799&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F349155%2Fd4d126293797bfc52b0770d6284789b5%2FScreenshot%20from%202020-12-20%2011-51-34.png?generation=1608457951000976&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F349155%2F8a3e64d4ac55578268f23b2d41e7c02b%2FScreenshot%20from%202020-12-20%2012-15-46.png?generation=1608459432127121&alt=media\")
So I was looking at parts where my validation fails miserably and found these particular masks. I assume it slipped there erroneously or am I wrong? There are also lots of inconsistencies in how masks are labeled, but there were numerous discussions about that.
\n
\n
\n
P.S. I'm also pretty sure that most of the predicted masks are of higher quality than the ground truth masks.
", "messages": [ { "id": 1119689, "postDate": "2020-12-20T09:52:34.010Z", "content": "So I was looking at parts where my validation fails miserably and found these particular masks. I assume it slipped there erroneously or am I wrong? There are also lots of inconsistencies in how masks are labeled, but there were numerous discussions about that.
\n
\n
\n
P.S. I'm also pretty sure that most of the predicted masks are of higher quality than the ground truth masks.
", "rawMarkdown": "So I was looking at parts where my validation fails miserably and found these particular masks. I assume it slipped there erroneously or am I wrong? There are also lots of inconsistencies in how masks are labeled, but there were numerous discussions about that. \n\n\n\n\nP.S. I'm also pretty sure that most of the predicted masks are of higher quality than the ground truth masks. \n\n", "votes": 17 }, { "id": 1120137, "postDate": "2020-12-20T15:42:17.503Z", "content": "Also, there are masks in the empty region. See a couple of examples from 1e24xxxx image (red![\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F349155%2F1657fc7d1891c7c85ac54f735a433afd%2FScreenshot%20from%202020-12-20%2017-39-29.png?generation=1608478871989032&alt=media\")
\nAnd here is the mask definition
\n {
\n \"type\": \"Feature\",
\n \"id\": \"PathDetectionObject\",
\n \"geometry\": {
\n \"type\": \"Polygon\",
\n \"coordinates\": [
\n [
\n [28688, 17466],
\n [28673, 17487],
\n [28600, 17580],
\n [28596, 17600],
\n [28591, 17608],
\n [28592, 17620],
\n [28586, 17654],
\n [28602, 17727],
\n [28604, 17730],
\n [28605, 17733],
\n [28638, 17769],
\n [28653, 17787],
\n [28762, 17838],
\n [28905, 17880],
\n [28944, 17871],
\n [29006, 17775],
\n [29015, 17723],
\n [29017, 17721],
\n [29016, 17717],
\n [29019, 17707],
\n [29016, 17679],
\n [29015, 17634],
\n [29009, 17624],
\n [29009, 17615],
\n [28973, 17549],
\n [28971, 17547],
\n [28970, 17545],
\n [28887, 17466],
\n [28688, 17466]
\n ]
\n ]
\n },
\n \"properties\": {
\n \"classification\": {
\n \"name\": \"glomerulus\",
\n \"colorRGB\": -3140401
\n },
\n \"isLocked\": false,
\n \"measurements\": [
\n {
\n \"name\": \"detection_score\",
\n \"value\": 0.9950796365737915
\n }
\n ]
\n }
\n },
i am quite sure that the labeling processing involves some semi-automatic labeling software.
\nHence these errors may be correlated (i.e. it may happen? in the public/private test images)
especially in the second example you have shown, the annotation cut across the white area. it seems like segmentation model results. it doesn't seem to be human annotation.
", "rawMarkdown": "i am quite sure that the labeling processing involves some semi-automatic labeling software.\nHence these errors may be correlated (i.e. it may happen? in the public/private test images)\n\nespecially in the second example you have shown, the annotation cut across the white area. it seems like segmentation model results. it doesn't seem to be human annotation.", "votes": 5, "replies": [ { "id": 1120122, "postDate": "2020-12-20T15:25:48.927Z", "content": "Yep, it definitelyy looks as if some sort of active learning/semi-supervised learning was used. Also, some masks have sharp edges. This happens when a non-overlapping sliding window cuts most part of the mask.
", "rawMarkdown": "Yep, it definitelyy looks as if some sort of active learning/semi-supervised learning was used. Also, some masks have sharp edges. This happens when a non-overlapping sliding window cuts most part of the mask. ", "votes": 1 }, { "id": 1121024, "postDate": "2020-12-21T09:43:24.303Z", "content": "Well, it is described in the competition somewhere that the masks where obtained using a DL approach. The later was then manually corrected. For instance \"\"value\": 0.9950796365737915\" describes the confidence score of the region obtained from the DL segmentation approach.
\nFor me it is unclear what the goal of the competition is. Developping better models, but benchmarking based on a unperfect one + some not so good correction…
", "rawMarkdown": "Well, it is described in the competition somewhere that the masks where obtained using a DL approach. The later was then manually corrected. For instance \"\"value\": 0.9950796365737915\" describes the confidence score of the region obtained from the DL segmentation approach. \n\nFor me it is unclear what the goal of the competition is. Developping better models, but benchmarking based on a unperfect one + some not so good correction...", "votes": 4 } ] }, { "id": 1121364, "postDate": "2020-12-21T15:36:27.160Z", "content": "All, again, please be patient. We are actively addressing these issues, but it takes time. Thank you.
", "rawMarkdown": "All, again, please be patient. We are actively addressing these issues, but it takes time. Thank you.", "votes": 6 }, { "id": 1120704, "postDate": "2020-12-21T04:10:13.507Z", "content": "The ground-truth masks are indeed not precise and borders are very sharp and very rough.
\n![\"sample\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4497523%2Fc330d32632eef51ec7c87a9fca81d44a%2Fdownload.png?generation=1608523696297972&alt=media\")
my predictions look more realistic than the ground-truth.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4497523%2Fff3b24bd05ba8578da89ebc65d27703b%2Fdownload%20(3).png?generation=1608565448353878&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4497523%2Feac841f03a597f022fe2a230609770f9%2Fdownload%20(1).png?generation=1608565424080387&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4497523%2F2230ec0cbca77d93fd87e4f0a1c54d07%2Fdownload%20(2).png?generation=1608565436525902&alt=media\")
I also found several clearly erroneous masks
\ne79de561c, top , bottom
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489375%2F088b773d45ba6a592cb044ba5747c9cb%2Fe79de561c_top.png?generation=1608561888251633&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489375%2Fd53d1ea49f9f8905f27d84ae6451ba4d%2Fe79de561c_center.png?generation=1608562125787054&alt=media\")
1e2425f28 in the bottom left
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489375%2Fd48d0d8a9d9027bb209b4eb0294e2920%2F1e2425f28_bottom_left.png?generation=1608561960130569&alt=media\")
54f2eec69, top
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489375%2F196aee55020975d31603f3a9c3ac83f4%2F54f2eec69_top.png?generation=1608561993622904&alt=media\")
095bf7a1f, top, bottom
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489375%2Fd830b31de40d0d41ed8299ded654a75a%2F095bf7a1f_top.png?generation=1608562012764672&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1489375%2Fb0c25e3ac5c206ca68a51027a7247ec7%2F095bf7a1f_lower.png?generation=1608562039260318&alt=media\")
I have similar problems
", "rawMarkdown": "I have similar problems", "votes": 1 } ] } ], "comments": [ { "id": 1120137, "author_name": "DennisSakva", "author_url": "", "post_date": "2020-12-20T15:42:17.503000", "content": "Also, there are masks in the empty region. See a couple of examples from 1e24xxxx image (red
\nAnd here is the mask definition
\n {
\n \"type\": \"Feature\",
\n \"id\": \"PathDetectionObject\",
\n \"geometry\": {
\n \"type\": \"Polygon\",
\n \"coordinates\": [
\n [
\n [28688, 17466],
\n [28673, 17487],
\n [28600, 17580],
\n [28596, 17600],
\n [28591, 17608],
\n [28592, 17620],
\n [28586, 17654],
\n [28602, 17727],
\n [28604, 17730],
\n [28605, 17733],
\n [28638, 17769],
\n [28653, 17787],
\n [28762, 17838],
\n [28905, 17880],
\n [28944, 17871],
\n [29006, 17775],
\n [29015, 17723],
\n [29017, 17721],
\n [29016, 17717],
\n [29019, 17707],
\n [29016, 17679],
\n [29015, 17634],
\n [29009, 17624],
\n [29009, 17615],
\n [28973, 17549],
\n [28971, 17547],
\n [28970, 17545],
\n [28887, 17466],
\n [28688, 17466]
\n ]
\n ]
\n },
\n \"properties\": {
\n \"classification\": {
\n \"name\": \"glomerulus\",
\n \"colorRGB\": -3140401
\n },
\n \"isLocked\": false,
\n \"measurements\": [
\n {
\n \"name\": \"detection_score\",
\n \"value\": 0.9950796365737915
\n }
\n ]
\n }
\n },
i am quite sure that the labeling processing involves some semi-automatic labeling software.
\nHence these errors may be correlated (i.e. it may happen? in the public/private test images)
especially in the second example you have shown, the annotation cut across the white area. it seems like segmentation model results. it doesn't seem to be human annotation.
", "votes": 5, "replies": [ { "id": 1120122, "author_name": "DennisSakva", "author_url": "", "post_date": "2020-12-20T15:25:48.927000", "content": "Yep, it definitelyy looks as if some sort of active learning/semi-supervised learning was used. Also, some masks have sharp edges. This happens when a non-overlapping sliding window cuts most part of the mask.
", "votes": 1, "replies": [] }, { "id": 1121024, "author_name": "jeandebleau", "author_url": "", "post_date": "2020-12-21T09:43:24.303000", "content": "Well, it is described in the competition somewhere that the masks where obtained using a DL approach. The later was then manually corrected. For instance \"\"value\": 0.9950796365737915\" describes the confidence score of the region obtained from the DL segmentation approach.
\nFor me it is unclear what the goal of the competition is. Developping better models, but benchmarking based on a unperfect one + some not so good correction…
", "votes": 4, "replies": [] } ] }, { "id": 1121364, "author_name": "Leah Godwin", "author_url": "", "post_date": "2020-12-21T15:36:27.160000", "content": "All, again, please be patient. We are actively addressing these issues, but it takes time. Thank you.
", "votes": 6, "replies": [] }, { "id": 1120704, "author_name": "Aramis Vesal", "author_url": "", "post_date": "2020-12-21T04:10:13.507000", "content": "The ground-truth masks are indeed not precise and borders are very sharp and very rough.
\n
my predictions look more realistic than the ground-truth.
\n
\n
\n
I also found several clearly erroneous masks
\ne79de561c, top , bottom
\n
\n
1e2425f28 in the bottom left
\n
54f2eec69, top
\n
095bf7a1f, top, bottom
\n
I have similar problems
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1119689": "So I was looking at parts where my validation fails miserably and found these particular masks. I assume it slipped there erroneously or am I wrong? There are also lots of inconsistencies in how masks are labeled, but there were numerous discussions about that. \n\n\n\n\nP.S. I'm also pretty sure that most of the predicted masks are of higher quality than the ground truth masks. \n\n", "1120137": "Also, there are masks in the empty region. See a couple of examples from 1e24xxxx image (red\nAnd here is the mask definition\n {\n \"type\": \"Feature\",\n \"id\": \"PathDetectionObject\",\n \"geometry\": {\n \"type\": \"Polygon\",\n \"coordinates\": [\n [\n [28688, 17466],\n [28673, 17487],\n [28600, 17580],\n [28596, 17600],\n [28591, 17608],\n [28592, 17620],\n [28586, 17654],\n [28602, 17727],\n [28604, 17730],\n [28605, 17733],\n [28638, 17769],\n [28653, 17787],\n [28762, 17838],\n [28905, 17880],\n [28944, 17871],\n [29006, 17775],\n [29015, 17723],\n [29017, 17721],\n [29016, 17717],\n [29019, 17707],\n [29016, 17679],\n [29015, 17634],\n [29009, 17624],\n [29009, 17615],\n [28973, 17549],\n [28971, 17547],\n [28970, 17545],\n [28887, 17466],\n [28688, 17466]\n ]\n ]\n },\n \"properties\": {\n \"classification\": {\n \"name\": \"glomerulus\",\n \"colorRGB\": -3140401\n },\n \"isLocked\": false,\n \"measurements\": [\n {\n \"name\": \"detection_score\",\n \"value\": 0.9950796365737915\n }\n ]\n }\n },", "1119733": "i am quite sure that the labeling processing involves some semi-automatic labeling software.\nHence these errors may be correlated (i.e. it may happen? in the public/private test images)\n\nespecially in the second example you have shown, the annotation cut across the white area. it seems like segmentation model results. it doesn't seem to be human annotation.", "1121364": "All, again, please be patient. We are actively addressing these issues, but it takes time. Thank you.", "1120704": "The ground-truth masks are indeed not precise and borders are very sharp and very rough. \n\n", "1121319": "I also found several clearly erroneous masks\ne79de561c, top , bottom\n\n\n\n1e2425f28 in the bottom left\n\n\n54f2eec69, top\n\n\n095bf7a1f, top, bottom\n\n\n", "1119800": "" } }