While I'm far from an expert in diagnosing diabetic retinopathy, I've read up a bit and rellevant studied medical diagrams. To my untrained eye, both these images seem to feature Exudates on a similar scale, though 1199_left is rated "0" and 1196_right is rated "3".
\nI can't seem to find any abnormalities in 1196 that are not present at a similar scale in 1199 - what am I missing?
", "messages": [ { "id": "65146", "postDate": "02/28/2015 19:38:11", "content": "While I'm far from an expert in diagnosing diabetic retinopathy, I've read up a bit and rellevant studied medical diagrams. To my untrained eye, both these images seem to feature Exudates on a similar scale, though 1199_left is rated "0" and 1196_right is rated "3".
\nI can't seem to find any abnormalities in 1196 that are not present at a similar scale in 1199 - what am I missing?
", "rawMarkdown": "", "votes": null }, { "id": "65147", "postDate": "02/28/2015 20:32:32", "content": "They did say there will be noise in the dataset, so I'm not surprised. The classifier you build should be able to deal with things like that; errors in training data, noise in images, etc...
\nFor another example of noise in the image itself, see 1986_left and _right. Both images are darker than usual and have uneven contrast, but the left one is almost entirely black!
", "rawMarkdown": "", "votes": null }, { "id": "65148", "postDate": "02/28/2015 20:41:26", "content": "I'm not so much worried about noise in the data-set as I am about misinterpreting the images myself. I'm attempting to manually construct features that will allow differentiating healthy eyes from affected ones, in order to do so I must be able to at least make a ball-park diagnosis myself and identify outliers in the data.
So in essence, my question is: am I correctly interpreting this as a misclassification, or am I missing something?
[quote=Boaz;65148]
\nSo in essence, my question is: am I correctly interpreting this as a misclassification, or am I missing something?
\n[/quote]
\nI see what you mean now.
\nThat question probably needs a trained doctor to answer with any degree of certainty.
\nI think you might be able to assume it's an error and move on, without sacrificing any accuracy, unless you find this to be a more frequent occurrence than just the one image. And if it is more frequent, then you'll have more data to do your comparison, then you might be able to see the pattern if one exists.
", "rawMarkdown": "", "votes": null }, { "id": "65150", "postDate": "02/28/2015 21:34:35", "content": "Keep in mind that drusen may look llike hard exudates to the untrained eye.
", "rawMarkdown": "", "votes": null }, { "id": "65152", "postDate": "02/28/2015 22:05:07", "content": "Jorge (or anyone else): Do you know what kappa value we might expect if we compared two expert human raters instead of human vs machine? I'm just interested to know how high a score might be possible.
", "rawMarkdown": "", "votes": null }, { "id": "65153", "postDate": "02/28/2015 22:28:16", "content": "http://www.ncbi.nlm.nih.gov/pubmed/23615341
\nEdit: The above is not strictly comparing inter/intra observer kappas, but I recall a few other sources I dug up in setting up this competition. Googling "kappa retinopathy" will lead you to a lot of them. e.g. http://www.aaojournal.org/article/S0161-6420(05)01475-2/abstract
", "rawMarkdown": "", "votes": null }, { "id": "65369", "postDate": "03/03/2015 22:54:06", "content": "Wow. The best kappa scores in those two links aren't too much higher than the top leaderboard score.
", "rawMarkdown": "", "votes": null }, { "id": "65541", "postDate": "03/05/2015 20:35:35", "content": "A recent poster listed Kappa values for primary care readers of EyePACS images and found incidentally that the values were similar for eye care providers (both optometrists and ophthalmologists). The Kappas that we found were .78 for doctors finding DR level greater than Mild, and .84 and .79 for doctors finding DR greater than Moderate. Note that these values don't include the readings of the non-eye care clinician from location C who scored significantly worse.
Attached is the poster.
Thanks for the pointer, Jorge. How does the data given to the reviewers in that study compare to the competition data? Do the extra fields tend to provide significantly greater information content?
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 65147, "author_name": "ielnabarawy", "author_url": "", "post_date": "02/28/2015 20:32:32", "content": "They did say there will be noise in the dataset, so I'm not surprised. The classifier you build should be able to deal with things like that; errors in training data, noise in images, etc...
\nFor another example of noise in the image itself, see 1986_left and _right. Both images are darker than usual and have uneven contrast, but the left one is almost entirely black!
", "votes": null, "replies": [] }, { "id": 65148, "author_name": "boazarad", "author_url": "", "post_date": "02/28/2015 20:41:26", "content": "I'm not so much worried about noise in the data-set as I am about misinterpreting the images myself. I'm attempting to manually construct features that will allow differentiating healthy eyes from affected ones, in order to do so I must be able to at least make a ball-park diagnosis myself and identify outliers in the data.
So in essence, my question is: am I correctly interpreting this as a misclassification, or am I missing something?
[quote=Boaz;65148]
\nSo in essence, my question is: am I correctly interpreting this as a misclassification, or am I missing something?
\n[/quote]
\nI see what you mean now.
\nThat question probably needs a trained doctor to answer with any degree of certainty.
\nI think you might be able to assume it's an error and move on, without sacrificing any accuracy, unless you find this to be a more frequent occurrence than just the one image. And if it is more frequent, then you'll have more data to do your comparison, then you might be able to see the pattern if one exists.
", "votes": null, "replies": [] }, { "id": 65150, "author_name": "jorge9", "author_url": "", "post_date": "02/28/2015 21:34:35", "content": "Keep in mind that drusen may look llike hard exudates to the untrained eye.
", "votes": null, "replies": [] }, { "id": 65152, "author_name": "jfkingiii", "author_url": "", "post_date": "02/28/2015 22:05:07", "content": "Jorge (or anyone else): Do you know what kappa value we might expect if we compared two expert human raters instead of human vs machine? I'm just interested to know how high a score might be possible.
", "votes": null, "replies": [] }, { "id": 65153, "author_name": "wcukierski", "author_url": "", "post_date": "02/28/2015 22:28:16", "content": "http://www.ncbi.nlm.nih.gov/pubmed/23615341
\nEdit: The above is not strictly comparing inter/intra observer kappas, but I recall a few other sources I dug up in setting up this competition. Googling "kappa retinopathy" will lead you to a lot of them. e.g. http://www.aaojournal.org/article/S0161-6420(05)01475-2/abstract
", "votes": null, "replies": [] }, { "id": 65369, "author_name": "alexcoventry", "author_url": "", "post_date": "03/03/2015 22:54:06", "content": "Wow. The best kappa scores in those two links aren't too much higher than the top leaderboard score.
", "votes": null, "replies": [] }, { "id": 65541, "author_name": "jorge9", "author_url": "", "post_date": "03/05/2015 20:35:35", "content": "A recent poster listed Kappa values for primary care readers of EyePACS images and found incidentally that the values were similar for eye care providers (both optometrists and ophthalmologists). The Kappas that we found were .78 for doctors finding DR level greater than Mild, and .84 and .79 for doctors finding DR greater than Moderate. Note that these values don't include the readings of the non-eye care clinician from location C who scored significantly worse.
Attached is the poster.
Thanks for the pointer, Jorge. How does the data given to the reviewers in that study compare to the competition data? Do the extra fields tend to provide significantly greater information content?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "65146": "", "65147": "", "65148": "", "65149": "", "65150": "", "65152": "", "65153": "", "65369": "", "65541": "", "65645": "" }, "source": "meta" }