I am interested in the uniformity of approaches taken by those who have posted here. Every one used a convolutional neural network. A quick read indicated that all used the same python based development tools.
\n\nYet if you search the very large literature on diabetic retinopathy detection almost all references use what I would call classical computer vision techniques: preprocessing; localization and segmentation of the optic disk; segmentation of the retinal vasculature; localization of the macula and fovea; localization and segmentation of retinopathy.
\n\nI am curious the reason for this divergence in approaches. Is it because the convnet approach is so much superior? Did anyone use the "classical" approach and not report it? Is anyone aware of references where the two approaches are compared quantitatively?
", "messages": [ { "id": "89043", "postDate": "08/10/2015 17:23:14", "content": "I am interested in the uniformity of approaches taken by those who have posted here. Every one used a convolutional neural network. A quick read indicated that all used the same python based development tools.
\n\nYet if you search the very large literature on diabetic retinopathy detection almost all references use what I would call classical computer vision techniques: preprocessing; localization and segmentation of the optic disk; segmentation of the retinal vasculature; localization of the macula and fovea; localization and segmentation of retinopathy.
\n\nI am curious the reason for this divergence in approaches. Is it because the convnet approach is so much superior? Did anyone use the "classical" approach and not report it? Is anyone aware of references where the two approaches are compared quantitatively?
", "rawMarkdown": "I am interested in the uniformity of approaches taken by those who have posted here. Every one used a convolutional neural network. A quick read indicated that all used the same python based development tools.\r\n\r\nYet if you search the very large literature on diabetic retinopathy detection almost all references use what I would call classical computer vision techniques: preprocessing; localization and segmentation of the optic disk; segmentation of the retinal vasculature; localization of the macula and fovea; localization and segmentation of retinopathy.\r\n\r\nI am curious the reason for this divergence in approaches. Is it because the convnet approach is so much superior? Did anyone use the \"classical\" approach and not report it? Is anyone aware of references where the two approaches are compared quantitatively?", "votes": null }, { "id": "89044", "postDate": "08/10/2015 17:30:15", "content": "Great question. I noticed on one of the competitors posted confusion matrix that the accuracy on type 1 lesions (micro-aneurisms) was pretty poor. Intuitively very small features like those would be harder for convnets to detect and might favor CV techniques.
", "rawMarkdown": "Great question. I noticed on one of the competitors posted confusion matrix that the accuracy on type 1 lesions (micro-aneurisms) was pretty poor. Intuitively very small features like those would be harder for convnets to detect and might favor CV techniques.", "votes": null }, { "id": "89095", "postDate": "08/11/2015 11:31:54", "content": "This is a recurring theme with image classification Kaggle competitions. Convnets are the norm now in CV, but a lot of the more specific application areas haven't really caught up yet.
", "rawMarkdown": "This is a recurring theme with image classification Kaggle competitions. Convnets are the norm now in CV, but a lot of the more specific application areas haven't really caught up yet.", "votes": null }, { "id": "89101", "postDate": "08/11/2015 13:43:16", "content": "I started with a "classical" approach but using a convnet as classifier.\nMy assumption was that such large images needed to be scaled down to such small sizes that small symptoms like anneurisms are missed by the convnet. So I wanted to make a sliding window convnet at the full resolution and detect symptoms and landmark spot like in the literature.
\n\nI spent much time on that approach but in the end it just gave me a small benefit in the competition.
\nHowever.. in a real life scenario I think it has benefits! You can show the doctor/patient where you see things..
Here are some conclusions..
\nI could have gotten to about Kappa 75 without a black box convnet..
@Harold..\nI used the confusion matrix as a guide.
\nHowever.. by eyeballing I could not really make sense of the labeling.. Some eye with only one anneurism would get a 1 or even a 2. And sometimes an eye that features an absolute bloodbath got a 1 or a 0.
That is one reason I think that a end-to-end machine learning approach was better..
\nMachine learning finds the best distribution to get the best score given the doctor's labels. However.. this does not mean you have the best classifier in absolute terms.
@Julian,
\n\nThanks for posting the confusion matrix, by the way.
\n\nI tried to classify a small set of images visually between 0s and 1s by looking for microaneurysms. I misclassified most of them.
\n\nI'm wondering if the successful convnets were actually detecting microaneurysms or what?
\n\nIt would be interesting to see each teams' accuracy by training class.
", "rawMarkdown": "Julian,\r\n\r\nThanks for posting the confusion matrix, by the way.\r\n\r\nI tried to classify a small set of images visually between 0s and 1s by looking for microaneurysms. I misclassified most of them. \r\n\r\nI'm wondering if the successful convnets were actually detecting microaneurysms or what?\r\n\r\nIt would be interesting to see each teams' accuracy by training class.", "votes": null }, { "id": "89145", "postDate": "08/12/2015 01:12:08", "content": "You will a lot of disagreement with that statement from researchers in computer vision including me. Convnets are just one tool among many in CV that may or may not be the best choice depending on the application. For example, as Julian points out, straightforward convnets are not the best choice if you want to locate retinal pathology to report to the doctor. Also, convnets take a lot of data to train and as we have seen in the competition, in medical imaging high quality data with a good gold standard are expensive and difficult to obtain. As an aside, a major problem in gathering data for medical applications is getting Institutional Review Boards to sign off on giving you access to the data without getting signed informed consent from each individual patient.
\n\nCan you cite any studies to back up your claim that convnets are the "norm in CV"?
", "rawMarkdown": "sedielem: \" Convnets are the norm now in CV, but a lot of the more specific application areas haven't really caught up yet.\"\r\n-------\r\nYou will a lot of disagreement with that statement from researchers in computer vision including me. Convnets are just one tool among many in CV that may or may not be the best choice depending on the application. For example, as Julian points out, straightforward convnets are not the best choice if you want to locate retinal pathology to report to the doctor. Also, convnets take a lot of data to train and as we have seen in the competition, in medical imaging high quality data with a good gold standard are expensive and difficult to obtain. As an aside, a major problem in gathering data for medical applications is getting Institutional Review Boards to sign off on giving you access to the data without getting signed informed consent from each individual patient. \r\n\r\nCan you cite any studies to back up your claim that convnets are the \"norm in CV\"?", "votes": null }, { "id": "89147", "postDate": "08/12/2015 01:43:28", "content": "Convet can be used to detect the anneurisms. The approach is not too different from this work (http://people.idsia.ch/~juergen/deeplearningwinsMICCAIgrandchallenge.html). We tried used Convnet to detect anneurisms using just a handful of hand labeled images. In my view, the result was already better than most of the published traditional approaches. We appended the image with detected anneurisms highlighted as an additional channel to the input image and tried to train another convnet. It clearly helped quite a lot to tiny input images, but did not really help much with large images. I suspect large net was able to learn to identify anneurism.
\n\nIf traditional approaches meet your need, you certainly do not need waste your energy to try something new. If they do not meet your need, you should really consider using convnet. You will be surprised how much it can learn. For good quality images, you may not need too many images to train a good classifier. We trained good performing nets with just several hundred images.
", "rawMarkdown": "Convet can be used to detect the anneurisms. The approach is not too different from this work (http://people.idsia.ch/~juergen/deeplearningwinsMICCAIgrandchallenge.html). We tried used Convnet to detect anneurisms using just a handful of hand labeled images. In my view, the result was already better than most of the published traditional approaches. We appended the image with detected anneurisms highlighted as an additional channel to the input image and tried to train another convnet. It clearly helped quite a lot to tiny input images, but did not really help much with large images. I suspect large net was able to learn to identify anneurism. \r\n\r\nIf traditional approaches meet your need, you certainly do not need waste your energy to try something new. If they do not meet your need, you should really consider using convnet. You will be surprised how much it can learn. For good quality images, you may not need too many images to train a good classifier. We trained good performing nets with just several hundred images.", "votes": null } ], "comments": [ { "id": 89044, "author_name": "haroldsquid", "author_url": "", "post_date": "08/10/2015 17:30:15", "content": "Great question. I noticed on one of the competitors posted confusion matrix that the accuracy on type 1 lesions (micro-aneurisms) was pretty poor. Intuitively very small features like those would be harder for convnets to detect and might favor CV techniques.
", "votes": null, "replies": [] }, { "id": 89095, "author_name": "sedielem", "author_url": "", "post_date": "08/11/2015 11:31:54", "content": "This is a recurring theme with image classification Kaggle competitions. Convnets are the norm now in CV, but a lot of the more specific application areas haven't really caught up yet.
", "votes": null, "replies": [] }, { "id": 89101, "author_name": "juliandewit", "author_url": "", "post_date": "08/11/2015 13:43:16", "content": "I started with a "classical" approach but using a convnet as classifier.\nMy assumption was that such large images needed to be scaled down to such small sizes that small symptoms like anneurisms are missed by the convnet. So I wanted to make a sliding window convnet at the full resolution and detect symptoms and landmark spot like in the literature.
\n\nI spent much time on that approach but in the end it just gave me a small benefit in the competition.
\nHowever.. in a real life scenario I think it has benefits! You can show the doctor/patient where you see things..
Here are some conclusions..
\nI could have gotten to about Kappa 75 without a black box convnet..
@Harold..\nI used the confusion matrix as a guide.
\nHowever.. by eyeballing I could not really make sense of the labeling.. Some eye with only one anneurism would get a 1 or even a 2. And sometimes an eye that features an absolute bloodbath got a 1 or a 0.
That is one reason I think that a end-to-end machine learning approach was better..
\nMachine learning finds the best distribution to get the best score given the doctor's labels. However.. this does not mean you have the best classifier in absolute terms.
@Julian,
\n\nThanks for posting the confusion matrix, by the way.
\n\nI tried to classify a small set of images visually between 0s and 1s by looking for microaneurysms. I misclassified most of them.
\n\nI'm wondering if the successful convnets were actually detecting microaneurysms or what?
\n\nIt would be interesting to see each teams' accuracy by training class.
", "votes": null, "replies": [] }, { "id": 89145, "author_name": "bobalv", "author_url": "", "post_date": "08/12/2015 01:12:08", "content": "You will a lot of disagreement with that statement from researchers in computer vision including me. Convnets are just one tool among many in CV that may or may not be the best choice depending on the application. For example, as Julian points out, straightforward convnets are not the best choice if you want to locate retinal pathology to report to the doctor. Also, convnets take a lot of data to train and as we have seen in the competition, in medical imaging high quality data with a good gold standard are expensive and difficult to obtain. As an aside, a major problem in gathering data for medical applications is getting Institutional Review Boards to sign off on giving you access to the data without getting signed informed consent from each individual patient.
\n\nCan you cite any studies to back up your claim that convnets are the "norm in CV"?
", "votes": null, "replies": [] }, { "id": 89147, "author_name": "pathbinder", "author_url": "", "post_date": "08/12/2015 01:43:28", "content": "Convet can be used to detect the anneurisms. The approach is not too different from this work (http://people.idsia.ch/~juergen/deeplearningwinsMICCAIgrandchallenge.html). We tried used Convnet to detect anneurisms using just a handful of hand labeled images. In my view, the result was already better than most of the published traditional approaches. We appended the image with detected anneurisms highlighted as an additional channel to the input image and tried to train another convnet. It clearly helped quite a lot to tiny input images, but did not really help much with large images. I suspect large net was able to learn to identify anneurism.
\n\nIf traditional approaches meet your need, you certainly do not need waste your energy to try something new. If they do not meet your need, you should really consider using convnet. You will be surprised how much it can learn. For good quality images, you may not need too many images to train a good classifier. We trained good performing nets with just several hundred images.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "89043": "I am interested in the uniformity of approaches taken by those who have posted here. Every one used a convolutional neural network. A quick read indicated that all used the same python based development tools.\r\n\r\nYet if you search the very large literature on diabetic retinopathy detection almost all references use what I would call classical computer vision techniques: preprocessing; localization and segmentation of the optic disk; segmentation of the retinal vasculature; localization of the macula and fovea; localization and segmentation of retinopathy.\r\n\r\nI am curious the reason for this divergence in approaches. Is it because the convnet approach is so much superior? Did anyone use the \"classical\" approach and not report it? Is anyone aware of references where the two approaches are compared quantitatively?", "89044": "Great question. I noticed on one of the competitors posted confusion matrix that the accuracy on type 1 lesions (micro-aneurisms) was pretty poor. Intuitively very small features like those would be harder for convnets to detect and might favor CV techniques.", "89095": "This is a recurring theme with image classification Kaggle competitions. Convnets are the norm now in CV, but a lot of the more specific application areas haven't really caught up yet.", "89101": "I started with a \"classical\" approach but using a convnet as classifier.\r\nMy assumption was that such large images needed to be scaled down to such small sizes that small symptoms like anneurisms are missed by the convnet. So I wanted to make a sliding window convnet at the full resolution and detect symptoms and landmark spot like in the literature.\r\n\r\nI spent much time on that approach but in the end it just gave me a small benefit in the competition.