Hello,
\n\nAs the competition progressed I became more and more convinced that automatic screening would really be very helpful.
\n\nI've seen many cases where I'd think that the doctor made mistake.\nOf course I've also seen cases where the computer missed something.\nHowever I really do think both might complement eachother.
\n\nMy idea was to make some kind of collaboration site where we can discuss every eye in detail.
\n\nTo bad I could not find the time to really build it.\nHowever.. I did put a starter on github with a live confusion matrix.
\n\nIf you want to collaborate.. please join the project.\nBut perhaps to also attract doctors the site should be a but more user friendly.\nSo if anyone has ideas on how to do this.. Be my guest!
\n\n", "messages": [ { "id": "87245", "postDate": "07/28/2015 00:25:40", "content": "Hello,
\n\nAs the competition progressed I became more and more convinced that automatic screening would really be very helpful.
\n\nI've seen many cases where I'd think that the doctor made mistake.\nOf course I've also seen cases where the computer missed something.\nHowever I really do think both might complement eachother.
\n\nMy idea was to make some kind of collaboration site where we can discuss every eye in detail.
\n\nTo bad I could not find the time to really build it.\nHowever.. I did put a starter on github with a live confusion matrix.
\n\nIf you want to collaborate.. please join the project.\nBut perhaps to also attract doctors the site should be a but more user friendly.\nSo if anyone has ideas on how to do this.. Be my guest!
\n\n", "rawMarkdown": "Hello,\r\n\r\nAs the competition progressed I became more and more convinced that automatic screening would really be very helpful. \r\n\r\nI've seen many cases where I'd think that the doctor made mistake.\r\nOf course I've also seen cases where the computer missed something.\r\nHowever I really do think both might complement eachother.\r\n\r\nMy idea was to make some kind of collaboration site where we can discuss every eye in detail.\r\n\r\nTo bad I could not find the time to really build it.\r\nHowever.. I did put a starter on github with a live confusion matrix.\r\n\r\nIf you want to collaborate.. please join the project.\r\nBut perhaps to also attract doctors the site should be a but more user friendly.\r\nSo if anyone has ideas on how to do this.. Be my guest!\r\n\r\n[live confusion matrix][1]\r\n\r\n\r\n [1]: https://github.com/juliandewit/kaggle_retinopathy", "votes": null }, { "id": "87283", "postDate": "07/28/2015 09:15:42", "content": "Julian, congratulations on your outstanding result!
\n\nHow were you able to determine which doctors rated which images?
\n\nMy approach was very straightforward. I first extracted features by giving the first convnet the difficult task of first categorizing a balanced dataset (same number of samples of each class) based on a 227x227 random patch of the 872x872 image using SoftmaxWithLoss . The features extracted in this way were then fed to my phase 2 network which added 2 extra conv layers and used an EuclideanLoss layer and an unbalanced dataset (to approximate the quadratic kappa). That gave me ~0.75 right off the bat.
\n\nHowever the net refused to classify any images as 4, so I was stuck at that score. Perhaps this has something to do with the difference in how the doctors were classifying the image.
\n\nAnyway I still have a lot to learn and have had fun with this my 3rd Kaggle competition. See you at the top one day! :)
", "rawMarkdown": "Julian, congratulations on your outstanding result! \r\n\r\nHow were you able to determine which doctors rated which images?\r\n\r\nMy approach was very straightforward. I first extracted features by giving the first convnet the difficult task of first categorizing a balanced dataset (same number of samples of each class) based on a 227x227 random patch of the 872x872 image using SoftmaxWithLoss . The features extracted in this way were then fed to my phase 2 network which added 2 extra conv layers and used an EuclideanLoss layer and an unbalanced dataset (to approximate the quadratic kappa). That gave me ~0.75 right off the bat. \r\n\r\nHowever the net refused to classify any images as 4, so I was stuck at that score. Perhaps this has something to do with the difference in how the doctors were classifying the image.\r\n\r\nAnyway I still have a lot to learn and have had fun with this my 3rd Kaggle competition. See you at the top one day! :)", "votes": null }, { "id": "87284", "postDate": "07/28/2015 09:27:30", "content": "Thanks!
\nDoctor 0 means : The doctor(s) gave this eye a '0'.
\nDoctor 1 means : The doctor(s) gave this eye a '1'.
\nThe matrix allows you to zoom in quickly on parts where the computer and doctors disagree.
For instance: Doctor 4, Algo 0.
\nThat brings you to this list:\nList of Doctor labeled 4, Algo predicted 0 eyes
\nDecide for yourself who is right.. :)
Omg I tried your approach too but only with a simple classifier on top.
\nI ditched the results.. But perhaps I should have built a NN on top..
[quote=Julian de Wit;87284]
\n\nHello..\nDoctor 0 means : The doctor(s) gave this eye a '0'.\nDoctor 1 means : The doctor(s) gave this eye a '1'.\nThe matrix allows you to zoom in quickly on parts where the computer and doctors disagree.\nFor instance: Doctor 4, Pred 0.
\n\nOmg.. I tried your approach too but only with a simple classifier on top.\nI ditched the results.. But perhaps I should have built a NN on top..
\n\n[/quote]
\n\nOk got it.
\n\nI actually had 3 fully connected layers fc6, f7,fc8 at the top of both nets. I simply took the last pooling layer of the first net for my features into the second net. This takes advantage of the fact that, in a convnet, the weight matrix stays the same size regardless of input image size. So now I could replace my 227x227 images with 872x872. :) If I had a bigger, faster gpu card I may have used a single net.
", "rawMarkdown": "[quote=Julian de Wit;87284]\r\n\r\nHello..\r\nDoctor 0 means : The doctor(s) gave this eye a '0'.\r\nDoctor 1 means : The doctor(s) gave this eye a '1'.\r\nThe matrix allows you to zoom in quickly on parts where the computer and doctors disagree.\r\nFor instance: Doctor 4, Pred 0.\r\n\r\nOmg.. I tried your approach too but only with a simple classifier on top.\r\nI ditched the results.. But perhaps I should have built a NN on top..\r\n\r\n[/quote]\r\n\r\nOk got it.\r\n\r\nI actually had 3 fully connected layers fc6, f7,fc8 at the top of both nets. I simply took the last pooling layer of the first net for my features into the second net. This takes advantage of the fact that, in a convnet, the weight matrix stays the same size regardless of input image size. So now I could replace my 227x227 images with 872x872. :) If I had a bigger, faster gpu card I may have used a single net.", "votes": null } ], "comments": [ { "id": 87283, "author_name": "gstieger", "author_url": "", "post_date": "07/28/2015 09:15:42", "content": "Julian, congratulations on your outstanding result!
\n\nHow were you able to determine which doctors rated which images?
\n\nMy approach was very straightforward. I first extracted features by giving the first convnet the difficult task of first categorizing a balanced dataset (same number of samples of each class) based on a 227x227 random patch of the 872x872 image using SoftmaxWithLoss . The features extracted in this way were then fed to my phase 2 network which added 2 extra conv layers and used an EuclideanLoss layer and an unbalanced dataset (to approximate the quadratic kappa). That gave me ~0.75 right off the bat.
\n\nHowever the net refused to classify any images as 4, so I was stuck at that score. Perhaps this has something to do with the difference in how the doctors were classifying the image.
\n\nAnyway I still have a lot to learn and have had fun with this my 3rd Kaggle competition. See you at the top one day! :)
", "votes": null, "replies": [] }, { "id": 87284, "author_name": "juliandewit", "author_url": "", "post_date": "07/28/2015 09:27:30", "content": "Thanks!
\nDoctor 0 means : The doctor(s) gave this eye a '0'.
\nDoctor 1 means : The doctor(s) gave this eye a '1'.
\nThe matrix allows you to zoom in quickly on parts where the computer and doctors disagree.
For instance: Doctor 4, Algo 0.
\nThat brings you to this list:\nList of Doctor labeled 4, Algo predicted 0 eyes
\nDecide for yourself who is right.. :)
Omg I tried your approach too but only with a simple classifier on top.
\nI ditched the results.. But perhaps I should have built a NN on top..
[quote=Julian de Wit;87284]
\n\nHello..\nDoctor 0 means : The doctor(s) gave this eye a '0'.\nDoctor 1 means : The doctor(s) gave this eye a '1'.\nThe matrix allows you to zoom in quickly on parts where the computer and doctors disagree.\nFor instance: Doctor 4, Pred 0.
\n\nOmg.. I tried your approach too but only with a simple classifier on top.\nI ditched the results.. But perhaps I should have built a NN on top..
\n\n[/quote]
\n\nOk got it.
\n\nI actually had 3 fully connected layers fc6, f7,fc8 at the top of both nets. I simply took the last pooling layer of the first net for my features into the second net. This takes advantage of the fact that, in a convnet, the weight matrix stays the same size regardless of input image size. So now I could replace my 227x227 images with 872x872. :) If I had a bigger, faster gpu card I may have used a single net.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "87245": "Hello,\r\n\r\nAs the competition progressed I became more and more convinced that automatic screening would really be very helpful. \r\n\r\nI've seen many cases where I'd think that the doctor made mistake.\r\nOf course I've also seen cases where the computer missed something.\r\nHowever I really do think both might complement eachother.\r\n\r\nMy idea was to make some kind of collaboration site where we can discuss every eye in detail.\r\n\r\nTo bad I could not find the time to really build it.\r\nHowever.. I did put a starter on github with a live confusion matrix.\r\n\r\nIf you want to collaborate.. please join the project.\r\nBut perhaps to also attract doctors the site should be a but more user friendly.\r\nSo if anyone has ideas on how to do this.. Be my guest!\r\n\r\n[live confusion matrix][1]\r\n\r\n\r\n [1]: https://github.com/juliandewit/kaggle_retinopathy", "87283": "Julian, congratulations on your outstanding result! \r\n\r\nHow were you able to determine which doctors rated which images?\r\n\r\nMy approach was very straightforward. I first extracted features by giving the first convnet the difficult task of first categorizing a balanced dataset (same number of samples of each class) based on a 227x227 random patch of the 872x872 image using SoftmaxWithLoss . The features extracted in this way were then fed to my phase 2 network which added 2 extra conv layers and used an EuclideanLoss layer and an unbalanced dataset (to approximate the quadratic kappa). That gave me ~0.75 right off the bat. \r\n\r\nHowever the net refused to classify any images as 4, so I was stuck at that score. Perhaps this has something to do with the difference in how the doctors were classifying the image.\r\n\r\nAnyway I still have a lot to learn and have had fun with this my 3rd Kaggle competition. See you at the top one day! :)", "87284": "Thanks!