For practice I tried to obtain a high score on this competition, obviously it's not fair since there's been a lot of advances since the deadline of the comeptition but nevertheless I think my solution is quite simple and obtains a top 1% solution in the leaderboard with a single model and no ensemble.
\nI've created a YouTube video of me detailing each step of the solution:
\nhttps://youtu.be/YxQYvhap3kE
Here is the GitHub code:
\nhttps://github.com/aladdinpersson/Machine-Learning-Collection/tree/master/ML/Kaggles/DiabeticRetinopathy
But let me also do a quick summary of what works (and what didn't for me). I first did a baseline solution with EfficientNet-B3 and image size of 120x120 in order for relatively quick iterations.
\nBASELINE (B3, 120x120 image resolution):
\nI then added preprocessing of the images by removing the black borders and maining the aspect ratio of each.
\nI then improved the loss function, first I used CrossEntropyLoss but since this doesn't correspond very well to the QuadraticWeightedKappa score, I changed to MSE.
\nImprove the loss function (MSE -> convert to integer predictions):
Val score: 0.65 (+0.05)
Get balanced data loader (Didn't work for me)
\n(class_weights=[1,2,2,2,2] -> Doesn't work either)
Heavy data augmentation (for some odd reason, this didn't work?)
Val score: 0.63 (-0.02)
Weird that we didn't get an improvement here. Thinking maybe it's the
\nimage size, maybe using some bad augmentation?
Blending (Left and Right eye information)
Val score: 0.67 (+0.02)
Increase image resolution
\nRes 300x300: 0.736 (+0.08)
\nRes 500x500: 0.787 (+0.05)
\nRes 728x728: 0.837 (+0.05)
Then using left and right eye information and trained the small neural network on top of it got me:
\nPublic score: 0.84613
\nPrivate score: 0.83880
Ideas for improving further:
\nFor practice I tried to obtain a high score on this competition, obviously it's not fair since there's been a lot of advances since the deadline of the comeptition but nevertheless I think my solution is quite simple and obtains a top 1% solution in the leaderboard with a single model and no ensemble.
\nI've created a YouTube video of me detailing each step of the solution:
\nhttps://youtu.be/YxQYvhap3kE
Here is the GitHub code:
\nhttps://github.com/aladdinpersson/Machine-Learning-Collection/tree/master/ML/Kaggles/DiabeticRetinopathy
But let me also do a quick summary of what works (and what didn't for me). I first did a baseline solution with EfficientNet-B3 and image size of 120x120 in order for relatively quick iterations.
\nBASELINE (B3, 120x120 image resolution):
\nI then added preprocessing of the images by removing the black borders and maining the aspect ratio of each.
\nI then improved the loss function, first I used CrossEntropyLoss but since this doesn't correspond very well to the QuadraticWeightedKappa score, I changed to MSE.
\nImprove the loss function (MSE -> convert to integer predictions):
Val score: 0.65 (+0.05)
Get balanced data loader (Didn't work for me)
\n(class_weights=[1,2,2,2,2] -> Doesn't work either)
Heavy data augmentation (for some odd reason, this didn't work?)
Val score: 0.63 (-0.02)
Weird that we didn't get an improvement here. Thinking maybe it's the
\nimage size, maybe using some bad augmentation?
Blending (Left and Right eye information)
Val score: 0.67 (+0.02)
Increase image resolution
\nRes 300x300: 0.736 (+0.08)
\nRes 500x500: 0.787 (+0.05)
\nRes 728x728: 0.837 (+0.05)
Then using left and right eye information and trained the small neural network on top of it got me:
\nPublic score: 0.84613
\nPrivate score: 0.83880
Ideas for improving further:
\nHey @aladdinpersson, really nice video.
\nI wanted to know how to load/train with such huge data, like in this case 83GB? I have seen few competitions with more than 50GB of data, how to deal with this?
@aladdinpersson Thank you for the great ideas you gave us, I have an extra idea here. Can you show us how we can use Ben Graham(Competition winner) pre-processing method (specially color version) and train the model from different transfer learning models? I think this way the result will be improved more.
", "rawMarkdown": "aladdinpersson Thank you for the great ideas you gave us, I have an extra idea here. Can you show us how we can use Ben Graham(Competition winner) pre-processing method (specially color version) and train the model from different transfer learning models? I think this way the result will be improved more.", "votes": null }, { "id": "1383942", "postDate": "07/11/2021 11:08:46", "content": "How many epochs and how long did it take to train if i may ask?
", "rawMarkdown": "How many epochs and how long did it take to train if i may ask?", "votes": null }, { "id": "2142439", "postDate": "02/13/2023 14:28:33", "content": "thank you for making video with such detailed explanation
", "rawMarkdown": "thank you for making video with such detailed explanation", "votes": null } ], "comments": [ { "id": 1339533, "author_name": "abdulwaheedsoudagar", "author_url": "", "post_date": "06/07/2021 09:59:31", "content": "Hey @aladdinpersson, really nice video.
\nI wanted to know how to load/train with such huge data, like in this case 83GB? I have seen few competitions with more than 50GB of data, how to deal with this?
@aladdinpersson Thank you for the great ideas you gave us, I have an extra idea here. Can you show us how we can use Ben Graham(Competition winner) pre-processing method (specially color version) and train the model from different transfer learning models? I think this way the result will be improved more.
", "votes": null, "replies": [] }, { "id": 1383942, "author_name": "mahmoudlimam", "author_url": "", "post_date": "07/11/2021 11:08:46", "content": "How many epochs and how long did it take to train if i may ask?
", "votes": null, "replies": [] }, { "id": 2142439, "author_name": "anastasiiarepalo", "author_url": "", "post_date": "02/13/2023 14:28:33", "content": "thank you for making video with such detailed explanation
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1328803": "For practice I tried to obtain a high score on this competition, obviously it's not fair since there's been a lot of advances since the deadline of the comeptition but nevertheless I think my solution is quite simple and obtains a top 1% solution in the leaderboard with a single model and no ensemble. \n\nI've created a YouTube video of me detailing each step of the solution: \nhttps://youtu.be/YxQYvhap3kE\n\nHere is the GitHub code:\nhttps://github.com/aladdinpersson/Machine-Learning-Collection/tree/master/ML/Kaggles/DiabeticRetinopathy\n\nBut let me also do a quick summary of what works (and what didn't for me). I first did a baseline solution with EfficientNet-B3 and image size of 120x120 in order for relatively quick iterations.\n\nBASELINE (B3, 120x120 image resolution):\n* Val score: 0.55\n* Public score: 0.52648\n\nI then added preprocessing of the images by removing the black borders and maining the aspect ratio of each.\n\n+ Preprocess the images:\n* Val score: 0.60 (+0.05)\n\nI then improved the loss function, first I used CrossEntropyLoss but since this doesn't correspond very well to the QuadraticWeightedKappa score, I changed to MSE.\n\n+ Improve the loss function (MSE -> convert to integer predictions):\n* Val score: 0.65 (+0.05)\n\n+ Get balanced data loader (Didn't work for me)\n(class_weights=[1,2,2,2,2] -> Doesn't work either)\n\n+ Heavy data augmentation (for some odd reason, this didn't work?)\n* Val score: 0.63 (-0.02)\n\nWeird that we didn't get an improvement here. Thinking maybe it's the\nimage size, maybe using some bad augmentation?\n\n+ Blending (Left and Right eye information)\n* Val score: 0.67 (+0.02)\n\n+ Increase image resolution\nRes 300x300: 0.736 (+0.08)\nRes 500x500: 0.787 (+0.05)\nRes 728x728: 0.837 (+0.05)\n\nThen using left and right eye information and trained the small neural network on top of it got me:\nPublic score: 0.84613\nPrivate score: 0.83880\n\nIdeas for improving further:\n* Train on the validation data (4000 examples)\n* Larger model (B4,B5,B6,B7?)\n* Larger image resolution\n* Better data augmentations method (Randaugment?)\n* Ensemble for the blending model (fast to train)\n* Ensemble of various CNN models\n* Tweaking hyperparameters\n* Better loss function, better proxy for WeightedKappa\n* ? Comment ideas below", "1339533": "Hey @aladdinpersson, really nice video. \nI wanted to know how to load/train with such huge data, like in this case 83GB? I have seen few competitions with more than 50GB of data, how to deal with this?", "1345104": "aladdinpersson Thank you for the great ideas you gave us, I have an extra idea here. Can you show us how we can use Ben Graham(Competition winner) pre-processing method (specially color version) and train the model from different transfer learning models? I think this way the result will be improved more.", "1383942": "How many epochs and how long did it take to train if i may ask?", "2142439": "thank you for making video with such detailed explanation" }, "source": "meta" }