![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1073879%2Fed5e641a3017f356fbaf909a5a9482d2%2Fimage.png?generation=1568107876432985&alt=media\")
Congratulation to all winners and thanks for great competition. This would be my first medal, so I decided to write up our approach.
\n\nMain problems to solve:\n- inbalanced classes\n- not enough data\n- different distribution of the train and public test data set,
\n\nData augmentation:\nI think that one of the best way to improve score on public set was strong augmentation. So, to augment we differentiated two types of the images - first where image contains whole eyeball, and second where image contains cropped eyeball. In first case to augment we cropped both – rows and columns and in the second case we cropped only rows. Cropping was done randomly from 0 to 25% of the size of the eyeball in each side. After that we rotated image from 0 to 360 degrees. However to avoid cropping during rotation we used this function from [1]. I believed that this also helped to train model in different scales. We also used some pixel-wise augmentation - like addition, multiplications and so on. Then we used the same as Ben’s processing. During validation we got rid of black area in image. Picture below shows: original data, training data and validation/test.
\n\n
Model\nBasically we used the same augmentation and settings to pretrain model on data from 2015. We trained model with weights computed from sklearn.utils.class_weight. As a loss function we used slightly modified categorical_crossentropy loss to penalize more distant predictions (from [2]) and this was our checkpoint metric. I also tried to use as a checkpoint metric accuracy, f1, or kappa but it scored worse in public data set. We used Adam optimizer with start learning rate 8e-5 and then learning rate was scheduled as follow.
\n\n\ndef scheduler(epoch,learning_rate):\n lr = learning_rate\n if epoch==3:\n lr = 3e-5\n if epoch==6:\n lr = 1e-5\n if epoch==9:\n lr = 8e-6\n if epoch==12:\n lr = 4e-6\n return lr\n
Validation was done on 10% data. For single model the validation kappa score was about 0.87 (without TTA)
\n\nPrediction Time\nDuring prediction we used simple cropping image to contain only eyeball and 7 TTA, each of them was only rotation (without cropping) by additional 45 degrees (0, 45, 90, and so on). Doing that allowed to improve score for EfficientNet-B5 from 80.4 (without TTA) to 81.8 on public test. Some people wrote that it is useless, but I really think it really depends on how you trained your network and how it is done TTA itself.
\n\nEnsembling\nAs a final submission we used soft-voting on 7 TTA on EfficientNet-B5 (image size 456) and 7 TTA on EfficientNet-B4 (image size 380). It scored 81.6 on public set and 92.0 on private set.
\n\nWe believed that our solution was stable (after previous competition I tried to avoid my fallacy in relying only on public score), because improvement on validation fold was scored better on public LB, slight changes in settings did not changed public LB score, and we used strong augmentation. Any comment what could we do in different manner are welcomed. Thank you, and I'm starting to dive into fascinated readings of top solutions.
\n\n[1] https://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image/33247373#33247373\n[2]\nhttps://github.com/JHart96/keras_ordinal_categorical_crossentropy/blob/master/ordinal_categorical_crossentropy.py
", "messages": [ { "id": "622956", "postDate": "09/10/2019 09:58:46", "content": "Congratulation to all winners and thanks for great competition. This would be my first medal, so I decided to write up our approach.
\n\nMain problems to solve:\n- inbalanced classes\n- not enough data\n- different distribution of the train and public test data set,
\n\nData augmentation:\nI think that one of the best way to improve score on public set was strong augmentation. So, to augment we differentiated two types of the images - first where image contains whole eyeball, and second where image contains cropped eyeball. In first case to augment we cropped both – rows and columns and in the second case we cropped only rows. Cropping was done randomly from 0 to 25% of the size of the eyeball in each side. After that we rotated image from 0 to 360 degrees. However to avoid cropping during rotation we used this function from [1]. I believed that this also helped to train model in different scales. We also used some pixel-wise augmentation - like addition, multiplications and so on. Then we used the same as Ben’s processing. During validation we got rid of black area in image. Picture below shows: original data, training data and validation/test.
\n\n
Model\nBasically we used the same augmentation and settings to pretrain model on data from 2015. We trained model with weights computed from sklearn.utils.class_weight. As a loss function we used slightly modified categorical_crossentropy loss to penalize more distant predictions (from [2]) and this was our checkpoint metric. I also tried to use as a checkpoint metric accuracy, f1, or kappa but it scored worse in public data set. We used Adam optimizer with start learning rate 8e-5 and then learning rate was scheduled as follow.
\n\n\ndef scheduler(epoch,learning_rate):\n lr = learning_rate\n if epoch==3:\n lr = 3e-5\n if epoch==6:\n lr = 1e-5\n if epoch==9:\n lr = 8e-6\n if epoch==12:\n lr = 4e-6\n return lr\n
Validation was done on 10% data. For single model the validation kappa score was about 0.87 (without TTA)
\n\nPrediction Time\nDuring prediction we used simple cropping image to contain only eyeball and 7 TTA, each of them was only rotation (without cropping) by additional 45 degrees (0, 45, 90, and so on). Doing that allowed to improve score for EfficientNet-B5 from 80.4 (without TTA) to 81.8 on public test. Some people wrote that it is useless, but I really think it really depends on how you trained your network and how it is done TTA itself.
\n\nEnsembling\nAs a final submission we used soft-voting on 7 TTA on EfficientNet-B5 (image size 456) and 7 TTA on EfficientNet-B4 (image size 380). It scored 81.6 on public set and 92.0 on private set.
\n\nWe believed that our solution was stable (after previous competition I tried to avoid my fallacy in relying only on public score), because improvement on validation fold was scored better on public LB, slight changes in settings did not changed public LB score, and we used strong augmentation. Any comment what could we do in different manner are welcomed. Thank you, and I'm starting to dive into fascinated readings of top solutions.
\n\n[1] https://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image/33247373#33247373\n[2]\nhttps://github.com/JHart96/keras_ordinal_categorical_crossentropy/blob/master/ordinal_categorical_crossentropy.py
", "rawMarkdown": "Congratulation to all winners and thanks for great competition. This would be my first medal, so I decided to write up our approach.\n \n**Main problems to solve:**\n- inbalanced classes\n- not enough data\n- different distribution of the train and public test data set,\n\n**Data augmentation:**\nI think that one of the best way to improve score on public set was strong augmentation. So, to augment we differentiated two types of the images - first where image contains whole eyeball, and second where image contains cropped eyeball. In first case to augment we cropped both – rows and columns and in the second case we cropped only rows. Cropping was done randomly from 0 to 25% of the size of the eyeball in each side. After that we rotated image from 0 to 360 degrees. However to avoid cropping during rotation we used this function from [[1](https://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image/33247373#33247373)]. I believed that this also helped to train model in different scales. We also used some pixel-wise augmentation - like addition, multiplications and so on. Then we used the same as Ben’s processing. During validation we got rid of black area in image. Picture below shows: original data, training data and validation/test.\n\n\n\n**Model**\nBasically we used the same augmentation and settings to pretrain model on data from 2015. We trained model with weights computed from sklearn.utils.class_weight. As a loss function we used slightly modified categorical_crossentropy loss to penalize more distant predictions (from [[2](https://github.com/JHart96/keras_ordinal_categorical_crossentropy/blob/master/ordinal_categorical_crossentropy.py)]) and this was our checkpoint metric. I also tried to use as a checkpoint metric accuracy, f1, or kappa but it scored worse in public data set. We used Adam optimizer with start learning rate 8e-5 and then learning rate was scheduled as follow. \n\n```\ndef scheduler(epoch,learning_rate):\n lr = learning_rate\n if epoch==3:\n lr = 3e-5\n if epoch==6:\n lr = 1e-5\n if epoch==9:\n lr = 8e-6\n if epoch==12:\n lr = 4e-6\n return lr\n```\n\nValidation was done on 10% data. For single model the validation kappa score was about 0.87 (without TTA) \n \n**Prediction Time**\nDuring prediction we used simple cropping image to contain only eyeball and 7 TTA, each of them was only rotation (without cropping) by additional 45 degrees (0, 45, 90, and so on). Doing that allowed to improve score for EfficientNet-B5 from 80.4 (without TTA) to 81.8 on public test. Some people wrote that it is useless, but I really think it really depends on how you trained your network and how it is done TTA itself.\n\n**Ensembling**\nAs a final submission we used soft-voting on 7 TTA on EfficientNet-B5 (image size 456) and 7 TTA on EfficientNet-B4 (image size 380). It scored 81.6 on public set and 92.0 on private set.\n\nWe believed that our solution was stable (after previous competition I tried to avoid my fallacy in relying only on public score), because improvement on validation fold was scored better on public LB, slight changes in settings did not changed public LB score, and we used strong augmentation. Any comment what could we do in different manner are welcomed. Thank you, and I'm starting to dive into fascinated readings of top solutions.\n\n[1] https://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image/33247373#33247373\n[2]\nhttps://github.com/JHart96/keras_ordinal_categorical_crossentropy/blob/master/ordinal_categorical_crossentropy.py", "votes": null }, { "id": "622959", "postDate": "09/10/2019 10:01:00", "content": "Congrats\nGreat Work...\nThanks for Sharing your Approach & Insights...!! @rafzy15
", "rawMarkdown": "Congrats\nGreat Work...\nThanks for Sharing your Approach & Insights...!! @rafzy15", "votes": null }, { "id": "622976", "postDate": "09/10/2019 10:26:56", "content": "THANKS SIR !!!
\n\nI learned a lot from this kernel, I will definitely reuse some concepts in my work\n🔥
", "rawMarkdown": "THANKS SIR !!!\n\nI learned a lot from this kernel, I will definitely reuse some concepts in my work\n🔥", "votes": null }, { "id": "623206", "postDate": "09/10/2019 15:41:44", "content": "Congrats. Good work. Simple and effective approach.
", "rawMarkdown": "Congrats. Good work. Simple and effective approach.", "votes": null }, { "id": "623483", "postDate": "09/11/2019 02:36:22", "content": "Congratulations. How big was the batch size for B-4?
", "rawMarkdown": "Congratulations. How big was the batch size for B-4?", "votes": null }, { "id": "623670", "postDate": "09/11/2019 07:35:56", "content": "Thank you. Batch size was 16.
", "rawMarkdown": "Thank you. Batch size was 16.", "votes": null }, { "id": "624701", "postDate": "09/12/2019 09:43:53", "content": "Grats. Nice work.
", "rawMarkdown": "Grats. Nice work.", "votes": null } ], "comments": [ { "id": 622959, "author_name": "veeralakrishna", "author_url": "", "post_date": "09/10/2019 10:01:00", "content": "Congrats\nGreat Work...\nThanks for Sharing your Approach & Insights...!! @rafzy15
", "votes": null, "replies": [] }, { "id": 622976, "author_name": "lodetomasi1995", "author_url": "", "post_date": "09/10/2019 10:26:56", "content": "THANKS SIR !!!
\n\nI learned a lot from this kernel, I will definitely reuse some concepts in my work\n🔥
", "votes": null, "replies": [] }, { "id": 623206, "author_name": "manojprabhaakr", "author_url": "", "post_date": "09/10/2019 15:41:44", "content": "Congrats. Good work. Simple and effective approach.
", "votes": null, "replies": [] }, { "id": 623483, "author_name": "abualabed", "author_url": "", "post_date": "09/11/2019 02:36:22", "content": "Congratulations. How big was the batch size for B-4?
", "votes": null, "replies": [ { "id": 623670, "author_name": "rafzy15", "author_url": "", "post_date": "09/11/2019 07:35:56", "content": "Thank you. Batch size was 16.
", "votes": null, "replies": [] } ] }, { "id": 624701, "author_name": "alexanderthestudent", "author_url": "", "post_date": "09/12/2019 09:43:53", "content": "Grats. Nice work.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "622956": "Congratulation to all winners and thanks for great competition. This would be my first medal, so I decided to write up our approach.\n \n**Main problems to solve:**\n- inbalanced classes\n- not enough data\n- different distribution of the train and public test data set,\n\n**Data augmentation:**\nI think that one of the best way to improve score on public set was strong augmentation. So, to augment we differentiated two types of the images - first where image contains whole eyeball, and second where image contains cropped eyeball. In first case to augment we cropped both – rows and columns and in the second case we cropped only rows. Cropping was done randomly from 0 to 25% of the size of the eyeball in each side. After that we rotated image from 0 to 360 degrees. However to avoid cropping during rotation we used this function from [[1](https://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image/33247373#33247373)]. I believed that this also helped to train model in different scales. We also used some pixel-wise augmentation - like addition, multiplications and so on. Then we used the same as Ben’s processing. During validation we got rid of black area in image. Picture below shows: original data, training data and validation/test.\n\n\n\n**Model**\nBasically we used the same augmentation and settings to pretrain model on data from 2015. We trained model with weights computed from sklearn.utils.class_weight. As a loss function we used slightly modified categorical_crossentropy loss to penalize more distant predictions (from [[2](https://github.com/JHart96/keras_ordinal_categorical_crossentropy/blob/master/ordinal_categorical_crossentropy.py)]) and this was our checkpoint metric. I also tried to use as a checkpoint metric accuracy, f1, or kappa but it scored worse in public data set. We used Adam optimizer with start learning rate 8e-5 and then learning rate was scheduled as follow. \n\n```\ndef scheduler(epoch,learning_rate):\n lr = learning_rate\n if epoch==3:\n lr = 3e-5\n if epoch==6:\n lr = 1e-5\n if epoch==9:\n lr = 8e-6\n if epoch==12:\n lr = 4e-6\n return lr\n```\n\nValidation was done on 10% data. For single model the validation kappa score was about 0.87 (without TTA) \n \n**Prediction Time**\nDuring prediction we used simple cropping image to contain only eyeball and 7 TTA, each of them was only rotation (without cropping) by additional 45 degrees (0, 45, 90, and so on). Doing that allowed to improve score for EfficientNet-B5 from 80.4 (without TTA) to 81.8 on public test. Some people wrote that it is useless, but I really think it really depends on how you trained your network and how it is done TTA itself.\n\n**Ensembling**\nAs a final submission we used soft-voting on 7 TTA on EfficientNet-B5 (image size 456) and 7 TTA on EfficientNet-B4 (image size 380). It scored 81.6 on public set and 92.0 on private set.\n\nWe believed that our solution was stable (after previous competition I tried to avoid my fallacy in relying only on public score), because improvement on validation fold was scored better on public LB, slight changes in settings did not changed public LB score, and we used strong augmentation. Any comment what could we do in different manner are welcomed. Thank you, and I'm starting to dive into fascinated readings of top solutions.\n\n[1] https://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image/33247373#33247373\n[2]\nhttps://github.com/JHart96/keras_ordinal_categorical_crossentropy/blob/master/ordinal_categorical_crossentropy.py", "622959": "Congrats\nGreat Work...\nThanks for Sharing your Approach & Insights...!! @rafzy15", "622976": "THANKS SIR !!!\n\nI learned a lot from this kernel, I will definitely reuse some concepts in my work\n🔥", "623206": "Congrats. Good work. Simple and effective approach.", "623483": "Congratulations. How big was the batch size for B-4?", "623670": "Thank you. Batch size was 16.", "624701": "Grats. Nice work." }, "source": "meta" }