{
  "id": 108046,
  "title": "163rd place report and code repository",
  "url": "/competitions/aptos2019-blindness-detection/writeups/dimitreoliveira-163rd-place-report-and-code-reposi",
  "author_name": "",
  "post_date": "2019-09-21T12:10:35.757Z",
  "votes": 6,
  "comment_count": 6,
  "views": 0,
  "content": "<p>This competition was very special for me, it's my first competition medal, I worked and learned a lot, thanks for all that shared valuable information.</p>\n\n<p>A brief report (I won't get into too many details because wasn't anything special)</p>\n\n<ul>\n<li><strong>Data:</strong>\n<ul><li>Old data was sampled to a more balanced distribution, was used only 25% of samples from class 0.</li>\n<li>Each fold had all sampled old data and 20% of new data.</li>\n<li>Training set: All sampled old data and 80% of new data (15770 samples from old data and 2929 samples from new data).</li>\n<li>Validation set: 20% of new data (733 samples).</li></ul></li>\n<li><strong>Model:</strong> EfficientNetB5\n<ul><li>Imagenet pre-trained weights.</li>\n<li>Replaced top head with GlobalAveragePooling2D and a Dense layer as linear output.</li></ul></li>\n<li><strong>Training</strong>\n<ul><li>Parameters:\n<ul><li>Batch size: 32</li></ul></li>\n<li>Warm-up step:\n<ul><li>Freeze all layers except the last 2.</li>\n<li>Learning rate: 4e-3, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (2 epochs warm-up and hold base LR for 2 epochs).</li>\n<li>Train for 5 epochs.</li></ul></li>\n<li>Fine-tunning step:\n<ul><li>Unfreeze all layers.</li>\n<li>Early stopping monitoring validation loss for 5 epochs.</li>\n<li>Learning rate: 4e-4, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (5 epochs warm-up and hold base LR for 3 epochs).</li>\n<li>Train for 20 epochs.</li></ul></li></ul></li>\n<li><strong>Preprocess:</strong> This step was applied before modeling to make training faster.\n<ul><li>Convert image to RGB channels.</li>\n<li>Circle crop image. This crop was used to remove black areas from original square images, and also crop a circular area based on the image center.</li>\n<li>Resize image to 224x224. The original images size were reduced to 224, I have tried many others but this worked best, also made possible to train with larger batch size (32).</li></ul></li>\n<li><strong>Augmentation:</strong> All augmentations were from Keras ImageDataGenerator.\n<ul><li>Rescale (divide by 255).</li>\n<li>Random rotation 360°.</li>\n<li>Random flip (horizontal and vertical).</li></ul></li>\n<li><strong>Inference:</strong>\n<ul><li>Test time augmentation (TTA) x10, using the same augmentations of the training step.</li>\n<li>Averaged all the 5 models and their TTA predictions.</li>\n<li>Threshold: [0.5, 1.5, 2.5, 3.5]</li></ul></li>\n</ul>\n\n<p>I usually take some extra effort and create <a href=\"https://github.com/dimitreOliveira/APTOS2019BlindnessDetection\">repositories</a> for my competitions, there you can find a more <a href=\"https://github.com/dimitreOliveira/APTOS2019BlindnessDetection/tree/master/Best%20solution%20%28Bronze%20medal%20-%20175th%20place%29%20\">detailed report and my solution stack</a>.</p>\n\n<p>For me, it was a lot more work to put everything together in an organized way, but I think it worth it, also would be nice to see something similar for some top submissions, for beginner see the way that more experienced people structure and build their workflow is really important.</p>",
  "messages": [
    {
      "id": "621583",
      "postDate": "09/08/2019 17:17:44",
      "content": "<p>This competition was very special for me, it's my first competition medal, I worked and learned a lot, thanks for all that shared valuable information.</p>\n\n<p>A brief report (I won't get into too many details because wasn't anything special)</p>\n\n<ul>\n<li><strong>Data:</strong>\n<ul><li>Old data was sampled to a more balanced distribution, was used only 25% of samples from class 0.</li>\n<li>Each fold had all sampled old data and 20% of new data.</li>\n<li>Training set: All sampled old data and 80% of new data (15770 samples from old data and 2929 samples from new data).</li>\n<li>Validation set: 20% of new data (733 samples).</li></ul></li>\n<li><strong>Model:</strong> EfficientNetB5\n<ul><li>Imagenet pre-trained weights.</li>\n<li>Replaced top head with GlobalAveragePooling2D and a Dense layer as linear output.</li></ul></li>\n<li><strong>Training</strong>\n<ul><li>Parameters:\n<ul><li>Batch size: 32</li></ul></li>\n<li>Warm-up step:\n<ul><li>Freeze all layers except the last 2.</li>\n<li>Learning rate: 4e-3, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (2 epochs warm-up and hold base LR for 2 epochs).</li>\n<li>Train for 5 epochs.</li></ul></li>\n<li>Fine-tunning step:\n<ul><li>Unfreeze all layers.</li>\n<li>Early stopping monitoring validation loss for 5 epochs.</li>\n<li>Learning rate: 4e-4, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (5 epochs warm-up and hold base LR for 3 epochs).</li>\n<li>Train for 20 epochs.</li></ul></li></ul></li>\n<li><strong>Preprocess:</strong> This step was applied before modeling to make training faster.\n<ul><li>Convert image to RGB channels.</li>\n<li>Circle crop image. This crop was used to remove black areas from original square images, and also crop a circular area based on the image center.</li>\n<li>Resize image to 224x224. The original images size were reduced to 224, I have tried many others but this worked best, also made possible to train with larger batch size (32).</li></ul></li>\n<li><strong>Augmentation:</strong> All augmentations were from Keras ImageDataGenerator.\n<ul><li>Rescale (divide by 255).</li>\n<li>Random rotation 360°.</li>\n<li>Random flip (horizontal and vertical).</li></ul></li>\n<li><strong>Inference:</strong>\n<ul><li>Test time augmentation (TTA) x10, using the same augmentations of the training step.</li>\n<li>Averaged all the 5 models and their TTA predictions.</li>\n<li>Threshold: [0.5, 1.5, 2.5, 3.5]</li></ul></li>\n</ul>\n\n<p>I usually take some extra effort and create <a href=\"https://github.com/dimitreOliveira/APTOS2019BlindnessDetection\">repositories</a> for my competitions, there you can find a more <a href=\"https://github.com/dimitreOliveira/APTOS2019BlindnessDetection/tree/master/Best%20solution%20%28Bronze%20medal%20-%20175th%20place%29%20\">detailed report and my solution stack</a>.</p>\n\n<p>For me, it was a lot more work to put everything together in an organized way, but I think it worth it, also would be nice to see something similar for some top submissions, for beginner see the way that more experienced people structure and build their workflow is really important.</p>",
      "rawMarkdown": "This competition was very special for me, it's my first competition medal, I worked and learned a lot, thanks for all that shared valuable information.\n\nA brief report (I won't get into too many details because wasn't anything special)\n\n- **Data:**\n  - Old data was sampled to a more balanced distribution, was used only 25% of samples from class 0.\n  - Each fold had all sampled old data and 20% of new data.\n  - Training set: All sampled old data and 80% of new data (15770 samples from old data and 2929 samples from new data).\n  - Validation set: 20% of new data (733 samples).\n- **Model:** EfficientNetB5\n  - Imagenet pre-trained weights.\n  - Replaced top head with GlobalAveragePooling2D and a Dense layer as linear output.\n- **Training**\n  - Parameters:\n     - Batch size: 32\n  - Warm-up step:\n     - Freeze all layers except the last 2.\n     - Learning rate: 4e-3, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (2 epochs warm-up and hold base LR for 2 epochs).\n     - Train for 5 epochs.\n  - Fine-tunning step:\n     - Unfreeze all layers.\n     - Early stopping monitoring validation loss for 5 epochs.\n     - Learning rate: 4e-4, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (5 epochs warm-up and hold base LR for 3 epochs).\n     - Train for 20 epochs.\n- **Preprocess:** This step was applied before modeling to make training faster.\n  - Convert image to RGB channels.\n  - Circle crop image. This crop was used to remove black areas from original square images, and also crop a circular area based on the image center.\n  - Resize image to 224x224. The original images size were reduced to 224, I have tried many others but this worked best, also made possible to train with larger batch size (32).\n- **Augmentation:** All augmentations were from Keras ImageDataGenerator.\n  - Rescale (divide by 255).\n  - Random rotation 360°.\n  - Random flip (horizontal and vertical).\n- **Inference:**\n  - Test time augmentation (TTA) x10, using the same augmentations of the training step.\n  - Averaged all the 5 models and their TTA predictions.\n  - Threshold: [0.5, 1.5, 2.5, 3.5]\n\nI usually take some extra effort and create [repositories](https://github.com/dimitreOliveira/APTOS2019BlindnessDetection) for my competitions, there you can find a more [detailed report and my solution stack](https://github.com/dimitreOliveira/APTOS2019BlindnessDetection/tree/master/Best%20solution%20(Bronze%20medal%20-%20175th%20place)%20).\n\nFor me, it was a lot more work to put everything together in an organized way, but I think it worth it, also would be nice to see something similar for some top submissions, for beginner see the way that more experienced people structure and build their workflow is really important.",
      "votes": null
    },
    {
      "id": "621589",
      "postDate": "09/08/2019 17:25:02",
      "content": "<p>Nice write up! Congratulation on your first Bronze =)</p>",
      "rawMarkdown": "Nice write up! Congratulation on your first Bronze =)",
      "votes": null
    },
    {
      "id": "621594",
      "postDate": "09/08/2019 17:28:27",
      "content": "<p>thanks <a href=\"/drhabib\">@drhabib</a> , your tips helped me a lot!</p>",
      "rawMarkdown": "thanks @drhabib , your tips helped me a lot!",
      "votes": null
    },
    {
      "id": "621901",
      "postDate": "09/09/2019 04:46:18",
      "content": "<p>Congratulations...\nGreat Work...\nThanks for Sharing your Approach &amp; Insights... !! <a href=\"/dimitreoliveira\">@dimitreoliveira</a> </p>",
      "rawMarkdown": "Congratulations...\nGreat Work...\nThanks for Sharing your Approach &amp; Insights... !! @dimitreoliveira",
      "votes": null
    },
    {
      "id": "622017",
      "postDate": "09/09/2019 07:06:25",
      "content": "<p>I am 176 😃. </p>",
      "rawMarkdown": "I am 176 😃.",
      "votes": null
    },
    {
      "id": "622193",
      "postDate": "09/09/2019 11:23:02",
      "content": "<p>Thanks <a href=\"/veeralakrishna\">@veeralakrishna</a> </p>",
      "rawMarkdown": "Thanks @veeralakrishna",
      "votes": null
    },
    {
      "id": "622194",
      "postDate": "09/09/2019 11:23:25",
      "content": "<p><a href=\"/neeraj17\">@neeraj17</a> haha, we were so close to silver 😄 </p>",
      "rawMarkdown": "neeraj17 haha, we were so close to silver 😄",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 621589,
      "author_name": "drhabib",
      "author_url": "",
      "post_date": "09/08/2019 17:25:02",
      "content": "<p>Nice write up! Congratulation on your first Bronze =)</p>",
      "votes": null,
      "replies": [
        {
          "id": 621594,
          "author_name": "dimitreoliveira",
          "author_url": "",
          "post_date": "09/08/2019 17:28:27",
          "content": "<p>thanks <a href=\"/drhabib\">@drhabib</a> , your tips helped me a lot!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 621901,
      "author_name": "veeralakrishna",
      "author_url": "",
      "post_date": "09/09/2019 04:46:18",
      "content": "<p>Congratulations...\nGreat Work...\nThanks for Sharing your Approach &amp; Insights... !! <a href=\"/dimitreoliveira\">@dimitreoliveira</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 622193,
          "author_name": "dimitreoliveira",
          "author_url": "",
          "post_date": "09/09/2019 11:23:02",
          "content": "<p>Thanks <a href=\"/veeralakrishna\">@veeralakrishna</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 622017,
      "author_name": "neeraj17",
      "author_url": "",
      "post_date": "09/09/2019 07:06:25",
      "content": "<p>I am 176 😃. </p>",
      "votes": null,
      "replies": [
        {
          "id": 622194,
          "author_name": "dimitreoliveira",
          "author_url": "",
          "post_date": "09/09/2019 11:23:25",
          "content": "<p><a href=\"/neeraj17\">@neeraj17</a> haha, we were so close to silver 😄 </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "621583": "This competition was very special for me, it's my first competition medal, I worked and learned a lot, thanks for all that shared valuable information.\n\nA brief report (I won't get into too many details because wasn't anything special)\n\n- **Data:**\n  - Old data was sampled to a more balanced distribution, was used only 25% of samples from class 0.\n  - Each fold had all sampled old data and 20% of new data.\n  - Training set: All sampled old data and 80% of new data (15770 samples from old data and 2929 samples from new data).\n  - Validation set: 20% of new data (733 samples).\n- **Model:** EfficientNetB5\n  - Imagenet pre-trained weights.\n  - Replaced top head with GlobalAveragePooling2D and a Dense layer as linear output.\n- **Training**\n  - Parameters:\n     - Batch size: 32\n  - Warm-up step:\n     - Freeze all layers except the last 2.\n     - Learning rate: 4e-3, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (2 epochs warm-up and hold base LR for 2 epochs).\n     - Train for 5 epochs.\n  - Fine-tunning step:\n     - Unfreeze all layers.\n     - Early stopping monitoring validation loss for 5 epochs.\n     - Learning rate: 4e-4, Adam optimizer and Cosine &amp; Warm-up learning rate scheduler (5 epochs warm-up and hold base LR for 3 epochs).\n     - Train for 20 epochs.\n- **Preprocess:** This step was applied before modeling to make training faster.\n  - Convert image to RGB channels.\n  - Circle crop image. This crop was used to remove black areas from original square images, and also crop a circular area based on the image center.\n  - Resize image to 224x224. The original images size were reduced to 224, I have tried many others but this worked best, also made possible to train with larger batch size (32).\n- **Augmentation:** All augmentations were from Keras ImageDataGenerator.\n  - Rescale (divide by 255).\n  - Random rotation 360°.\n  - Random flip (horizontal and vertical).\n- **Inference:**\n  - Test time augmentation (TTA) x10, using the same augmentations of the training step.\n  - Averaged all the 5 models and their TTA predictions.\n  - Threshold: [0.5, 1.5, 2.5, 3.5]\n\nI usually take some extra effort and create [repositories](https://github.com/dimitreOliveira/APTOS2019BlindnessDetection) for my competitions, there you can find a more [detailed report and my solution stack](https://github.com/dimitreOliveira/APTOS2019BlindnessDetection/tree/master/Best%20solution%20(Bronze%20medal%20-%20175th%20place)%20).\n\nFor me, it was a lot more work to put everything together in an organized way, but I think it worth it, also would be nice to see something similar for some top submissions, for beginner see the way that more experienced people structure and build their workflow is really important.",
    "621589": "Nice write up! Congratulation on your first Bronze =)",
    "621594": "thanks @drhabib , your tips helped me a lot!",
    "621901": "Congratulations...\nGreat Work...\nThanks for Sharing your Approach &amp; Insights... !! @dimitreoliveira",
    "622017": "I am 176 😃.",
    "622193": "Thanks @veeralakrishna",
    "622194": "neeraj17 haha, we were so close to silver 😄"
  },
  "source": "meta"
}