{
  "id": 107958,
  "title": "11th place solution",
  "url": "/competitions/aptos2019-blindness-detection/discussion/107958",
  "author_name": "4ui_iurz1",
  "post_date": "2019-09-08T06:34:06.977000",
  "votes": 32,
  "comment_count": 26,
  "views": 0,
  "content": "<p>Congrats to all the competitors!</p>\n\n<p>local code: <a href=\"https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection\">https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection</a>\nkernel: <a href=\"https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution\">https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution</a></p>\n\n<h2>Preprocessing</h2>\n\n<p>I used only Ben's crop (scale radius).</p>\n\n<h2>Augmentation</h2>\n\n<p>Output image size is 256x256.\n<code>python\ntrain_transform = transforms.Compose([\n    transforms.Resize((288, 288)),\n    transforms.RandomAffine(\n        degrees=(-180, 180),\n        scale=(0.8889, 1.0),\n        shear=(-36, 36)),\n    transforms.CenterCrop(256),\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.RandomVerticalFlip(p=0.5),\n    transforms.ColorJitter(contrast=(0.9, 1.1)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n])\n</code></p>\n\n<h2>1st-level models (run on local)</h2>\n\n<ul>\n<li>Models: SE-ResNeXt50_32x4d, SE-ResNeXt101_32x4d, SENet154</li>\n<li>Loss: MSE</li>\n<li>Optimizer: SGD (momentum=0.9)</li>\n<li>LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)</li>\n<li>30 epochs</li>\n<li>Dataset: 2019 train dataset (5-folds cv) + 2015 dataset (like <a href=\"https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#581042\">https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#581042</a>)</li>\n</ul>\n\n<h2>2nd-level models (run on <a href=\"https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution\">kernel</a>)</h2>\n\n<ul>\n<li>Models: SE-ResNeXt50_32x4d, SE-ResNeXt101_32x4d (1st-level models' weights)</li>\n<li>Loss: MSE</li>\n<li>Optimizer: RAdam</li>\n<li>LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)</li>\n<li>10 epochs</li>\n<li>Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (<strong>public + private</strong>,  divided into 5 and used different data each fold. )</li>\n<li>Pseudo labels: weighted average of 1st-level models</li>\n</ul>\n\n<h2>Ensemble</h2>\n\n<p>Finally, averaged 2nd-level models' predictions.</p>\n\n<ul>\n<li>PublicLB: 0.826</li>\n<li>PrivateLB: 0.930</li>\n</ul>\n\n<p>Fortunately, I selected the best submission.\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F849476%2Ff697521599e135faf71027923c90b0f2%2Fmy_submissions.png?generation=1568189226782044&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 621011,
      "postDate": "2019-09-08T06:34:06.977Z",
      "content": "<p>Congrats to all the competitors!</p>\n\n<p>local code: <a href=\"https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection\">https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection</a>\nkernel: <a href=\"https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution\">https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution</a></p>\n\n<h2>Preprocessing</h2>\n\n<p>I used only Ben's crop (scale radius).</p>\n\n<h2>Augmentation</h2>\n\n<p>Output image size is 256x256.\n<code>python\ntrain_transform = transforms.Compose([\n    transforms.Resize((288, 288)),\n    transforms.RandomAffine(\n        degrees=(-180, 180),\n        scale=(0.8889, 1.0),\n        shear=(-36, 36)),\n    transforms.CenterCrop(256),\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.RandomVerticalFlip(p=0.5),\n    transforms.ColorJitter(contrast=(0.9, 1.1)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n])\n</code></p>\n\n<h2>1st-level models (run on local)</h2>\n\n<ul>\n<li>Models: SE-ResNeXt50_32x4d, SE-ResNeXt101_32x4d, SENet154</li>\n<li>Loss: MSE</li>\n<li>Optimizer: SGD (momentum=0.9)</li>\n<li>LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)</li>\n<li>30 epochs</li>\n<li>Dataset: 2019 train dataset (5-folds cv) + 2015 dataset (like <a href=\"https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#581042\">https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#581042</a>)</li>\n</ul>\n\n<h2>2nd-level models (run on <a href=\"https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution\">kernel</a>)</h2>\n\n<ul>\n<li>Models: SE-ResNeXt50_32x4d, SE-ResNeXt101_32x4d (1st-level models' weights)</li>\n<li>Loss: MSE</li>\n<li>Optimizer: RAdam</li>\n<li>LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)</li>\n<li>10 epochs</li>\n<li>Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (<strong>public + private</strong>,  divided into 5 and used different data each fold. )</li>\n<li>Pseudo labels: weighted average of 1st-level models</li>\n</ul>\n\n<h2>Ensemble</h2>\n\n<p>Finally, averaged 2nd-level models' predictions.</p>\n\n<ul>\n<li>PublicLB: 0.826</li>\n<li>PrivateLB: 0.930</li>\n</ul>\n\n<p>Fortunately, I selected the best submission.\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F849476%2Ff697521599e135faf71027923c90b0f2%2Fmy_submissions.png?generation=1568189226782044&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "Congrats to all the competitors!\n\nlocal code: https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection\nkernel: https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution\n\n## Preprocessing\nI used only Ben's crop (scale radius).\n\n## Augmentation\nOutput image size is 256x256.\n```python\ntrain_transform = transforms.Compose([\n    transforms.Resize((288, 288)),\n    transforms.RandomAffine(\n        degrees=(-180, 180),\n        scale=(0.8889, 1.0),\n        shear=(-36, 36)),\n    transforms.CenterCrop(256),\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.RandomVerticalFlip(p=0.5),\n    transforms.ColorJitter(contrast=(0.9, 1.1)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n])\n```\n\n## 1st-level models (run on local)\n- Models: SE-ResNeXt50\\_32x4d, SE-ResNeXt101\\_32x4d, SENet154\n- Loss: MSE\n- Optimizer: SGD (momentum=0.9)\n- LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)\n- 30 epochs\n- Dataset: 2019 train dataset (5-folds cv) + 2015 dataset (like https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#581042)\n\n## 2nd-level models (run on [kernel](https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution)) \n- Models: SE-ResNeXt50\\_32x4d, SE-ResNeXt101\\_32x4d (1st-level models' weights)\n- Loss: MSE\n- Optimizer: RAdam\n- LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)\n- 10 epochs\n- Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (**public + private**,  divided into 5 and used different data each fold. )\n- Pseudo labels: weighted average of 1st-level models\n\n## Ensemble\nFinally, averaged 2nd-level models' predictions.\n\n- PublicLB: 0.826\n- PrivateLB: 0.930\n\nFortunately, I selected the best submission.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F849476%2Ff697521599e135faf71027923c90b0f2%2Fmy_submissions.png?generation=1568189226782044&amp;alt=media)\n",
      "votes": 32
    },
    {
      "id": 621077,
      "postDate": "2019-09-08T07:25:32.097Z",
      "content": "<p>Congrats. Great work. If you can share  the code link, that would be great. Thanks. </p>",
      "rawMarkdown": "Congrats. Great work. If you can share  the code link, that would be great. Thanks. ",
      "votes": 1,
      "replies": [
        {
          "id": 622200,
          "postDate": "2019-09-09T11:29:53.873Z",
          "content": "<p>Thanks! Sure.</p>",
          "rawMarkdown": "Thanks! Sure."
        },
        {
          "id": 622598,
          "postDate": "2019-09-09T21:29:17.483Z",
          "content": "<p><a href=\"/manojprabhaakr\">@manojprabhaakr</a> code link: <a href=\"https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection\">https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection</a></p>",
          "rawMarkdown": "@manojprabhaakr code link: https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection"
        },
        {
          "id": 622690,
          "postDate": "2019-09-10T01:12:16.797Z",
          "content": "<p>Thanks for sharing the link. Congrats once again. </p>",
          "rawMarkdown": "Thanks for sharing the link. Congrats once again. ",
          "votes": 1
        }
      ]
    },
    {
      "id": 621030,
      "postDate": "2019-09-08T06:53:41.783Z",
      "content": "<p>Thank you for sharing and Congrats! </p>",
      "rawMarkdown": "Thank you for sharing and Congrats! ",
      "votes": 1,
      "replies": [
        {
          "id": 622198,
          "postDate": "2019-09-09T11:27:56.887Z",
          "content": "<p>Thanks!</p>",
          "rawMarkdown": "Thanks!"
        }
      ]
    },
    {
      "id": 632278,
      "postDate": "2019-09-23T12:28:39.740Z",
      "content": "<p>Sorry  for  late  to  say  thanks  ,  I  read  both   your     1st-level  code  and  2nd-level   code  carefully and  rerun  it , your  logic is  clear with  no  confusing  in your code . Very  simple but  very  useful  ,   thanks   for    your kind  to  open  your   code ,  I  learn  a  lot   from  your code .</p>\n\n<p>one advise：I can't understand the file <code>SUMMARY.md</code> lives in <code>../markdowns/diabetic_retinopathy_detection</code>. </p>",
      "rawMarkdown": "Sorry  for  late  to  say  thanks  ,  I  read  both   your     1st-level  code  and  2nd-level   code  carefully and  rerun  it , your  logic is  clear with  no  confusing  in your code . Very  simple but  very  useful  ,   thanks   for    your kind  to  open  your   code ,  I  learn  a  lot   from  your code .\n\none advise：I can't understand the file `SUMMARY.md` lives in `../markdowns/diabetic_retinopathy_detection`. ",
      "replies": [
        {
          "id": 639473,
          "postDate": "2019-10-03T08:58:11.803Z",
          "content": "<p><a href=\"/xujingzhao\">@xujingzhao</a> Thanks!\nSorry for the confusion.\nSUMMARY.md is the 2015 solutions translated into Japanese.\nIt doesn't matter with my solution.</p>",
          "rawMarkdown": "@xujingzhao Thanks!\nSorry for the confusion.\nSUMMARY.md is the 2015 solutions translated into Japanese.\nIt doesn't matter with my solution."
        }
      ]
    },
    {
      "id": 624522,
      "postDate": "2019-09-12T06:53:49.883Z",
      "content": "<p>Congratulations…, could you share your private dataset of the kernel? I want to reproduce some experiments. Thanks</p>",
      "rawMarkdown": "Congratulations…, could you share your private dataset of the kernel? I want to reproduce some experiments. Thanks\n\n",
      "replies": [
        {
          "id": 624700,
          "postDate": "2019-09-12T09:41:01.903Z",
          "content": "<p><a href=\"/abnerzhang\">@abnerzhang</a> Thanks! Sure.</p>",
          "rawMarkdown": "@abnerzhang Thanks! Sure."
        }
      ]
    },
    {
      "id": 624494,
      "postDate": "2019-09-12T06:15:35.823Z",
      "content": "<p><a href=\"/uiiurz1\">@uiiurz1</a>  In your 2nd level model, you say Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (public + private, divided into 5 and used different data each fold. )</p>\n\n<p>So you trained your model in kernel using pseudo labels from private test set too?</p>",
      "rawMarkdown": "@uiiurz1  In your 2nd level model, you say Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (public + private, divided into 5 and used different data each fold. )\n\nSo you trained your model in kernel using pseudo labels from private test set too?",
      "replies": [
        {
          "id": 624697,
          "postDate": "2019-09-12T09:35:16.850Z",
          "content": "<p><a href=\"/yousof9\">@yousof9</a> That's right.</p>",
          "rawMarkdown": "@yousof9 That's right."
        }
      ]
    },
    {
      "id": 623627,
      "postDate": "2019-09-11T06:50:59.173Z",
      "content": "<p>Thank you so much for sharing. Very generous of you!</p>",
      "rawMarkdown": "Thank you so much for sharing. Very generous of you!",
      "replies": [
        {
          "id": 623696,
          "postDate": "2019-09-11T08:08:02.350Z",
          "content": "<p>Thanks!</p>",
          "rawMarkdown": "Thanks!"
        }
      ]
    },
    {
      "id": 621479,
      "postDate": "2019-09-08T15:15:12.197Z",
      "content": "<p>Thanks for sharing, your solution is very simple and beautiful!!\nI have a quick question, is RAdam is better than normal Adam or SGD on 2nd level model??</p>",
      "rawMarkdown": "Thanks for sharing, your solution is very simple and beautiful!!\nI have a quick question, is RAdam is better than normal Adam or SGD on 2nd level model??",
      "replies": [
        {
          "id": 622197,
          "postDate": "2019-09-09T11:27:33.783Z",
          "content": "<p>Thanks! <a href=\"/bamps53\">@bamps53</a> \nI trained 2nd level models on kernel.\nSince the execution time of the kernel is limited, it's necessary to use the optimizer with fast convergence.\nRAdam converged faster than SGD and Adam.</p>",
          "rawMarkdown": "Thanks! @bamps53 \nI trained 2nd level models on kernel.\nSince the execution time of the kernel is limited, it's necessary to use the optimizer with fast convergence.\nRAdam converged faster than SGD and Adam."
        }
      ]
    },
    {
      "id": 621470,
      "postDate": "2019-09-08T14:49:54.250Z",
      "content": "<p>It seems that your models generalized best in the gold zone. Congratulations!</p>",
      "rawMarkdown": "It seems that your models generalized best in the gold zone. Congratulations!",
      "replies": [
        {
          "id": 622203,
          "postDate": "2019-09-09T11:30:32.420Z",
          "content": "<p>Thanks!!</p>",
          "rawMarkdown": "Thanks!!"
        }
      ]
    },
    {
      "id": 621407,
      "postDate": "2019-09-08T13:42:17.470Z",
      "content": "<p>Congrats <a href=\"/uiiurz1\">@uiiurz1</a> and thanks for sharing your solution overview.</p>",
      "rawMarkdown": "Congrats @uiiurz1 and thanks for sharing your solution overview.",
      "replies": [
        {
          "id": 622202,
          "postDate": "2019-09-09T11:30:18.230Z",
          "content": "<p>Thanks!</p>",
          "rawMarkdown": "Thanks!"
        }
      ]
    },
    {
      "id": 621399,
      "postDate": "2019-09-08T13:35:03.447Z",
      "content": "<p>Congrats and thanks for sharing :)</p>",
      "rawMarkdown": "Congrats and thanks for sharing :)",
      "replies": [
        {
          "id": 622201,
          "postDate": "2019-09-09T11:30:02.380Z",
          "content": "<p>Thanks!</p>",
          "rawMarkdown": "Thanks!"
        }
      ]
    },
    {
      "id": 621012,
      "postDate": "2019-09-08T06:36:30.337Z",
      "content": "<p>Congratulations! <a href=\"/uiiurz1\">@uiiurz1</a> </p>",
      "rawMarkdown": "Congratulations! @uiiurz1 ",
      "replies": [
        {
          "id": 621018,
          "postDate": "2019-09-08T06:43:26.027Z",
          "content": "<p>Thanks! <a href=\"/chanhu\">@chanhu</a> </p>",
          "rawMarkdown": "Thanks! @chanhu "
        }
      ]
    },
    {
      "id": 621953,
      "postDate": "2019-09-09T05:49:09.777Z",
      "rawMarkdown": "",
      "votes": -1,
      "isDeleted": true,
      "replies": [
        {
          "id": 622204,
          "postDate": "2019-09-09T11:30:50.540Z",
          "content": "<p>Thanks!</p>",
          "rawMarkdown": "Thanks!"
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 621077,
      "author_name": "Manoj Prabhakar",
      "author_url": "",
      "post_date": "2019-09-08T07:25:32.097000",
      "content": "<p>Congrats. Great work. If you can share  the code link, that would be great. Thanks. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 622200,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:29:53.873000",
          "content": "<p>Thanks! Sure.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 622598,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T21:29:17.483000",
          "content": "<p><a href=\"/manojprabhaakr\">@manojprabhaakr</a> code link: <a href=\"https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection\">https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 622690,
          "author_name": "Manoj Prabhakar",
          "author_url": "",
          "post_date": "2019-09-10T01:12:16.797000",
          "content": "<p>Thanks for sharing the link. Congrats once again. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 621030,
      "author_name": "Anand Selvadurai",
      "author_url": "",
      "post_date": "2019-09-08T06:53:41.783000",
      "content": "<p>Thank you for sharing and Congrats! </p>",
      "votes": 1,
      "replies": [
        {
          "id": 622198,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:27:56.887000",
          "content": "<p>Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 632278,
      "author_name": "哈尔的移动城堡",
      "author_url": "",
      "post_date": "2019-09-23T12:28:39.740000",
      "content": "<p>Sorry  for  late  to  say  thanks  ,  I  read  both   your     1st-level  code  and  2nd-level   code  carefully and  rerun  it , your  logic is  clear with  no  confusing  in your code . Very  simple but  very  useful  ,   thanks   for    your kind  to  open  your   code ,  I  learn  a  lot   from  your code .</p>\n\n<p>one advise：I can't understand the file <code>SUMMARY.md</code> lives in <code>../markdowns/diabetic_retinopathy_detection</code>. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 639473,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-10-03T08:58:11.803000",
          "content": "<p><a href=\"/xujingzhao\">@xujingzhao</a> Thanks!\nSorry for the confusion.\nSUMMARY.md is the 2015 solutions translated into Japanese.\nIt doesn't matter with my solution.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 624522,
      "author_name": "abnerzhang",
      "author_url": "",
      "post_date": "2019-09-12T06:53:49.883000",
      "content": "<p>Congratulations…, could you share your private dataset of the kernel? I want to reproduce some experiments. Thanks</p>",
      "votes": 0,
      "replies": [
        {
          "id": 624700,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-12T09:41:01.903000",
          "content": "<p><a href=\"/abnerzhang\">@abnerzhang</a> Thanks! Sure.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 624494,
      "author_name": "Yousef Rabi",
      "author_url": "",
      "post_date": "2019-09-12T06:15:35.823000",
      "content": "<p><a href=\"/uiiurz1\">@uiiurz1</a>  In your 2nd level model, you say Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (public + private, divided into 5 and used different data each fold. )</p>\n\n<p>So you trained your model in kernel using pseudo labels from private test set too?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 624697,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-12T09:35:16.850000",
          "content": "<p><a href=\"/yousof9\">@yousof9</a> That's right.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 623627,
      "author_name": "Yousef Rabi",
      "author_url": "",
      "post_date": "2019-09-11T06:50:59.173000",
      "content": "<p>Thank you so much for sharing. Very generous of you!</p>",
      "votes": 0,
      "replies": [
        {
          "id": 623696,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-11T08:08:02.350000",
          "content": "<p>Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 621479,
      "author_name": "Camaro",
      "author_url": "",
      "post_date": "2019-09-08T15:15:12.197000",
      "content": "<p>Thanks for sharing, your solution is very simple and beautiful!!\nI have a quick question, is RAdam is better than normal Adam or SGD on 2nd level model??</p>",
      "votes": 0,
      "replies": [
        {
          "id": 622197,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:27:33.783000",
          "content": "<p>Thanks! <a href=\"/bamps53\">@bamps53</a> \nI trained 2nd level models on kernel.\nSince the execution time of the kernel is limited, it's necessary to use the optimizer with fast convergence.\nRAdam converged faster than SGD and Adam.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 621470,
      "author_name": "Qishen Ha",
      "author_url": "",
      "post_date": "2019-09-08T14:49:54.250000",
      "content": "<p>It seems that your models generalized best in the gold zone. Congratulations!</p>",
      "votes": 0,
      "replies": [
        {
          "id": 622203,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:30:32.420000",
          "content": "<p>Thanks!!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 621407,
      "author_name": "YaGana Sheriff-Hussaini",
      "author_url": "",
      "post_date": "2019-09-08T13:42:17.470000",
      "content": "<p>Congrats <a href=\"/uiiurz1\">@uiiurz1</a> and thanks for sharing your solution overview.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 622202,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:30:18.230000",
          "content": "<p>Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 621399,
      "author_name": "JM100",
      "author_url": "",
      "post_date": "2019-09-08T13:35:03.447000",
      "content": "<p>Congrats and thanks for sharing :)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 622201,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:30:02.380000",
          "content": "<p>Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 621012,
      "author_name": "Chanhu",
      "author_url": "",
      "post_date": "2019-09-08T06:36:30.337000",
      "content": "<p>Congratulations! <a href=\"/uiiurz1\">@uiiurz1</a> </p>",
      "votes": 0,
      "replies": [
        {
          "id": 621018,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-08T06:43:26.027000",
          "content": "<p>Thanks! <a href=\"/chanhu\">@chanhu</a> </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 621953,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-09-09T05:49:09.777000",
      "content": "",
      "votes": -1,
      "replies": [
        {
          "id": 622204,
          "author_name": "4ui_iurz1",
          "author_url": "",
          "post_date": "2019-09-09T11:30:50.540000",
          "content": "<p>Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "621011": "Congrats to all the competitors!\n\nlocal code: https://github.com/4uiiurz1/kaggle-aptos2019-blindness-detection\nkernel: https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution\n\n## Preprocessing\nI used only Ben's crop (scale radius).\n\n## Augmentation\nOutput image size is 256x256.\n```python\ntrain_transform = transforms.Compose([\n    transforms.Resize((288, 288)),\n    transforms.RandomAffine(\n        degrees=(-180, 180),\n        scale=(0.8889, 1.0),\n        shear=(-36, 36)),\n    transforms.CenterCrop(256),\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.RandomVerticalFlip(p=0.5),\n    transforms.ColorJitter(contrast=(0.9, 1.1)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n])\n```\n\n## 1st-level models (run on local)\n- Models: SE-ResNeXt50\\_32x4d, SE-ResNeXt101\\_32x4d, SENet154\n- Loss: MSE\n- Optimizer: SGD (momentum=0.9)\n- LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)\n- 30 epochs\n- Dataset: 2019 train dataset (5-folds cv) + 2015 dataset (like https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#581042)\n\n## 2nd-level models (run on [kernel](https://www.kaggle.com/uiiurz1/aptos-2019-14th-place-solution)) \n- Models: SE-ResNeXt50\\_32x4d, SE-ResNeXt101\\_32x4d (1st-level models' weights)\n- Loss: MSE\n- Optimizer: RAdam\n- LR scheduler: CosineAnnealingLR (lr=1e-3 -&gt; 1e-5)\n- 10 epochs\n- Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (**public + private**,  divided into 5 and used different data each fold. )\n- Pseudo labels: weighted average of 1st-level models\n\n## Ensemble\nFinally, averaged 2nd-level models' predictions.\n\n- PublicLB: 0.826\n- PrivateLB: 0.930\n\nFortunately, I selected the best submission.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F849476%2Ff697521599e135faf71027923c90b0f2%2Fmy_submissions.png?generation=1568189226782044&amp;alt=media)\n",
    "621077": "Congrats. Great work. If you can share  the code link, that would be great. Thanks. ",
    "621030": "Thank you for sharing and Congrats! ",
    "632278": "Sorry  for  late  to  say  thanks  ,  I  read  both   your     1st-level  code  and  2nd-level   code  carefully and  rerun  it , your  logic is  clear with  no  confusing  in your code . Very  simple but  very  useful  ,   thanks   for    your kind  to  open  your   code ,  I  learn  a  lot   from  your code .\n\none advise：I can't understand the file `SUMMARY.md` lives in `../markdowns/diabetic_retinopathy_detection`. ",
    "624522": "Congratulations…, could you share your private dataset of the kernel? I want to reproduce some experiments. Thanks\n\n",
    "624494": "@uiiurz1  In your 2nd level model, you say Dataset: 2019 train dataset (5-folds cv) + 2019 test dataset (public + private, divided into 5 and used different data each fold. )\n\nSo you trained your model in kernel using pseudo labels from private test set too?",
    "623627": "Thank you so much for sharing. Very generous of you!",
    "621479": "Thanks for sharing, your solution is very simple and beautiful!!\nI have a quick question, is RAdam is better than normal Adam or SGD on 2nd level model??",
    "621470": "It seems that your models generalized best in the gold zone. Congratulations!",
    "621407": "Congrats @uiiurz1 and thanks for sharing your solution overview.",
    "621399": "Congrats and thanks for sharing :)",
    "621012": "Congratulations! @uiiurz1 ",
    "621953": ""
  }
}