{
  "id": 234952,
  "title": "LB drops with 512x512 tiles",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/234952",
  "author_name": "",
  "post_date": "2021-04-27T02:36:05.997948600Z",
  "votes": 9,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Need Help!<br>\nWhen the size of the tile is 256x256, the CV is 0.9330 and LB is 0.920<br>\nFor the case of 512x512, the CV is 0.9322 and LB is about 0.890<br>\nWhy do the results drop after I increased the input size? Does this have anything to do with batch size?</p>\n<p>model: unet + efficientnet<br>\n4-fold<br>\nbatch size: 16 for 256x256 and 8 for 512x512, both use 2 gpus</p>",
  "messages": [
    {
      "id": "1285492",
      "postDate": "04/27/2021 02:36:05",
      "content": "<p>Need Help!<br>\nWhen the size of the tile is 256x256, the CV is 0.9330 and LB is 0.920<br>\nFor the case of 512x512, the CV is 0.9322 and LB is about 0.890<br>\nWhy do the results drop after I increased the input size? Does this have anything to do with batch size?</p>\n<p>model: unet + efficientnet<br>\n4-fold<br>\nbatch size: 16 for 256x256 and 8 for 512x512, both use 2 gpus</p>",
      "rawMarkdown": "Need Help!\nWhen the size of the tile is 256x256, the CV is 0.9330 and LB is 0.920\nFor the case of 512x512, the CV is 0.9322 and LB is about 0.890\nWhy do the results drop after I increased the input size? Does this have anything to do with batch size?\n\nmodel: unet + efficientnet\n4-fold\nbatch size: 16 for 256x256 and 8 for 512x512, both use 2 gpus",
      "votes": null
    },
    {
      "id": "1285619",
      "postDate": "04/27/2021 06:15:56",
      "content": "<p>It could be. Small batch size makes BN layers inconsistent. You might try to train with 256x256 at the first then switch to 512x512 with smaller batch size and BN frozen </p>",
      "rawMarkdown": "It could be. Small batch size makes BN layers inconsistent. You might try to train with 256x256 at the first then switch to 512x512 with smaller batch size and BN frozen",
      "votes": null
    },
    {
      "id": "1285739",
      "postDate": "04/27/2021 08:17:15",
      "content": "<p>newbee question. how to realize BN forzen.</p>",
      "rawMarkdown": "newbee question. how to realize BN forzen.",
      "votes": null
    },
    {
      "id": "1286043",
      "postDate": "04/27/2021 14:14:38",
      "content": "<p>You may also want to change the type of encoder that you use (e.g. try to use a more powerful encoder), but I guess efficientnet is good enough already. Batch size changes will also affect the results (already mentioned BN issue below). It also could be that you need to train longer to achieve the same results. </p>",
      "rawMarkdown": "You may also want to change the type of encoder that you use (e.g. try to use a more powerful encoder), but I guess efficientnet is good enough already. Batch size changes will also affect the results (already mentioned BN issue below). It also could be that you need to train longer to achieve the same results.",
      "votes": null
    },
    {
      "id": "1286590",
      "postDate": "04/28/2021 07:17:10",
      "content": "<p>It might be possible that you are using the same encoder for size of tiles. Try using a bigger encoder for 512 tiles, my performance improved when I used effnet b4 on 512</p>",
      "rawMarkdown": "It might be possible that you are using the same encoder for size of tiles. Try using a bigger encoder for 512 tiles, my performance improved when I used effnet b4 on 512",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1285619,
      "author_name": "jihangz",
      "author_url": "",
      "post_date": "04/27/2021 06:15:56",
      "content": "<p>It could be. Small batch size makes BN layers inconsistent. You might try to train with 256x256 at the first then switch to 512x512 with smaller batch size and BN frozen </p>",
      "votes": null,
      "replies": [
        {
          "id": 1285739,
          "author_name": "southsakura",
          "author_url": "",
          "post_date": "04/27/2021 08:17:15",
          "content": "<p>newbee question. how to realize BN forzen.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1286043,
      "author_name": "resolut",
      "author_url": "",
      "post_date": "04/27/2021 14:14:38",
      "content": "<p>You may also want to change the type of encoder that you use (e.g. try to use a more powerful encoder), but I guess efficientnet is good enough already. Batch size changes will also affect the results (already mentioned BN issue below). It also could be that you need to train longer to achieve the same results. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1286590,
      "author_name": "deveshdarshan",
      "author_url": "",
      "post_date": "04/28/2021 07:17:10",
      "content": "<p>It might be possible that you are using the same encoder for size of tiles. Try using a bigger encoder for 512 tiles, my performance improved when I used effnet b4 on 512</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1285492": "Need Help!\nWhen the size of the tile is 256x256, the CV is 0.9330 and LB is 0.920\nFor the case of 512x512, the CV is 0.9322 and LB is about 0.890\nWhy do the results drop after I increased the input size? Does this have anything to do with batch size?\n\nmodel: unet + efficientnet\n4-fold\nbatch size: 16 for 256x256 and 8 for 512x512, both use 2 gpus",
    "1285619": "It could be. Small batch size makes BN layers inconsistent. You might try to train with 256x256 at the first then switch to 512x512 with smaller batch size and BN frozen",
    "1285739": "newbee question. how to realize BN forzen.",
    "1286043": "You may also want to change the type of encoder that you use (e.g. try to use a more powerful encoder), but I guess efficientnet is good enough already. Batch size changes will also affect the results (already mentioned BN issue below). It also could be that you need to train longer to achieve the same results.",
    "1286590": "It might be possible that you are using the same encoder for size of tiles. Try using a bigger encoder for 512 tiles, my performance improved when I used effnet b4 on 512"
  },
  "source": "meta"
}