{
  "id": 302443,
  "title": "how is the batch size related to LB using yolov5?",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/302443",
  "author_name": "",
  "post_date": "2022-01-22T15:20:05.972405Z",
  "votes": 7,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Due to GPU RAM limitation,  I only can train with batch_size =1,  which I got at most 0.61x LB,</p>\n<p>why the weight shared by Sheep using batch_size=4 with LB 0.62x.</p>\n<p>Anyone boosts LB using big batch size?</p>",
  "messages": [
    {
      "id": "1660302",
      "postDate": "01/22/2022 15:20:05",
      "content": "<p>Due to GPU RAM limitation,  I only can train with batch_size =1,  which I got at most 0.61x LB,</p>\n<p>why the weight shared by Sheep using batch_size=4 with LB 0.62x.</p>\n<p>Anyone boosts LB using big batch size?</p>",
      "rawMarkdown": "Due to GPU RAM limitation,  I only can train with batch_size =1,  which I got at most 0.61x LB,\n\nwhy the weight shared by Sheep using batch_size=4 with LB 0.62x.\n\nAnyone boosts LB using big batch size?",
      "votes": null
    },
    {
      "id": "1660595",
      "postDate": "01/22/2022 19:22:40",
      "content": "<p>I don't have a GPU but what is yours GPU RAM limit? Besides, someone knows what is the GPU RAM limit in kaggle GPU accelerator? (I've just started at kaggle)</p>",
      "rawMarkdown": "I don't have a GPU but what is yours GPU RAM limit? Besides, someone knows what is the GPU RAM limit in kaggle GPU accelerator? (I've just started at kaggle)",
      "votes": null
    },
    {
      "id": "1660647",
      "postDate": "01/22/2022 20:26:33",
      "content": "<p>You can check this using </p>\n<p><code>!nvidia-smi</code></p>\n<p>Tesla P100 / 16280MiB</p>",
      "rawMarkdown": "You can check this using \n\n`!nvidia-smi `\n\nTesla P100 / 16280MiB",
      "votes": null
    },
    {
      "id": "1662401",
      "postDate": "01/24/2022 08:50:25",
      "content": "<p>If you have enough GPU RAM, higher batch size means smoother loss gradients, and generally better training result.</p>\n<p>Having said that, if you have enough GPU RAM, you might be better off increasing resolution even further; who knows…</p>",
      "rawMarkdown": "If you have enough GPU RAM, higher batch size means smoother loss gradients, and generally better training result.\n\nHaving said that, if you have enough GPU RAM, you might be better off increasing resolution even further; who knows...",
      "votes": null
    },
    {
      "id": "1662592",
      "postDate": "01/24/2022 12:10:01",
      "content": "<p>thx for your reply. I just have the kaggle/colab pro, 16G ram is the limit.</p>",
      "rawMarkdown": "thx for your reply. I just have the kaggle/colab pro, 16G ram is the limit.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1660595,
      "author_name": "henriqueabpassos",
      "author_url": "",
      "post_date": "01/22/2022 19:22:40",
      "content": "<p>I don't have a GPU but what is yours GPU RAM limit? Besides, someone knows what is the GPU RAM limit in kaggle GPU accelerator? (I've just started at kaggle)</p>",
      "votes": null,
      "replies": [
        {
          "id": 1660647,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "01/22/2022 20:26:33",
          "content": "<p>You can check this using </p>\n<p><code>!nvidia-smi</code></p>\n<p>Tesla P100 / 16280MiB</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1662401,
      "author_name": "alexchwong",
      "author_url": "",
      "post_date": "01/24/2022 08:50:25",
      "content": "<p>If you have enough GPU RAM, higher batch size means smoother loss gradients, and generally better training result.</p>\n<p>Having said that, if you have enough GPU RAM, you might be better off increasing resolution even further; who knows…</p>",
      "votes": null,
      "replies": [
        {
          "id": 1662592,
          "author_name": "dragonzhang",
          "author_url": "",
          "post_date": "01/24/2022 12:10:01",
          "content": "<p>thx for your reply. I just have the kaggle/colab pro, 16G ram is the limit.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1660302": "Due to GPU RAM limitation,  I only can train with batch_size =1,  which I got at most 0.61x LB,\n\nwhy the weight shared by Sheep using batch_size=4 with LB 0.62x.\n\nAnyone boosts LB using big batch size?",
    "1660595": "I don't have a GPU but what is yours GPU RAM limit? Besides, someone knows what is the GPU RAM limit in kaggle GPU accelerator? (I've just started at kaggle)",
    "1660647": "You can check this using \n\n`!nvidia-smi `\n\nTesla P100 / 16280MiB",
    "1662401": "If you have enough GPU RAM, higher batch size means smoother loss gradients, and generally better training result.\n\nHaving said that, if you have enough GPU RAM, you might be better off increasing resolution even further; who knows...",
    "1662592": "thx for your reply. I just have the kaggle/colab pro, 16G ram is the limit."
  },
  "source": "meta"
}