{
  "id": 376863,
  "title": "Training time per epoch?",
  "url": "/competitions/nfl-player-contact-detection/discussion/376863",
  "author_name": "",
  "post_date": "2023-01-08T22:40:31.329053900Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>per epoch the training time was 1 hr.<br>\nHow long does it take for others?</p>",
  "messages": [
    {
      "id": "2091972",
      "postDate": "01/08/2023 22:40:31",
      "content": "<p>per epoch the training time was 1 hr.<br>\nHow long does it take for others?</p>",
      "rawMarkdown": "per epoch the training time was 1 hr.\nHow long does it take for others?",
      "votes": null
    },
    {
      "id": "2092042",
      "postDate": "01/09/2023 02:13:17",
      "content": "<p>2 hr ~ 5 hr</p>",
      "rawMarkdown": "2 hr ~ 5 hr",
      "votes": null
    },
    {
      "id": "2093758",
      "postDate": "01/10/2023 09:44:27",
      "content": "<p>It depends on the network and GPU. How about your model and device?</p>",
      "rawMarkdown": "It depends on the network and GPU. How about your model and device?",
      "votes": null
    },
    {
      "id": "2093821",
      "postDate": "01/10/2023 10:50:26",
      "content": "<p>I use Efficientnet and the GPU is 12GB training time 2 hrs 30 mins per epoch</p>",
      "rawMarkdown": "I use Efficientnet and the GPU is 12GB training time 2 hrs 30 mins per epoch",
      "votes": null
    },
    {
      "id": "2101049",
      "postDate": "01/15/2023 15:41:27",
      "content": "<p>Things that come to my mind:</p>\n<ul>\n<li>Did you try lowering precision? (to half, use mixed precision, …)</li>\n<li>Did you resize images or model to get shorter training time?</li>\n<li>Do you have any fancy augmentation methods or do some processing during training?</li>\n<li>How fast does the model need to converge?</li>\n</ul>",
      "rawMarkdown": "Things that come to my mind:\n\n- Did you try lowering precision? (to half, use mixed precision, ...)\n- Did you resize images or model to get shorter training time?\n- Do you have any fancy augmentation methods or do some processing during training?\n- How fast does the model need to converge?",
      "votes": null
    },
    {
      "id": "2137397",
      "postDate": "02/10/2023 00:15:17",
      "content": "<p>I'm getting around 3 hours per epoch. The bottleneck is the CPU while reading the frames for me.</p>",
      "rawMarkdown": "I'm getting around 3 hours per epoch. The bottleneck is the CPU while reading the frames for me.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2092042,
      "author_name": "crazywinds",
      "author_url": "",
      "post_date": "01/09/2023 02:13:17",
      "content": "<p>2 hr ~ 5 hr</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2093758,
      "author_name": "sakura1986",
      "author_url": "",
      "post_date": "01/10/2023 09:44:27",
      "content": "<p>It depends on the network and GPU. How about your model and device?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2093821,
          "author_name": "arunodhayan",
          "author_url": "",
          "post_date": "01/10/2023 10:50:26",
          "content": "<p>I use Efficientnet and the GPU is 12GB training time 2 hrs 30 mins per epoch</p>",
          "votes": null,
          "replies": [
            {
              "id": 2137397,
              "author_name": "omarabidthegreat",
              "author_url": "",
              "post_date": "02/10/2023 00:15:17",
              "content": "<p>I'm getting around 3 hours per epoch. The bottleneck is the CPU while reading the frames for me.</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2101049,
      "author_name": "aliabdin1",
      "author_url": "",
      "post_date": "01/15/2023 15:41:27",
      "content": "<p>Things that come to my mind:</p>\n<ul>\n<li>Did you try lowering precision? (to half, use mixed precision, …)</li>\n<li>Did you resize images or model to get shorter training time?</li>\n<li>Do you have any fancy augmentation methods or do some processing during training?</li>\n<li>How fast does the model need to converge?</li>\n</ul>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2091972": "per epoch the training time was 1 hr.\nHow long does it take for others?",
    "2092042": "2 hr ~ 5 hr",
    "2093758": "It depends on the network and GPU. How about your model and device?",
    "2093821": "I use Efficientnet and the GPU is 12GB training time 2 hrs 30 mins per epoch",
    "2101049": "Things that come to my mind:\n\n- Did you try lowering precision? (to half, use mixed precision, ...)\n- Did you resize images or model to get shorter training time?\n- Do you have any fancy augmentation methods or do some processing during training?\n- How fast does the model need to converge?",
    "2137397": "I'm getting around 3 hours per epoch. The bottleneck is the CPU while reading the frames for me."
  },
  "source": "meta"
}