{
  "id": 103091,
  "title": "How long is it taking you to run your kernel, GPU ?",
  "url": "/competitions/aptos2019-blindness-detection/discussion/103091",
  "author_name": "",
  "post_date": "2019-08-07T03:55:29.154124200Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hello, \nI have a pretty simple kernel and it's taking me about 8 minutes to run one epoch, which means running the entire kernel can be over an hour.  Is this normal for a kernel, is it short?</p>\n\n<p>As a related point, how can I tell if my kernel is actually using the GPU ? I have GPU turned on, but when I talked with a friend, they suggested that it seemed like the kernel wasn't using the GPU.  I am using fast.ai . Is there a way I can tell if the GPU is actually being used ?</p>",
  "messages": [
    {
      "id": "593748",
      "postDate": "08/07/2019 03:55:29",
      "content": "<p>Hello, \nI have a pretty simple kernel and it's taking me about 8 minutes to run one epoch, which means running the entire kernel can be over an hour.  Is this normal for a kernel, is it short?</p>\n\n<p>As a related point, how can I tell if my kernel is actually using the GPU ? I have GPU turned on, but when I talked with a friend, they suggested that it seemed like the kernel wasn't using the GPU.  I am using fast.ai . Is there a way I can tell if the GPU is actually being used ?</p>",
      "rawMarkdown": "Hello, \nI have a pretty simple kernel and it's taking me about 8 minutes to run one epoch, which means running the entire kernel can be over an hour.  Is this normal for a kernel, is it short?\n\nAs a related point, how can I tell if my kernel is actually using the GPU ? I have GPU turned on, but when I talked with a friend, they suggested that it seemed like the kernel wasn't using the GPU.  I am using fast.ai . Is there a way I can tell if the GPU is actually being used ?",
      "votes": null
    },
    {
      "id": "593984",
      "postDate": "08/07/2019 11:43:35",
      "content": "<p>For me when I was using Fast.ai transfer learning for APTOS Blindness detection, one epoch in Resnet 152 architecture took 8 mins to complete while VGG19 architecture took 2 mins to complete.  </p>",
      "rawMarkdown": "For me when I was using Fast.ai transfer learning for APTOS Blindness detection, one epoch in Resnet 152 architecture took 8 mins to complete while VGG19 architecture took 2 mins to complete.",
      "votes": null
    },
    {
      "id": "594037",
      "postDate": "08/07/2019 13:32:26",
      "content": "<p>You can do \n<code>torch.cuda.is_available()</code>\nto check if fastai is using gpu, since it wraps pytorch.\nIt should return <code>True</code></p>\n\n<p>On my local machine, RTX2070 8gb, with using very little data transforms, it takes about 2 mins per epoch. I have managed to make it about 5 mins per epoch using a lot of data transforms, and my gpu is barely breaking a sweat. My CPU is on fire though. I think cpu is the biggest bottleneck for this competition. Maybe because of the very vastly different sized images, and the transformations to make them the right size for a neural net happens on the cpu.</p>",
      "rawMarkdown": "You can do \n`torch.cuda.is_available()`\nto check if fastai is using gpu, since it wraps pytorch.\nIt should return `True`\n\nOn my local machine, RTX2070 8gb, with using very little data transforms, it takes about 2 mins per epoch. I have managed to make it about 5 mins per epoch using a lot of data transforms, and my gpu is barely breaking a sweat. My CPU is on fire though. I think cpu is the biggest bottleneck for this competition. Maybe because of the very vastly different sized images, and the transformations to make them the right size for a neural net happens on the cpu.",
      "votes": null
    },
    {
      "id": "594380",
      "postDate": "08/07/2019 23:08:30",
      "content": "<p>As someone who is only using kernels, my efficientnets run for about 10 - 13 minutes per epoch but thats on the new AND old data and my whole training easily takes over 5 hours. That is why I just commit it overnight and run inference in the morning</p>",
      "rawMarkdown": "As someone who is only using kernels, my efficientnets run for about 10 - 13 minutes per epoch but thats on the new AND old data and my whole training easily takes over 5 hours. That is why I just commit it overnight and run inference in the morning",
      "votes": null
    },
    {
      "id": "594939",
      "postDate": "08/08/2019 17:02:21",
      "content": "<p>Thanks, \nI guess this is about as fast as it goes then. </p>",
      "rawMarkdown": "Thanks, \nI guess this is about as fast as it goes then.",
      "votes": null
    },
    {
      "id": "594943",
      "postDate": "08/08/2019 17:06:07",
      "content": "<p>Hmm, I will have to check that out. I hope it's available for Pytorch.  I am assuming your scores were about the same using each ?</p>",
      "rawMarkdown": "Hmm, I will have to check that out. I hope it's available for Pytorch.  I am assuming your scores were about the same using each ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 593984,
      "author_name": "manojprabhaakr",
      "author_url": "",
      "post_date": "08/07/2019 11:43:35",
      "content": "<p>For me when I was using Fast.ai transfer learning for APTOS Blindness detection, one epoch in Resnet 152 architecture took 8 mins to complete while VGG19 architecture took 2 mins to complete.  </p>",
      "votes": null,
      "replies": [
        {
          "id": 594943,
          "author_name": "chrisfs",
          "author_url": "",
          "post_date": "08/08/2019 17:06:07",
          "content": "<p>Hmm, I will have to check that out. I hope it's available for Pytorch.  I am assuming your scores were about the same using each ?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 594037,
      "author_name": "danshatzz",
      "author_url": "",
      "post_date": "08/07/2019 13:32:26",
      "content": "<p>You can do \n<code>torch.cuda.is_available()</code>\nto check if fastai is using gpu, since it wraps pytorch.\nIt should return <code>True</code></p>\n\n<p>On my local machine, RTX2070 8gb, with using very little data transforms, it takes about 2 mins per epoch. I have managed to make it about 5 mins per epoch using a lot of data transforms, and my gpu is barely breaking a sweat. My CPU is on fire though. I think cpu is the biggest bottleneck for this competition. Maybe because of the very vastly different sized images, and the transformations to make them the right size for a neural net happens on the cpu.</p>",
      "votes": null,
      "replies": [
        {
          "id": 594939,
          "author_name": "chrisfs",
          "author_url": "",
          "post_date": "08/08/2019 17:02:21",
          "content": "<p>Thanks, \nI guess this is about as fast as it goes then. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 594380,
      "author_name": "sidhanthholalkere",
      "author_url": "",
      "post_date": "08/07/2019 23:08:30",
      "content": "<p>As someone who is only using kernels, my efficientnets run for about 10 - 13 minutes per epoch but thats on the new AND old data and my whole training easily takes over 5 hours. That is why I just commit it overnight and run inference in the morning</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "593748": "Hello, \nI have a pretty simple kernel and it's taking me about 8 minutes to run one epoch, which means running the entire kernel can be over an hour.  Is this normal for a kernel, is it short?\n\nAs a related point, how can I tell if my kernel is actually using the GPU ? I have GPU turned on, but when I talked with a friend, they suggested that it seemed like the kernel wasn't using the GPU.  I am using fast.ai . Is there a way I can tell if the GPU is actually being used ?",
    "593984": "For me when I was using Fast.ai transfer learning for APTOS Blindness detection, one epoch in Resnet 152 architecture took 8 mins to complete while VGG19 architecture took 2 mins to complete.",
    "594037": "You can do \n`torch.cuda.is_available()`\nto check if fastai is using gpu, since it wraps pytorch.\nIt should return `True`\n\nOn my local machine, RTX2070 8gb, with using very little data transforms, it takes about 2 mins per epoch. I have managed to make it about 5 mins per epoch using a lot of data transforms, and my gpu is barely breaking a sweat. My CPU is on fire though. I think cpu is the biggest bottleneck for this competition. Maybe because of the very vastly different sized images, and the transformations to make them the right size for a neural net happens on the cpu.",
    "594380": "As someone who is only using kernels, my efficientnets run for about 10 - 13 minutes per epoch but thats on the new AND old data and my whole training easily takes over 5 hours. That is why I just commit it overnight and run inference in the morning",
    "594939": "Thanks, \nI guess this is about as fast as it goes then.",
    "594943": "Hmm, I will have to check that out. I hope it's available for Pytorch.  I am assuming your scores were about the same using each ?"
  },
  "source": "meta"
}