{
  "id": 134602,
  "title": "Inference speed issue related to pytorch",
  "url": "/competitions/bengaliai-cv19/discussion/134602",
  "author_name": "",
  "post_date": "2020-03-09T07:20:55.918798500Z",
  "votes": 14,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi,\nI found a strange issue with an inference speed, try yourself. </p>\n\n<p>With one of the torchvision model I infer 400+ 224x224 images per second locally (1080ti) but less than 100/second in Kaggle's kernel (P100 I guess). I/O issue? Not really, I removed all cpu I/O code and still the same.</p>\n\n<p>Installing torch==1.2.0 and torchvision==0.4.0 (compared to 1.4.0 and 0.5.0) solved my problem - almost x5 speedup, same time as locally. What do you think is it related to?</p>",
  "messages": [
    {
      "id": "767110",
      "postDate": "03/09/2020 07:20:55",
      "content": "<p>Hi,\nI found a strange issue with an inference speed, try yourself. </p>\n\n<p>With one of the torchvision model I infer 400+ 224x224 images per second locally (1080ti) but less than 100/second in Kaggle's kernel (P100 I guess). I/O issue? Not really, I removed all cpu I/O code and still the same.</p>\n\n<p>Installing torch==1.2.0 and torchvision==0.4.0 (compared to 1.4.0 and 0.5.0) solved my problem - almost x5 speedup, same time as locally. What do you think is it related to?</p>",
      "rawMarkdown": "Hi,\nI found a strange issue with an inference speed, try yourself. \n\nWith one of the torchvision model I infer 400+ 224x224 images per second locally (1080ti) but less than 100/second in Kaggle's kernel (P100 I guess). I/O issue? Not really, I removed all cpu I/O code and still the same.\n\nInstalling torch==1.2.0 and torchvision==0.4.0 (compared to 1.4.0 and 0.5.0) solved my problem - almost x5 speedup, same time as locally. What do you think is it related to?",
      "votes": null
    },
    {
      "id": "767140",
      "postDate": "03/09/2020 08:06:13",
      "content": "<p>Has anyone tried to install pytorch==1.2.0 in the kernels? Would be very helpful 👍 </p>",
      "rawMarkdown": "Has anyone tried to install pytorch==1.2.0 in the kernels? Would be very helpful 👍",
      "votes": null
    },
    {
      "id": "767252",
      "postDate": "03/09/2020 11:21:51",
      "content": "<blockquote>\n  <p>I removed all cpu I/O code and still the same.\n  What do you mean?  How do you mode images from storage to the gpu?</p>\n</blockquote>",
      "rawMarkdown": "&gt;  I removed all cpu I/O code and still the same.\nWhat do you mean?  How do you mode images from storage to the gpu?",
      "votes": null
    },
    {
      "id": "767276",
      "postDate": "03/09/2020 12:16:37",
      "content": "<p>I meant I tested the gpu performance on the same image in the loop to avoid i/o operations</p>",
      "rawMarkdown": "I meant I tested the gpu performance on the same image in the loop to avoid i/o operations",
      "votes": null
    },
    {
      "id": "767542",
      "postDate": "03/09/2020 19:20:43",
      "content": "<p>you used one image or a minibatch?  </p>",
      "rawMarkdown": "you used one image or a minibatch?",
      "votes": null
    },
    {
      "id": "767857",
      "postDate": "03/10/2020 07:12:19",
      "content": "<p>minibatch for sure.\nthe issue is related to torchvision only, not to efNets etc</p>",
      "rawMarkdown": "minibatch for sure.\nthe issue is related to torchvision only, not to efNets etc",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 767140,
      "author_name": "yaroshevskiy",
      "author_url": "",
      "post_date": "03/09/2020 08:06:13",
      "content": "<p>Has anyone tried to install pytorch==1.2.0 in the kernels? Would be very helpful 👍 </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 767252,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "03/09/2020 11:21:51",
      "content": "<blockquote>\n  <p>I removed all cpu I/O code and still the same.\n  What do you mean?  How do you mode images from storage to the gpu?</p>\n</blockquote>",
      "votes": null,
      "replies": [
        {
          "id": 767276,
          "author_name": "yaroshevskiy",
          "author_url": "",
          "post_date": "03/09/2020 12:16:37",
          "content": "<p>I meant I tested the gpu performance on the same image in the loop to avoid i/o operations</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 767542,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "03/09/2020 19:20:43",
          "content": "<p>you used one image or a minibatch?  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 767857,
          "author_name": "yaroshevskiy",
          "author_url": "",
          "post_date": "03/10/2020 07:12:19",
          "content": "<p>minibatch for sure.\nthe issue is related to torchvision only, not to efNets etc</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "767110": "Hi,\nI found a strange issue with an inference speed, try yourself. \n\nWith one of the torchvision model I infer 400+ 224x224 images per second locally (1080ti) but less than 100/second in Kaggle's kernel (P100 I guess). I/O issue? Not really, I removed all cpu I/O code and still the same.\n\nInstalling torch==1.2.0 and torchvision==0.4.0 (compared to 1.4.0 and 0.5.0) solved my problem - almost x5 speedup, same time as locally. What do you think is it related to?",
    "767140": "Has anyone tried to install pytorch==1.2.0 in the kernels? Would be very helpful 👍",
    "767252": "&gt;  I removed all cpu I/O code and still the same.\nWhat do you mean?  How do you mode images from storage to the gpu?",
    "767276": "I meant I tested the gpu performance on the same image in the loop to avoid i/o operations",
    "767542": "you used one image or a minibatch?",
    "767857": "minibatch for sure.\nthe issue is related to torchvision only, not to efNets etc"
  },
  "source": "meta"
}