{
  "id": 199485,
  "title": "Moving bottleneck from CPU to GPU",
  "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/199485",
  "author_name": "Miroslav Valan",
  "post_date": "2020-11-25T23:18:07.231000",
  "votes": 5,
  "comment_count": 0,
  "views": 0,
  "content": "<p>as I mentioned <a href=\"https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/196602\" target=\"_blank\">here</a> there is a simple trick one can use to avoid CPU bottleneck while rasterizing on the fly. The bottleneck is the dataloader so you just avoid it.</p>\n<p>Let's assume we wanna train a few models with different architectures, optimizes, augmentations, regularization. As long as we use the same parameters for rasterization we can do something like this:</p>\n<pre><code># pseudo code\nfor batch in dataloader:\n    model1.fit()\n    model2.fit()\n    model3.fit()\n    modelN.fit()\n</code></pre>\n<p>Here is <a href=\"https://www.kaggle.com/valanm/avoiding-cpu-bottleneck\" target=\"_blank\">the kernel</a> that demonstrates it.  Training of 2x resnet18, 2x resnet34, 2x resnet50 lasted as a single resnet18.</p>",
  "messages": [
    {
      "id": 1091292,
      "postDate": "2020-11-25T23:18:07.233Z",
      "content": "<p>as I mentioned <a href=\"https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/196602\" target=\"_blank\">here</a> there is a simple trick one can use to avoid CPU bottleneck while rasterizing on the fly. The bottleneck is the dataloader so you just avoid it.</p>\n<p>Let's assume we wanna train a few models with different architectures, optimizes, augmentations, regularization. As long as we use the same parameters for rasterization we can do something like this:</p>\n<pre><code># pseudo code\nfor batch in dataloader:\n    model1.fit()\n    model2.fit()\n    model3.fit()\n    modelN.fit()\n</code></pre>\n<p>Here is <a href=\"https://www.kaggle.com/valanm/avoiding-cpu-bottleneck\" target=\"_blank\">the kernel</a> that demonstrates it.  Training of 2x resnet18, 2x resnet34, 2x resnet50 lasted as a single resnet18.</p>",
      "rawMarkdown": "as I mentioned [here](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/196602) there is a simple trick one can use to avoid CPU bottleneck while rasterizing on the fly. The bottleneck is the dataloader so you just avoid it.\n\nLet's assume we wanna train a few models with different architectures, optimizes, augmentations, regularization. As long as we use the same parameters for rasterization we can do something like this:\n\n   ```\n# pseudo code\nfor batch in dataloader:\n    model1.fit()\n    model2.fit()\n    model3.fit()\n    modelN.fit()\n```\nHere is [the kernel](https://www.kaggle.com/valanm/avoiding-cpu-bottleneck) that demonstrates it.  Training of 2x resnet18, 2x resnet34, 2x resnet50 lasted as a single resnet18.",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1091292": "as I mentioned [here](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/discussion/196602) there is a simple trick one can use to avoid CPU bottleneck while rasterizing on the fly. The bottleneck is the dataloader so you just avoid it.\n\nLet's assume we wanna train a few models with different architectures, optimizes, augmentations, regularization. As long as we use the same parameters for rasterization we can do something like this:\n\n   ```\n# pseudo code\nfor batch in dataloader:\n    model1.fit()\n    model2.fit()\n    model3.fit()\n    modelN.fit()\n```\nHere is [the kernel](https://www.kaggle.com/valanm/avoiding-cpu-bottleneck) that demonstrates it.  Training of 2x resnet18, 2x resnet34, 2x resnet50 lasted as a single resnet18."
  }
}