{
  "id": 133954,
  "title": "Basics question on using facenet_pytorch (in a loop) with efficient memory",
  "url": "/competitions/deepfake-detection-challenge/discussion/133954",
  "author_name": "",
  "post_date": "2020-03-05T05:39:32.403225900Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi,</p>\n\n<p>I am struggling with memory issues. I extracted 300 faces (named <code>allFaces</code>) from the first video using mtcnn and stored it on disk (as torch tensor).</p>\n\n<p>Now, my code simply load the libraries and <code>allFaces</code> (dimesnion: 300x3x160x160), at this point when I call: <code>GPUtil.getGPUs()[0].memoryFre</code>e I get a number ? 9K so (&gt;9GB is free in my GPU).</p>\n\n<p>Now, I can't calculate embeddings for 300 faces in one go. So I am trying to do it in a loop as:</p>\n\n<p><code>\nmtcnn = MTCNN(device=device)#, device=device) #post_process=False\nresnet = InceptionResnetV1(pretrained='vggface2', device=device).eval()\n...\ndef getAllFaceEmb(allFaces):\n    global resnet\n    allEmb = []\n    for i in range(0, len(allFaces), 50):\n        print(GPUtil.getGPUs()[0].memoryFree)\n        allEmb.append(resnet(allFaces[i:i+50]))\n    return torch.cat(allEmb)\n</code></p>\n\n<p>But it fails while trying to extract embeddings for index 150:300, It gives output as:\n```</p>\n\n<p>9507.0\n7787.0\n6101.0\n4403.0\n2717.0</p>\n\n<h2>1031.0</h2>\n\n<p>RuntimeError                              Traceback (most recent call last)</p>\n\n<p>ipython-input-11-cf8848b57f8c in module\n----&gt; 1 allEmb = getAllFaceEmb(allFaces)</p>\n\n<p>ipython-input-10-482f0a18f064 in getAllFaceEmb(allFaces)\n      4     for i in range(0, len(allFaces), 50):\n      5         print(GPUtil.getGPUs()[0].memoryFree)\n----&gt; 6         allEmb.append(resnet(allFaces[i:i+50]))\n      7     return torch.cat(allEmb)\nRuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 10.76 GiB total capacity; 9.29 GiB already allocated; 14.88 MiB free; 9.47 GiB reserved in total by PyTorch)</p>\n\n<p>```</p>\n\n<p>So, how can I do this with efficient usage of memory? Is there any tutorial which explains this?</p>",
  "messages": [
    {
      "id": "764071",
      "postDate": "03/05/2020 05:39:32",
      "content": "<p>Hi,</p>\n\n<p>I am struggling with memory issues. I extracted 300 faces (named <code>allFaces</code>) from the first video using mtcnn and stored it on disk (as torch tensor).</p>\n\n<p>Now, my code simply load the libraries and <code>allFaces</code> (dimesnion: 300x3x160x160), at this point when I call: <code>GPUtil.getGPUs()[0].memoryFre</code>e I get a number ? 9K so (&gt;9GB is free in my GPU).</p>\n\n<p>Now, I can't calculate embeddings for 300 faces in one go. So I am trying to do it in a loop as:</p>\n\n<p><code>\nmtcnn = MTCNN(device=device)#, device=device) #post_process=False\nresnet = InceptionResnetV1(pretrained='vggface2', device=device).eval()\n...\ndef getAllFaceEmb(allFaces):\n    global resnet\n    allEmb = []\n    for i in range(0, len(allFaces), 50):\n        print(GPUtil.getGPUs()[0].memoryFree)\n        allEmb.append(resnet(allFaces[i:i+50]))\n    return torch.cat(allEmb)\n</code></p>\n\n<p>But it fails while trying to extract embeddings for index 150:300, It gives output as:\n```</p>\n\n<p>9507.0\n7787.0\n6101.0\n4403.0\n2717.0</p>\n\n<h2>1031.0</h2>\n\n<p>RuntimeError                              Traceback (most recent call last)</p>\n\n<p>ipython-input-11-cf8848b57f8c in module\n----&gt; 1 allEmb = getAllFaceEmb(allFaces)</p>\n\n<p>ipython-input-10-482f0a18f064 in getAllFaceEmb(allFaces)\n      4     for i in range(0, len(allFaces), 50):\n      5         print(GPUtil.getGPUs()[0].memoryFree)\n----&gt; 6         allEmb.append(resnet(allFaces[i:i+50]))\n      7     return torch.cat(allEmb)\nRuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 10.76 GiB total capacity; 9.29 GiB already allocated; 14.88 MiB free; 9.47 GiB reserved in total by PyTorch)</p>\n\n<p>```</p>\n\n<p>So, how can I do this with efficient usage of memory? Is there any tutorial which explains this?</p>",
      "rawMarkdown": "Hi,\n\nI am struggling with memory issues. I extracted 300 faces (named `allFaces`) from the first video using mtcnn and stored it on disk (as torch tensor).\n\nNow, my code simply load the libraries and `allFaces` (dimesnion: 300x3x160x160), at this point when I call: `GPUtil.getGPUs()[0].memoryFre`e I get a number ? 9K so (&gt;9GB is free in my GPU).\n\nNow, I can't calculate embeddings for 300 faces in one go. So I am trying to do it in a loop as:\n\n\n\n```\nmtcnn = MTCNN(device=device)#, device=device) #post_process=False\nresnet = InceptionResnetV1(pretrained='vggface2', device=device).eval()\n...\ndef getAllFaceEmb(allFaces):\n    global resnet\n    allEmb = []\n    for i in range(0, len(allFaces), 50):\n        print(GPUtil.getGPUs()[0].memoryFree)\n        allEmb.append(resnet(allFaces[i:i+50]))\n    return torch.cat(allEmb)\n```\n\nBut it fails while trying to extract embeddings for index 150:300, It gives output as:\n```\n\n9507.0\n7787.0\n6101.0\n4403.0\n2717.0\n1031.0\n---------------------------------------------------------------------------\nRuntimeError                              Traceback (most recent call last)\n\nipython-input-11-cf8848b57f8c in module\n----&gt; 1 allEmb = getAllFaceEmb(allFaces)\n\nipython-input-10-482f0a18f064 in getAllFaceEmb(allFaces)\n      4     for i in range(0, len(allFaces), 50):\n      5         print(GPUtil.getGPUs()[0].memoryFree)\n----&gt; 6         allEmb.append(resnet(allFaces[i:i+50]))\n      7     return torch.cat(allEmb)\nRuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 10.76 GiB total capacity; 9.29 GiB already allocated; 14.88 MiB free; 9.47 GiB reserved in total by PyTorch)\n\n```\n\nSo, how can I do this with efficient usage of memory? Is there any tutorial which explains this?",
      "votes": null
    },
    {
      "id": "764141",
      "postDate": "03/05/2020 07:01:24",
      "content": "<p>I'm not 100% sure on your code, but it looks like you need to use a smaller batch size.  Try running on say, 20 faces at a time instead of 50</p>",
      "rawMarkdown": "I'm not 100% sure on your code, but it looks like you need to use a smaller batch size.  Try running on say, 20 faces at a time instead of 50",
      "votes": null
    },
    {
      "id": "764232",
      "postDate": "03/05/2020 08:42:35",
      "content": "<p>Good kernel</p>",
      "rawMarkdown": "Good kernel",
      "votes": null
    },
    {
      "id": "764324",
      "postDate": "03/05/2020 10:48:50",
      "content": "<p>Trying wrapping the whole thing into a <code>with torch.no_grad():</code> block, so that it doesn't compute the gradients (which I'm assuming you won't need here). That should save some memory. (And make sure to call <code>resnet.eval()</code> beforehand too.)</p>\n\n<p>Also not sure what you're doing once the embeddings have been calculated, but if you're holding on to these tensors it's not a surprise that your free GPU memory goes down over time.</p>",
      "rawMarkdown": "Trying wrapping the whole thing into a `with torch.no_grad():` block, so that it doesn't compute the gradients (which I'm assuming you won't need here). That should save some memory. (And make sure to call `resnet.eval()` beforehand too.)\n\nAlso not sure what you're doing once the embeddings have been calculated, but if you're holding on to these tensors it's not a surprise that your free GPU memory goes down over time.",
      "votes": null
    },
    {
      "id": "764693",
      "postDate": "03/05/2020 18:37:07",
      "content": "<p>Thanks with no_grad, I can even get embedding of all the 300 in one go.</p>",
      "rawMarkdown": "Thanks with no_grad, I can even get embedding of all the 300 in one go.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 764141,
      "author_name": "maierman",
      "author_url": "",
      "post_date": "03/05/2020 07:01:24",
      "content": "<p>I'm not 100% sure on your code, but it looks like you need to use a smaller batch size.  Try running on say, 20 faces at a time instead of 50</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 764232,
      "author_name": "vladislav0804",
      "author_url": "",
      "post_date": "03/05/2020 08:42:35",
      "content": "<p>Good kernel</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 764324,
      "author_name": "humananalog",
      "author_url": "",
      "post_date": "03/05/2020 10:48:50",
      "content": "<p>Trying wrapping the whole thing into a <code>with torch.no_grad():</code> block, so that it doesn't compute the gradients (which I'm assuming you won't need here). That should save some memory. (And make sure to call <code>resnet.eval()</code> beforehand too.)</p>\n\n<p>Also not sure what you're doing once the embeddings have been calculated, but if you're holding on to these tensors it's not a surprise that your free GPU memory goes down over time.</p>",
      "votes": null,
      "replies": [
        {
          "id": 764693,
          "author_name": "aknirala",
          "author_url": "",
          "post_date": "03/05/2020 18:37:07",
          "content": "<p>Thanks with no_grad, I can even get embedding of all the 300 in one go.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "764071": "Hi,\n\nI am struggling with memory issues. I extracted 300 faces (named `allFaces`) from the first video using mtcnn and stored it on disk (as torch tensor).\n\nNow, my code simply load the libraries and `allFaces` (dimesnion: 300x3x160x160), at this point when I call: `GPUtil.getGPUs()[0].memoryFre`e I get a number ? 9K so (&gt;9GB is free in my GPU).\n\nNow, I can't calculate embeddings for 300 faces in one go. So I am trying to do it in a loop as:\n\n\n\n```\nmtcnn = MTCNN(device=device)#, device=device) #post_process=False\nresnet = InceptionResnetV1(pretrained='vggface2', device=device).eval()\n...\ndef getAllFaceEmb(allFaces):\n    global resnet\n    allEmb = []\n    for i in range(0, len(allFaces), 50):\n        print(GPUtil.getGPUs()[0].memoryFree)\n        allEmb.append(resnet(allFaces[i:i+50]))\n    return torch.cat(allEmb)\n```\n\nBut it fails while trying to extract embeddings for index 150:300, It gives output as:\n```\n\n9507.0\n7787.0\n6101.0\n4403.0\n2717.0\n1031.0\n---------------------------------------------------------------------------\nRuntimeError                              Traceback (most recent call last)\n\nipython-input-11-cf8848b57f8c in module\n----&gt; 1 allEmb = getAllFaceEmb(allFaces)\n\nipython-input-10-482f0a18f064 in getAllFaceEmb(allFaces)\n      4     for i in range(0, len(allFaces), 50):\n      5         print(GPUtil.getGPUs()[0].memoryFree)\n----&gt; 6         allEmb.append(resnet(allFaces[i:i+50]))\n      7     return torch.cat(allEmb)\nRuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 10.76 GiB total capacity; 9.29 GiB already allocated; 14.88 MiB free; 9.47 GiB reserved in total by PyTorch)\n\n```\n\nSo, how can I do this with efficient usage of memory? Is there any tutorial which explains this?",
    "764141": "I'm not 100% sure on your code, but it looks like you need to use a smaller batch size.  Try running on say, 20 faces at a time instead of 50",
    "764232": "Good kernel",
    "764324": "Trying wrapping the whole thing into a `with torch.no_grad():` block, so that it doesn't compute the gradients (which I'm assuming you won't need here). That should save some memory. (And make sure to call `resnet.eval()` beforehand too.)\n\nAlso not sure what you're doing once the embeddings have been calculated, but if you're holding on to these tensors it's not a surprise that your free GPU memory goes down over time.",
    "764693": "Thanks with no_grad, I can even get embedding of all the 300 in one go."
  },
  "source": "meta"
}