{
  "id": 106622,
  "title": "OpenCV Memory Leak?",
  "url": "/competitions/aptos2019-blindness-detection/discussion/106622",
  "author_name": "",
  "post_date": "2019-08-30T09:42:49.961240Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi, is it possible that there is a problem with opencv and releasing memory? I have the following minimal code example:</p>\n\n<pre><code>import cv2\nimport os\nimport gc\n\nimages_path = \"../input/aptos2019-blindness-detection/train_images/\"\nimage_names = os.listdir(images_path)\nfor image_name in image_names:\n    img = cv2.imread(images_path + image_name)\n    del img\n    gc.collect()\n</code></pre>\n\n<p>When I run this in a notebook, the memory usage will slowly rise up to 8,3 GB. Is there something that I do not consider? What is the problem here and how could it be solved?</p>",
  "messages": [
    {
      "id": "613196",
      "postDate": "08/30/2019 09:42:49",
      "content": "<p>Hi, is it possible that there is a problem with opencv and releasing memory? I have the following minimal code example:</p>\n\n<pre><code>import cv2\nimport os\nimport gc\n\nimages_path = \"../input/aptos2019-blindness-detection/train_images/\"\nimage_names = os.listdir(images_path)\nfor image_name in image_names:\n    img = cv2.imread(images_path + image_name)\n    del img\n    gc.collect()\n</code></pre>\n\n<p>When I run this in a notebook, the memory usage will slowly rise up to 8,3 GB. Is there something that I do not consider? What is the problem here and how could it be solved?</p>",
      "rawMarkdown": "Hi, is it possible that there is a problem with opencv and releasing memory? I have the following minimal code example:\n\n    import cv2\n    import os\n    import gc\n\n    images_path = \"../input/aptos2019-blindness-detection/train_images/\"\n    image_names = os.listdir(images_path)\n    for image_name in image_names:\n        img = cv2.imread(images_path + image_name)\n        del img\n        gc.collect()\n\nWhen I run this in a notebook, the memory usage will slowly rise up to 8,3 GB. Is there something that I do not consider? What is the problem here and how could it be solved?",
      "votes": null
    },
    {
      "id": "613601",
      "postDate": "08/30/2019 16:45:39",
      "content": "<p>Hi, I found the same issue a couple of weeks ago and reported it in the forums: <a href=\"https://www.kaggle.com/product-feedback/104464\">https://www.kaggle.com/product-feedback/104464</a> </p>\n\n<p>Basically, it happens no matter what library you use to read images (I tried 3 different ones). However, I have still not had any notebook crash because of it, so maybe it's just some kind of interface bug.</p>",
      "rawMarkdown": "Hi, I found the same issue a couple of weeks ago and reported it in the forums: https://www.kaggle.com/product-feedback/104464 \n\nBasically, it happens no matter what library you use to read images (I tried 3 different ones). However, I have still not had any notebook crash because of it, so maybe it's just some kind of interface bug.",
      "votes": null
    },
    {
      "id": "615633",
      "postDate": "09/02/2019 07:40:09",
      "content": "<p>Ok, thank you! I have tested it now with skimage and with the keras ImageDataGenerator and the same problem occurs. When I create a big numpy array instead of reading an image and when I delete it afterwards, the memory will be released. Very strange.</p>\n\n<p>I had kernels crashed because of this but maybe the problem is somewhere else in my code.</p>",
      "rawMarkdown": "Ok, thank you! I have tested it now with skimage and with the keras ImageDataGenerator and the same problem occurs. When I create a big numpy array instead of reading an image and when I delete it afterwards, the memory will be released. Very strange.\n\nI had kernels crashed because of this but maybe the problem is somewhere else in my code.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 613601,
      "author_name": "larlia",
      "author_url": "",
      "post_date": "08/30/2019 16:45:39",
      "content": "<p>Hi, I found the same issue a couple of weeks ago and reported it in the forums: <a href=\"https://www.kaggle.com/product-feedback/104464\">https://www.kaggle.com/product-feedback/104464</a> </p>\n\n<p>Basically, it happens no matter what library you use to read images (I tried 3 different ones). However, I have still not had any notebook crash because of it, so maybe it's just some kind of interface bug.</p>",
      "votes": null,
      "replies": [
        {
          "id": 615633,
          "author_name": "flzieg",
          "author_url": "",
          "post_date": "09/02/2019 07:40:09",
          "content": "<p>Ok, thank you! I have tested it now with skimage and with the keras ImageDataGenerator and the same problem occurs. When I create a big numpy array instead of reading an image and when I delete it afterwards, the memory will be released. Very strange.</p>\n\n<p>I had kernels crashed because of this but maybe the problem is somewhere else in my code.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "613196": "Hi, is it possible that there is a problem with opencv and releasing memory? I have the following minimal code example:\n\n    import cv2\n    import os\n    import gc\n\n    images_path = \"../input/aptos2019-blindness-detection/train_images/\"\n    image_names = os.listdir(images_path)\n    for image_name in image_names:\n        img = cv2.imread(images_path + image_name)\n        del img\n        gc.collect()\n\nWhen I run this in a notebook, the memory usage will slowly rise up to 8,3 GB. Is there something that I do not consider? What is the problem here and how could it be solved?",
    "613601": "Hi, I found the same issue a couple of weeks ago and reported it in the forums: https://www.kaggle.com/product-feedback/104464 \n\nBasically, it happens no matter what library you use to read images (I tried 3 different ones). However, I have still not had any notebook crash because of it, so maybe it's just some kind of interface bug.",
    "615633": "Ok, thank you! I have tested it now with skimage and with the keras ImageDataGenerator and the same problem occurs. When I create a big numpy array instead of reading an image and when I delete it afterwards, the memory will be released. Very strange.\n\nI had kernels crashed because of this but maybe the problem is somewhere else in my code."
  },
  "source": "meta"
}