{
  "id": 315904,
  "title": "My notebook is not using the GPU. ",
  "url": "/competitions/ultra-mnist/discussion/315904",
  "author_name": "",
  "post_date": "2022-03-30T09:01:59.549388200Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Even i set os.environ['CUDA_VISIBLE_DEVICES'] = '0'. I am using tf.keras for training the model.   The GPU is visible.  But TensorFlow is not using it. Could you tell me, how to solve it? </p>\n<p><code>tf.test.is_built_with_cuda()  \ntf.test.is_gpu_available(cuda_only = True) \n2022-03-30 08:58:30.099050: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\nTo enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n2022-03-30 08:58:30.153977: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.248528: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.249217: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.174191: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175064: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175768: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.176343: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /device:GPU:0 with 15403 MB memory:  -&gt; device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\nTrue</code></p>",
  "messages": [
    {
      "id": "1739685",
      "postDate": "03/30/2022 09:01:59",
      "content": "<p>Even i set os.environ['CUDA_VISIBLE_DEVICES'] = '0'. I am using tf.keras for training the model.   The GPU is visible.  But TensorFlow is not using it. Could you tell me, how to solve it? </p>\n<p><code>tf.test.is_built_with_cuda()  \ntf.test.is_gpu_available(cuda_only = True) \n2022-03-30 08:58:30.099050: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\nTo enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n2022-03-30 08:58:30.153977: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.248528: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.249217: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.174191: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175064: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175768: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.176343: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /device:GPU:0 with 15403 MB memory:  -&gt; device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\nTrue</code></p>",
      "rawMarkdown": "Even i set os.environ['CUDA_VISIBLE_DEVICES'] = '0'. I am using tf.keras for training the model.   The GPU is visible.  But TensorFlow is not using it. Could you tell me, how to solve it? \n\n`tf.test.is_built_with_cuda()  \ntf.test.is_gpu_available(cuda_only = True) \n2022-03-30 08:58:30.099050: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\nTo enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n2022-03-30 08:58:30.153977: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.248528: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.249217: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.174191: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175064: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175768: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.176343: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /device:GPU:0 with 15403 MB memory:  -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\nTrue`",
      "votes": null
    },
    {
      "id": "1740412",
      "postDate": "03/30/2022 21:55:18",
      "content": "<p>I had similar issue, where i was doing lot of preprocessing with Image data and GPU did not kickoff. I reduced the batch size and it worked fine. Try reducing the Batch size </p>",
      "rawMarkdown": "I had similar issue, where i was doing lot of preprocessing with Image data and GPU did not kickoff. I reduced the batch size and it worked fine. Try reducing the Batch size",
      "votes": null
    },
    {
      "id": "1740448",
      "postDate": "03/31/2022 00:38:26",
      "content": "<p><a href=\"https://www.kaggle.com/abhisai51\" target=\"_blank\">@abhisai51</a> thank you. I will try it</p>",
      "rawMarkdown": "abhisai51 thank you. I will try it",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1740412,
      "author_name": "abhisai51",
      "author_url": "",
      "post_date": "03/30/2022 21:55:18",
      "content": "<p>I had similar issue, where i was doing lot of preprocessing with Image data and GPU did not kickoff. I reduced the batch size and it worked fine. Try reducing the Batch size </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1740448,
      "author_name": "sabarinathan",
      "author_url": "",
      "post_date": "03/31/2022 00:38:26",
      "content": "<p><a href=\"https://www.kaggle.com/abhisai51\" target=\"_blank\">@abhisai51</a> thank you. I will try it</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1739685": "Even i set os.environ['CUDA_VISIBLE_DEVICES'] = '0'. I am using tf.keras for training the model.   The GPU is visible.  But TensorFlow is not using it. Could you tell me, how to solve it? \n\n`tf.test.is_built_with_cuda()  \ntf.test.is_gpu_available(cuda_only = True) \n2022-03-30 08:58:30.099050: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\nTo enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n2022-03-30 08:58:30.153977: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.248528: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:30.249217: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.174191: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175064: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.175768: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n2022-03-30 08:58:32.176343: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /device:GPU:0 with 15403 MB memory:  -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0\nTrue`",
    "1740412": "I had similar issue, where i was doing lot of preprocessing with Image data and GPU did not kickoff. I reduced the batch size and it worked fine. Try reducing the Batch size",
    "1740448": "abhisai51 thank you. I will try it"
  },
  "source": "meta"
}