{
  "id": 215867,
  "title": "memory allocation error is occurring, need help! ",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/215867",
  "author_name": "",
  "post_date": "2021-01-31T14:44:19.223371400Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I am using xception with keras. <br>\nmy model is training well with input_shape=(150,150,3) but when I am trying to train the model with larger input_shape=(300,300,3) for better result. I am getting following error.</p>\n<p>Your notebook tried to allocate more memory than is available. It has restarted.</p>\n<p>how to solve this problem, please help.</p>",
  "messages": [
    {
      "id": "1179410",
      "postDate": "01/31/2021 14:44:19",
      "content": "<p>I am using xception with keras. <br>\nmy model is training well with input_shape=(150,150,3) but when I am trying to train the model with larger input_shape=(300,300,3) for better result. I am getting following error.</p>\n<p>Your notebook tried to allocate more memory than is available. It has restarted.</p>\n<p>how to solve this problem, please help.</p>",
      "rawMarkdown": "I am using xception with keras. \nmy model is training well with input_shape=(150,150,3) but when I am trying to train the model with larger input_shape=(300,300,3) for better result. I am getting following error.\n\nYour notebook tried to allocate more memory than is available. It has restarted.\n \nhow to solve this problem, please help.",
      "votes": null
    },
    {
      "id": "1179430",
      "postDate": "01/31/2021 15:02:47",
      "content": "<p>Hello! </p>\n<p>I can suggest the following, top to bottom:</p>\n<ul>\n<li>if you're trying to fit the entire dataset to the memory at once, use <code>ImageDataGenerator</code> instead;</li>\n<li>try a smaller <code>batch_size</code> or a smaller model (e.g. <code>EfficientNet</code> B0 to B4 have fewer parameters than Xception);</li>\n<li>if you are running on GPU, switch to TPU as it has way more RAM. If it seems complicated, use this <strong><a href=\"https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease\" target=\"_blank\">community notebook</a></strong> for a quick start.</li>\n</ul>",
      "rawMarkdown": "Hello! \n\nI can suggest the following, top to bottom:\n* if you're trying to fit the entire dataset to the memory at once, use `ImageDataGenerator` instead;\n* try a smaller `batch_size` or a smaller model (e.g. `EfficientNet` B0 to B4 have fewer parameters than Xception);\n* if you are running on GPU, switch to TPU as it has way more RAM. If it seems complicated, use this **[community notebook](https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease)** for a quick start.",
      "votes": null
    },
    {
      "id": "1179796",
      "postDate": "01/31/2021 20:36:38",
      "content": "<p>Reduced the batch size when training </p>",
      "rawMarkdown": "Reduced the batch size when training",
      "votes": null
    },
    {
      "id": "1182231",
      "postDate": "02/02/2021 11:12:41",
      "content": "<p>I encountered the same problem with the EfficientNet. B3 model with ImageDataGenerator and 32 batch size works fine.</p>",
      "rawMarkdown": "I encountered the same problem with the EfficientNet. B3 model with ImageDataGenerator and 32 batch size works fine.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1179430,
      "author_name": "nickuzmenkov",
      "author_url": "",
      "post_date": "01/31/2021 15:02:47",
      "content": "<p>Hello! </p>\n<p>I can suggest the following, top to bottom:</p>\n<ul>\n<li>if you're trying to fit the entire dataset to the memory at once, use <code>ImageDataGenerator</code> instead;</li>\n<li>try a smaller <code>batch_size</code> or a smaller model (e.g. <code>EfficientNet</code> B0 to B4 have fewer parameters than Xception);</li>\n<li>if you are running on GPU, switch to TPU as it has way more RAM. If it seems complicated, use this <strong><a href=\"https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease\" target=\"_blank\">community notebook</a></strong> for a quick start.</li>\n</ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1179796,
      "author_name": "mnavaidd",
      "author_url": "",
      "post_date": "01/31/2021 20:36:38",
      "content": "<p>Reduced the batch size when training </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1182231,
      "author_name": "ravinmechu",
      "author_url": "",
      "post_date": "02/02/2021 11:12:41",
      "content": "<p>I encountered the same problem with the EfficientNet. B3 model with ImageDataGenerator and 32 batch size works fine.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1179410": "I am using xception with keras. \nmy model is training well with input_shape=(150,150,3) but when I am trying to train the model with larger input_shape=(300,300,3) for better result. I am getting following error.\n\nYour notebook tried to allocate more memory than is available. It has restarted.\n \nhow to solve this problem, please help.",
    "1179430": "Hello! \n\nI can suggest the following, top to bottom:\n* if you're trying to fit the entire dataset to the memory at once, use `ImageDataGenerator` instead;\n* try a smaller `batch_size` or a smaller model (e.g. `EfficientNet` B0 to B4 have fewer parameters than Xception);\n* if you are running on GPU, switch to TPU as it has way more RAM. If it seems complicated, use this **[community notebook](https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease)** for a quick start.",
    "1179796": "Reduced the batch size when training",
    "1182231": "I encountered the same problem with the EfficientNet. B3 model with ImageDataGenerator and 32 batch size works fine."
  },
  "source": "meta"
}