{
  "id": 172862,
  "title": "Tensorflow_Training and Prediction speed is very slow",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/172862",
  "author_name": "PraveenThenraj",
  "post_date": "2020-08-06T19:09:03.494000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>As part of this competition, I am using Tensorflow, train_generator and test_generator to generate batches of train and test images.Below are the config details of the generator. A single epoch of my model training takes 10 minutes and as a result am not able to run more epochs.As well a single run of prediction for test images (10982 images) takes 40 minutes which I think is quite very high. </p>\n\n<p>Because of the these shortcomings I am not able to create multiple models for same train data and consider an average prediction of these models output for test data. Am a newbie to Kaggle and Tensorflow, hence would like to reach out for help to Kagglers out there.</p>\n\n<pre><code>img_width=224,img_height=224\n\ntrain_generator_name = train_aug_name.flow_from_dataframe(\n    train,\n    x_col='images',\n    y_col='target',\n    target_size=(img_width, img_height),\n    batch_size=32,\n    shuffle=True,\n    class_mode='raw')\n\nval_generator_name = val_aug_name.flow_from_dataframe(\n    validation,\n    x_col='images',\n    y_col='target',\n    target_size=(img_width,img_height),\n    batch_size=16,\n    shuffle=False,\n    class_mode='raw')  \n\ntest_generator_name=test_aug_name.flow_from_dataframe(\n    test,\n    x_col='images',\n    y_col=[],\n    target_size=(224,224),\n    shuffle=False,\n    batch_size=16,\n    class_mode='raw'\n )\n</code></pre>",
  "messages": [
    {
      "id": 960899,
      "postDate": "2020-08-06T19:09:03.493Z",
      "content": "<p>As part of this competition, I am using Tensorflow, train_generator and test_generator to generate batches of train and test images.Below are the config details of the generator. A single epoch of my model training takes 10 minutes and as a result am not able to run more epochs.As well a single run of prediction for test images (10982 images) takes 40 minutes which I think is quite very high. </p>\n\n<p>Because of the these shortcomings I am not able to create multiple models for same train data and consider an average prediction of these models output for test data. Am a newbie to Kaggle and Tensorflow, hence would like to reach out for help to Kagglers out there.</p>\n\n<pre><code>img_width=224,img_height=224\n\ntrain_generator_name = train_aug_name.flow_from_dataframe(\n    train,\n    x_col='images',\n    y_col='target',\n    target_size=(img_width, img_height),\n    batch_size=32,\n    shuffle=True,\n    class_mode='raw')\n\nval_generator_name = val_aug_name.flow_from_dataframe(\n    validation,\n    x_col='images',\n    y_col='target',\n    target_size=(img_width,img_height),\n    batch_size=16,\n    shuffle=False,\n    class_mode='raw')  \n\ntest_generator_name=test_aug_name.flow_from_dataframe(\n    test,\n    x_col='images',\n    y_col=[],\n    target_size=(224,224),\n    shuffle=False,\n    batch_size=16,\n    class_mode='raw'\n )\n</code></pre>",
      "rawMarkdown": "As part of this competition, I am using Tensorflow, train_generator and test_generator to generate batches of train and test images.Below are the config details of the generator. A single epoch of my model training takes 10 minutes and as a result am not able to run more epochs.As well a single run of prediction for test images (10982 images) takes 40 minutes which I think is quite very high. \n\nBecause of the these shortcomings I am not able to create multiple models for same train data and consider an average prediction of these models output for test data. Am a newbie to Kaggle and Tensorflow, hence would like to reach out for help to Kagglers out there.\n\n   \n   \n    img_width=224,img_height=224\n\n    train_generator_name = train_aug_name.flow_from_dataframe(\n        train,\n        x_col='images',\n        y_col='target',\n        target_size=(img_width, img_height),\n        batch_size=32,\n        shuffle=True,\n        class_mode='raw')\n\n    val_generator_name = val_aug_name.flow_from_dataframe(\n        validation,\n        x_col='images',\n        y_col='target',\n        target_size=(img_width,img_height),\n        batch_size=16,\n        shuffle=False,\n        class_mode='raw')  \n\n    test_generator_name=test_aug_name.flow_from_dataframe(\n        test,\n        x_col='images',\n        y_col=[],\n        target_size=(224,224),\n        shuffle=False,\n        batch_size=16,\n        class_mode='raw'\n     )\n   \n\n  \n"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "960899": "As part of this competition, I am using Tensorflow, train_generator and test_generator to generate batches of train and test images.Below are the config details of the generator. A single epoch of my model training takes 10 minutes and as a result am not able to run more epochs.As well a single run of prediction for test images (10982 images) takes 40 minutes which I think is quite very high. \n\nBecause of the these shortcomings I am not able to create multiple models for same train data and consider an average prediction of these models output for test data. Am a newbie to Kaggle and Tensorflow, hence would like to reach out for help to Kagglers out there.\n\n   \n   \n    img_width=224,img_height=224\n\n    train_generator_name = train_aug_name.flow_from_dataframe(\n        train,\n        x_col='images',\n        y_col='target',\n        target_size=(img_width, img_height),\n        batch_size=32,\n        shuffle=True,\n        class_mode='raw')\n\n    val_generator_name = val_aug_name.flow_from_dataframe(\n        validation,\n        x_col='images',\n        y_col='target',\n        target_size=(img_width,img_height),\n        batch_size=16,\n        shuffle=False,\n        class_mode='raw')  \n\n    test_generator_name=test_aug_name.flow_from_dataframe(\n        test,\n        x_col='images',\n        y_col=[],\n        target_size=(224,224),\n        shuffle=False,\n        batch_size=16,\n        class_mode='raw'\n     )\n   \n\n  \n"
  }
}