{
  "id": 211270,
  "title": "Tip to speed up inference in Tensorflow",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/211270",
  "author_name": "",
  "post_date": "2021-01-14T13:15:51.716233800Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I found this pretty neat trick while searching for ways to ensemble more models:</p>\n<p>Instead of model.predict(\"filepath\"), use model(\"filepath\", training=False).</p>\n<p>Happy kaggling!</p>",
  "messages": [
    {
      "id": "1152815",
      "postDate": "01/14/2021 13:15:51",
      "content": "<p>I found this pretty neat trick while searching for ways to ensemble more models:</p>\n<p>Instead of model.predict(\"filepath\"), use model(\"filepath\", training=False).</p>\n<p>Happy kaggling!</p>",
      "rawMarkdown": "I found this pretty neat trick while searching for ways to ensemble more models:\n\nInstead of model.predict(\"filepath\"), use model(\"filepath\", training=False).\n\nHappy kaggling!",
      "votes": null
    },
    {
      "id": "1152885",
      "postDate": "01/14/2021 14:36:56",
      "content": "<p>Does not work for me. It reduces my items predicted each second from 13 items to 7 items</p>",
      "rawMarkdown": "Does not work for me. It reduces my items predicted each second from 13 items to 7 items",
      "votes": null
    },
    {
      "id": "1152905",
      "postDate": "01/14/2021 14:57:08",
      "content": "<p>I'm not sure how that happened for you. You might want to try this! <a href=\"https://medium.com/@micwurm/using-tensorflow-lite-to-speed-up-predictions-a3954886eb98\" target=\"_blank\">https://medium.com/@micwurm/using-tensorflow-lite-to-speed-up-predictions-a3954886eb98</a></p>\n<p>Anyway, I ran a %%time on my code cell just to be sure that it was faster, and it was more than 10x faster for me. Maybe you could show a snippet of your code and I will try to debug.</p>",
      "rawMarkdown": "I'm not sure how that happened for you. You might want to try this! https://medium.com/@micwurm/using-tensorflow-lite-to-speed-up-predictions-a3954886eb98\n\nAnyway, I ran a %%time on my code cell just to be sure that it was faster, and it was more than 10x faster for me. Maybe you could show a snippet of your code and I will try to debug.",
      "votes": null
    },
    {
      "id": "1152934",
      "postDate": "01/14/2021 15:06:13",
      "content": "<p>import tensorflow as tf<br>\nimport numpy as np<br>\nfrom tqdm import tqdm<br>\nimport os<br>\nimport pandas as pd<br>\nimport keras<br>\nimport numpy as np</p>\n<p>model1 = tf.keras.models.load_model(r\"../input/models-gcs/88effnetb3moredata\")</p>\n<p>path = \"../input/cassava-leaf-disease-classification/train_images\"</p>\n<p>test_file_list = os.listdir(path)<br>\npredictions = []<br>\nmodel1_predict_list = []<br>\nfor filename in tqdm(test_file_list):<br>\n    img = tf.keras.preprocessing.image.load_img(path + \"/\" + filename, target_size=(512, 512))<br>\n    arr = tf.keras.preprocessing.image.img_to_array(img)<br>\n    arr = tf.image.random_flip_left_right(arr)<br>\n    arr = tf.expand_dims(arr / 255., 0)<br>\n    model1_predict = (np.argmax(model1(arr,training=False)))</p>\n<pre><code>pre = [model1_predict]\npredictions.append(int(max(set(pre), key=pre.count)))\n</code></pre>\n<p>df = pd.DataFrame(zip(test_file_list, predictions), columns=[\"image_id\", \"label\"])<br>\ndf.to_csv(\"./submission.csv\", index=False)<br>\nprint(df)</p>",
      "rawMarkdown": "import tensorflow as tf\nimport numpy as np\nfrom tqdm import tqdm\nimport os\nimport pandas as pd\nimport keras\nimport numpy as np\n\nmodel1 = tf.keras.models.load_model(r\"../input/models-gcs/88effnetb3moredata\")\n\n\npath = \"../input/cassava-leaf-disease-classification/train_images\"\n\ntest_file_list = os.listdir(path)\npredictions = []\nmodel1_predict_list = []\nfor filename in tqdm(test_file_list):\n\timg = tf.keras.preprocessing.image.load_img(path + \"/\" + filename, target_size=(512, 512))\n\tarr = tf.keras.preprocessing.image.img_to_array(img)\n\tarr = tf.image.random_flip_left_right(arr)\n\tarr = tf.expand_dims(arr / 255., 0)\n\tmodel1_predict = (np.argmax(model1(arr,training=False)))\n\n\tpre = [model1_predict]\n\tpredictions.append(int(max(set(pre), key=pre.count)))\n\ndf = pd.DataFrame(zip(test_file_list, predictions), columns=[\"image_id\", \"label\"])\ndf.to_csv(\"./submission.csv\", index=False)\nprint(df)",
      "votes": null
    },
    {
      "id": "1152963",
      "postDate": "01/14/2021 15:14:28",
      "content": "<p>Code looks fine. Did you restart the session before you ran your benchmark? If you ran model.predict() after you ran model() for a comparison, it won't be fair due some data being cached.</p>\n<p>So you should either run both multiple times for a comparison, or if you just run each once, make sure you restart the session before you run either.</p>",
      "rawMarkdown": "Code looks fine. Did you restart the session before you ran your benchmark? If you ran model.predict() after you ran model() for a comparison, it won't be fair due some data being cached.\n\nSo you should either run both multiple times for a comparison, or if you just run each once, make sure you restart the session before you run either.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1152885,
      "author_name": "mithilsalunkhe",
      "author_url": "",
      "post_date": "01/14/2021 14:36:56",
      "content": "<p>Does not work for me. It reduces my items predicted each second from 13 items to 7 items</p>",
      "votes": null,
      "replies": [
        {
          "id": 1152905,
          "author_name": "junyingsg",
          "author_url": "",
          "post_date": "01/14/2021 14:57:08",
          "content": "<p>I'm not sure how that happened for you. You might want to try this! <a href=\"https://medium.com/@micwurm/using-tensorflow-lite-to-speed-up-predictions-a3954886eb98\" target=\"_blank\">https://medium.com/@micwurm/using-tensorflow-lite-to-speed-up-predictions-a3954886eb98</a></p>\n<p>Anyway, I ran a %%time on my code cell just to be sure that it was faster, and it was more than 10x faster for me. Maybe you could show a snippet of your code and I will try to debug.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1152934,
          "author_name": "mithilsalunkhe",
          "author_url": "",
          "post_date": "01/14/2021 15:06:13",
          "content": "<p>import tensorflow as tf<br>\nimport numpy as np<br>\nfrom tqdm import tqdm<br>\nimport os<br>\nimport pandas as pd<br>\nimport keras<br>\nimport numpy as np</p>\n<p>model1 = tf.keras.models.load_model(r\"../input/models-gcs/88effnetb3moredata\")</p>\n<p>path = \"../input/cassava-leaf-disease-classification/train_images\"</p>\n<p>test_file_list = os.listdir(path)<br>\npredictions = []<br>\nmodel1_predict_list = []<br>\nfor filename in tqdm(test_file_list):<br>\n    img = tf.keras.preprocessing.image.load_img(path + \"/\" + filename, target_size=(512, 512))<br>\n    arr = tf.keras.preprocessing.image.img_to_array(img)<br>\n    arr = tf.image.random_flip_left_right(arr)<br>\n    arr = tf.expand_dims(arr / 255., 0)<br>\n    model1_predict = (np.argmax(model1(arr,training=False)))</p>\n<pre><code>pre = [model1_predict]\npredictions.append(int(max(set(pre), key=pre.count)))\n</code></pre>\n<p>df = pd.DataFrame(zip(test_file_list, predictions), columns=[\"image_id\", \"label\"])<br>\ndf.to_csv(\"./submission.csv\", index=False)<br>\nprint(df)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1152963,
          "author_name": "junyingsg",
          "author_url": "",
          "post_date": "01/14/2021 15:14:28",
          "content": "<p>Code looks fine. Did you restart the session before you ran your benchmark? If you ran model.predict() after you ran model() for a comparison, it won't be fair due some data being cached.</p>\n<p>So you should either run both multiple times for a comparison, or if you just run each once, make sure you restart the session before you run either.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1152815": "I found this pretty neat trick while searching for ways to ensemble more models:\n\nInstead of model.predict(\"filepath\"), use model(\"filepath\", training=False).\n\nHappy kaggling!",
    "1152885": "Does not work for me. It reduces my items predicted each second from 13 items to 7 items",
    "1152905": "I'm not sure how that happened for you. You might want to try this! https://medium.com/@micwurm/using-tensorflow-lite-to-speed-up-predictions-a3954886eb98\n\nAnyway, I ran a %%time on my code cell just to be sure that it was faster, and it was more than 10x faster for me. Maybe you could show a snippet of your code and I will try to debug.",
    "1152934": "import tensorflow as tf\nimport numpy as np\nfrom tqdm import tqdm\nimport os\nimport pandas as pd\nimport keras\nimport numpy as np\n\nmodel1 = tf.keras.models.load_model(r\"../input/models-gcs/88effnetb3moredata\")\n\n\npath = \"../input/cassava-leaf-disease-classification/train_images\"\n\ntest_file_list = os.listdir(path)\npredictions = []\nmodel1_predict_list = []\nfor filename in tqdm(test_file_list):\n\timg = tf.keras.preprocessing.image.load_img(path + \"/\" + filename, target_size=(512, 512))\n\tarr = tf.keras.preprocessing.image.img_to_array(img)\n\tarr = tf.image.random_flip_left_right(arr)\n\tarr = tf.expand_dims(arr / 255., 0)\n\tmodel1_predict = (np.argmax(model1(arr,training=False)))\n\n\tpre = [model1_predict]\n\tpredictions.append(int(max(set(pre), key=pre.count)))\n\ndf = pd.DataFrame(zip(test_file_list, predictions), columns=[\"image_id\", \"label\"])\ndf.to_csv(\"./submission.csv\", index=False)\nprint(df)",
    "1152963": "Code looks fine. Did you restart the session before you ran your benchmark? If you ran model.predict() after you ran model() for a comparison, it won't be fair due some data being cached.\n\nSo you should either run both multiple times for a comparison, or if you just run each once, make sure you restart the session before you run either."
  },
  "source": "meta"
}