{
  "id": 178681,
  "title": "Tensorflow Predictions Super slow",
  "url": "/competitions/birdsong-recognition/discussion/178681",
  "author_name": "",
  "post_date": "2020-08-31T02:05:34.416096800Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Does anyone know how to speed up tensorflow predictions?</p>\n<p>My submissions keep timing out, so I tried messing around with the model.predict hyperparameters</p>\n<p><code>predict(\n    x, batch_size=None, verbose=0, steps=None, callbacks=None, max_queue_size=10,\n    workers=1, use_multiprocessing=False\n)\n</code></p>\n<p>I also tried</p>\n<p><code>model(x, training=False)</code></p>\n<p>and different sizes of x (1 sample, 16 samples, 32 samples 128 samples 256 samples) hoping that parallelization might give me a boost.</p>\n<p>None of it made any difference at all. Is plain old model.predict pretty much as fast as it gets?</p>",
  "messages": [
    {
      "id": "992159",
      "postDate": "08/31/2020 02:05:34",
      "content": "<p>Does anyone know how to speed up tensorflow predictions?</p>\n<p>My submissions keep timing out, so I tried messing around with the model.predict hyperparameters</p>\n<p><code>predict(\n    x, batch_size=None, verbose=0, steps=None, callbacks=None, max_queue_size=10,\n    workers=1, use_multiprocessing=False\n)\n</code></p>\n<p>I also tried</p>\n<p><code>model(x, training=False)</code></p>\n<p>and different sizes of x (1 sample, 16 samples, 32 samples 128 samples 256 samples) hoping that parallelization might give me a boost.</p>\n<p>None of it made any difference at all. Is plain old model.predict pretty much as fast as it gets?</p>",
      "rawMarkdown": "Does anyone know how to speed up tensorflow predictions?\n\nMy submissions keep timing out, so I tried messing around with the model.predict hyperparameters\n\n\n`predict(\n    x, batch_size=None, verbose=0, steps=None, callbacks=None, max_queue_size=10,\n    workers=1, use_multiprocessing=False\n)\n`\n\nI also tried\n\n`model(x, training=False)`\n\nand different sizes of x (1 sample, 16 samples, 32 samples 128 samples 256 samples) hoping that parallelization might give me a boost.\n\nNone of it made any difference at all. Is plain old model.predict pretty much as fast as it gets?",
      "votes": null
    },
    {
      "id": "992167",
      "postDate": "08/31/2020 02:19:52",
      "content": "<p>Most common reason for timing out submissions is attempting to load whole .mp3 files from scratch every time you need a new 5 second slice of it.</p>\n<p>While being the simplest way to code submission, it is not fast enough to complete it before timeout.</p>",
      "rawMarkdown": "Most common reason for timing out submissions is attempting to load whole .mp3 files from scratch every time you need a new 5 second slice of it.\n\nWhile being the simplest way to code submission, it is not fast enough to complete it before timeout.",
      "votes": null
    },
    {
      "id": "993696",
      "postDate": "09/01/2020 05:34:59",
      "content": "<p>In your pre-process do you use np.asarray?<br>\nTo convert from tensor to numpy array is very slow.</p>",
      "rawMarkdown": "In your pre-process do you use np.asarray?\nTo convert from tensor to numpy array is very slow.",
      "votes": null
    },
    {
      "id": "993704",
      "postDate": "09/01/2020 05:38:44",
      "content": "<p>Yes, i do use numpy arrays to load and preprocess the data. The preprocessing is definitely very slow, but the slowest part is the prediction itself. I'm wondering if anyone knows how to speed up the prediction specifically. </p>",
      "rawMarkdown": "Yes, i do use numpy arrays to load and preprocess the data. The preprocessing is definitely very slow, but the slowest part is the prediction itself. I'm wondering if anyone knows how to speed up the prediction specifically.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 992167,
      "author_name": "fffrrt",
      "author_url": "",
      "post_date": "08/31/2020 02:19:52",
      "content": "<p>Most common reason for timing out submissions is attempting to load whole .mp3 files from scratch every time you need a new 5 second slice of it.</p>\n<p>While being the simplest way to code submission, it is not fast enough to complete it before timeout.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 993696,
      "author_name": "enukuro",
      "author_url": "",
      "post_date": "09/01/2020 05:34:59",
      "content": "<p>In your pre-process do you use np.asarray?<br>\nTo convert from tensor to numpy array is very slow.</p>",
      "votes": null,
      "replies": [
        {
          "id": 993704,
          "author_name": "lewington",
          "author_url": "",
          "post_date": "09/01/2020 05:38:44",
          "content": "<p>Yes, i do use numpy arrays to load and preprocess the data. The preprocessing is definitely very slow, but the slowest part is the prediction itself. I'm wondering if anyone knows how to speed up the prediction specifically. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "992159": "Does anyone know how to speed up tensorflow predictions?\n\nMy submissions keep timing out, so I tried messing around with the model.predict hyperparameters\n\n\n`predict(\n    x, batch_size=None, verbose=0, steps=None, callbacks=None, max_queue_size=10,\n    workers=1, use_multiprocessing=False\n)\n`\n\nI also tried\n\n`model(x, training=False)`\n\nand different sizes of x (1 sample, 16 samples, 32 samples 128 samples 256 samples) hoping that parallelization might give me a boost.\n\nNone of it made any difference at all. Is plain old model.predict pretty much as fast as it gets?",
    "992167": "Most common reason for timing out submissions is attempting to load whole .mp3 files from scratch every time you need a new 5 second slice of it.\n\nWhile being the simplest way to code submission, it is not fast enough to complete it before timeout.",
    "993696": "In your pre-process do you use np.asarray?\nTo convert from tensor to numpy array is very slow.",
    "993704": "Yes, i do use numpy arrays to load and preprocess the data. The preprocessing is definitely very slow, but the slowest part is the prediction itself. I'm wondering if anyone knows how to speed up the prediction specifically."
  },
  "source": "meta"
}