{
  "id": 500518,
  "title": "Ideas to speed up inference during notebook submission?",
  "url": "/competitions/birdclef-2024/discussion/500518",
  "author_name": "",
  "post_date": "2024-05-06T01:12:11.088078800Z",
  "votes": 2,
  "comment_count": 12,
  "views": 0,
  "content": "<p>My efficientnet_b0 model timed out on submission. I am opening a thread here to discuss on ways to improve the inference speed.</p>",
  "messages": [
    {
      "id": "2795786",
      "postDate": "05/06/2024 01:12:11",
      "content": "<p>My efficientnet_b0 model timed out on submission. I am opening a thread here to discuss on ways to improve the inference speed.</p>",
      "rawMarkdown": "My efficientnet_b0 model timed out on submission. I am opening a thread here to discuss on ways to improve the inference speed.",
      "votes": null
    },
    {
      "id": "2795988",
      "postDate": "05/06/2024 04:40:44",
      "content": "<p>Using OpenVINO might enhance the inference speed.<br>\n<a href=\"https://www.kaggle.com/code/akinosora/efficientnetb0-inference-speed-analysis\" target=\"_blank\">https://www.kaggle.com/code/akinosora/efficientnetb0-inference-speed-analysis</a></p>",
      "rawMarkdown": "Using OpenVINO might enhance the inference speed.\nhttps://www.kaggle.com/code/akinosora/efficientnetb0-inference-speed-analysis",
      "votes": null
    },
    {
      "id": "2798557",
      "postDate": "05/07/2024 09:21:27",
      "content": "<p>In birdclef 2023, people can ensemble 4 models. How about this competition?</p>",
      "rawMarkdown": "In birdclef 2023, people can ensemble 4 models. How about this competition?",
      "votes": null
    },
    {
      "id": "2798921",
      "postDate": "05/07/2024 14:02:20",
      "content": "<p>By using torch.jit.optimize_for_inference, I can save about 20% of time on my end (100s &gt; 80s).</p>",
      "rawMarkdown": "By using torch.jit.optimize_for_inference, I can save about 20% of time on my end (100s > 80s).",
      "votes": null
    },
    {
      "id": "2799197",
      "postDate": "05/07/2024 16:13:01",
      "content": "<p>Thanks, let me try this quickly. On my first submission, there was no timeout. However, on my second submission with the same inference pipeline and new model (same architecture), it timed out.</p>",
      "rawMarkdown": "Thanks, let me try this quickly. On my first submission, there was no timeout. However, on my second submission with the same inference pipeline and new model (same architecture), it timed out.",
      "votes": null
    },
    {
      "id": "2799200",
      "postDate": "05/07/2024 16:15:36",
      "content": "<p>There shouldn't be any restriction on the number of models we can use. It's just that the environment for the submission is restricted to CPU and 2 hours of runtime.</p>",
      "rawMarkdown": "There shouldn't be any restriction on the number of models we can use. It's just that the environment for the submission is restricted to CPU and 2 hours of runtime.",
      "votes": null
    },
    {
      "id": "2799208",
      "postDate": "05/07/2024 16:17:34",
      "content": "<p>Thank you. Have you tried it? From your experience, can you tell me how long the submission and scoring take?</p>",
      "rawMarkdown": "Thank you. Have you tried it? From your experience, can you tell me how long the submission and scoring take?",
      "votes": null
    },
    {
      "id": "2800355",
      "postDate": "05/08/2024 06:55:39",
      "content": "<p>It seems that openvino is the best choice </p>",
      "rawMarkdown": "It seems that openvino is the best choice",
      "votes": null
    },
    {
      "id": "2800545",
      "postDate": "05/08/2024 08:29:45",
      "content": "<p>Hi yuanzhe, what other techniques did you try?</p>",
      "rawMarkdown": "Hi yuanzhe, what other techniques did you try?",
      "votes": null
    },
    {
      "id": "2800633",
      "postDate": "05/08/2024 09:27:09",
      "content": "<p>I'm using code from public notebooks at this moment. Did not try openvino yet, but it seems to be better choice.</p>",
      "rawMarkdown": "I'm using code from public notebooks at this moment. Did not try openvino yet, but it seems to be better choice.",
      "votes": null
    },
    {
      "id": "2800642",
      "postDate": "05/08/2024 09:32:57",
      "content": "<p>Okay, what is your average notebook submission time?</p>",
      "rawMarkdown": "Okay, what is your average notebook submission time?",
      "votes": null
    },
    {
      "id": "2801055",
      "postDate": "05/08/2024 13:07:35",
      "content": "<p>Submissions are completed in less than 30 minutes using <code>concurrent.futures.ThreadPoolExecutor(4)</code> and EfficientNetB0 with OpenVINO.</p>",
      "rawMarkdown": "Submissions are completed in less than 30 minutes using `concurrent.futures.ThreadPoolExecutor(4)` and EfficientNetB0 with OpenVINO.",
      "votes": null
    },
    {
      "id": "2802006",
      "postDate": "05/08/2024 20:25:26",
      "content": "<p>how do i do this though… I've just been using torchscript.</p>",
      "rawMarkdown": "how do i do this though... I've just been using torchscript.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2795988,
      "author_name": "akinosora",
      "author_url": "",
      "post_date": "05/06/2024 04:40:44",
      "content": "<p>Using OpenVINO might enhance the inference speed.<br>\n<a href=\"https://www.kaggle.com/code/akinosora/efficientnetb0-inference-speed-analysis\" target=\"_blank\">https://www.kaggle.com/code/akinosora/efficientnetb0-inference-speed-analysis</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 2799208,
          "author_name": "srinath9s",
          "author_url": "",
          "post_date": "05/07/2024 16:17:34",
          "content": "<p>Thank you. Have you tried it? From your experience, can you tell me how long the submission and scoring take?</p>",
          "votes": null,
          "replies": [
            {
              "id": 2801055,
              "author_name": "akinosora",
              "author_url": "",
              "post_date": "05/08/2024 13:07:35",
              "content": "<p>Submissions are completed in less than 30 minutes using <code>concurrent.futures.ThreadPoolExecutor(4)</code> and EfficientNetB0 with OpenVINO.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2802006,
                  "author_name": "willrice",
                  "author_url": "",
                  "post_date": "05/08/2024 20:25:26",
                  "content": "<p>how do i do this though… I've just been using torchscript.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        },
        {
          "id": 2800355,
          "author_name": "yuanzhezhou",
          "author_url": "",
          "post_date": "05/08/2024 06:55:39",
          "content": "<p>It seems that openvino is the best choice </p>",
          "votes": null,
          "replies": [
            {
              "id": 2800545,
              "author_name": "srinath9s",
              "author_url": "",
              "post_date": "05/08/2024 08:29:45",
              "content": "<p>Hi yuanzhe, what other techniques did you try?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2800633,
                  "author_name": "yuanzhezhou",
                  "author_url": "",
                  "post_date": "05/08/2024 09:27:09",
                  "content": "<p>I'm using code from public notebooks at this moment. Did not try openvino yet, but it seems to be better choice.</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2800642,
                      "author_name": "srinath9s",
                      "author_url": "",
                      "post_date": "05/08/2024 09:32:57",
                      "content": "<p>Okay, what is your average notebook submission time?</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2798557,
      "author_name": "yuanzhezhou",
      "author_url": "",
      "post_date": "05/07/2024 09:21:27",
      "content": "<p>In birdclef 2023, people can ensemble 4 models. How about this competition?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2799200,
          "author_name": "srinath9s",
          "author_url": "",
          "post_date": "05/07/2024 16:15:36",
          "content": "<p>There shouldn't be any restriction on the number of models we can use. It's just that the environment for the submission is restricted to CPU and 2 hours of runtime.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2798921,
      "author_name": "shawntung",
      "author_url": "",
      "post_date": "05/07/2024 14:02:20",
      "content": "<p>By using torch.jit.optimize_for_inference, I can save about 20% of time on my end (100s &gt; 80s).</p>",
      "votes": null,
      "replies": [
        {
          "id": 2799197,
          "author_name": "srinath9s",
          "author_url": "",
          "post_date": "05/07/2024 16:13:01",
          "content": "<p>Thanks, let me try this quickly. On my first submission, there was no timeout. However, on my second submission with the same inference pipeline and new model (same architecture), it timed out.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2795786": "My efficientnet_b0 model timed out on submission. I am opening a thread here to discuss on ways to improve the inference speed.",
    "2795988": "Using OpenVINO might enhance the inference speed.\nhttps://www.kaggle.com/code/akinosora/efficientnetb0-inference-speed-analysis",
    "2798557": "In birdclef 2023, people can ensemble 4 models. How about this competition?",
    "2798921": "By using torch.jit.optimize_for_inference, I can save about 20% of time on my end (100s > 80s).",
    "2799197": "Thanks, let me try this quickly. On my first submission, there was no timeout. However, on my second submission with the same inference pipeline and new model (same architecture), it timed out.",
    "2799200": "There shouldn't be any restriction on the number of models we can use. It's just that the environment for the submission is restricted to CPU and 2 hours of runtime.",
    "2799208": "Thank you. Have you tried it? From your experience, can you tell me how long the submission and scoring take?",
    "2800355": "It seems that openvino is the best choice",
    "2800545": "Hi yuanzhe, what other techniques did you try?",
    "2800633": "I'm using code from public notebooks at this moment. Did not try openvino yet, but it seems to be better choice.",
    "2800642": "Okay, what is your average notebook submission time?",
    "2801055": "Submissions are completed in less than 30 minutes using `concurrent.futures.ThreadPoolExecutor(4)` and EfficientNetB0 with OpenVINO.",
    "2802006": "how do i do this though... I've just been using torchscript."
  },
  "source": "meta"
}