{
  "id": 403228,
  "title": "Optimizing Inference Speed",
  "url": "/competitions/birdclef-2023/discussion/403228",
  "author_name": "Peleg Shilo",
  "post_date": "2023-04-21T22:45:01.863000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>After training my model, I got to the part where I do inference. However, it seems like I am significantly above the maximum inference time of 2 hours. The 1 visible test file takes me slightly more than 1 minute to process. I checked my code to see if I have bottlenecks due to bad implementation and that doesn't seem to be the case. What can I do to make my model run faster? I'm using a pre-trained, transformer-based Pytorch model.</p>",
  "messages": [
    {
      "id": 2229996,
      "postDate": "2023-04-21T22:45:01.863Z",
      "content": "<p>After training my model, I got to the part where I do inference. However, it seems like I am significantly above the maximum inference time of 2 hours. The 1 visible test file takes me slightly more than 1 minute to process. I checked my code to see if I have bottlenecks due to bad implementation and that doesn't seem to be the case. What can I do to make my model run faster? I'm using a pre-trained, transformer-based Pytorch model.</p>",
      "rawMarkdown": "After training my model, I got to the part where I do inference. However, it seems like I am significantly above the maximum inference time of 2 hours. The 1 visible test file takes me slightly more than 1 minute to process. I checked my code to see if I have bottlenecks due to bad implementation and that doesn't seem to be the case. What can I do to make my model run faster? I'm using a pre-trained, transformer-based Pytorch model."
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2229996": "After training my model, I got to the part where I do inference. However, it seems like I am significantly above the maximum inference time of 2 hours. The 1 visible test file takes me slightly more than 1 minute to process. I checked my code to see if I have bottlenecks due to bad implementation and that doesn't seem to be the case. What can I do to make my model run faster? I'm using a pre-trained, transformer-based Pytorch model."
  }
}