{
  "id": 224410,
  "title": "Can I speed up inference mode?",
  "url": "/competitions/ranzcr-clip-catheter-line-classification/discussion/224410",
  "author_name": "",
  "post_date": "2021-03-08T11:05:09.928201300Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I joined this competition week ago, so I didn't still understand this competition well.</p>\n<p>I tried some submission, but I saw submission takes long time.</p>\n<p>Do you have idea speeding up inference time?</p>\n<p>Below is my inference code. model is one.<br>\nThis code takes more than one hour with 5 model.</p>\n<pre><code>def infer(model, transforms=None):\n\n    dataset_test = MyDataset(df=test_df, transform=transforms)\n    dataloader_test = DataLoader(dataset_test, batch_size=TEST_BATCH, num_workers=os.cpu_count(), shuffle=False, pin_memory=True)\n\n    image_ids = []\n    labels = []\n    preds = []\n\n    with torch.no_grad():\n        for step, batch in tqdm(enumerate(dataloader_test), total=len(dataloader_test)):\n\n            Xs = batch[0].to(device) # image\n            _image_ids = batch[1] # target\n\n            outputs = model(Xs)\n            outputs = outputs.sigmoid().to('cpu').numpy()\n\n            outputs2 = model(Xs.flip(-1))\n            outputs2 = outputs2.sigmoid().to('cpu').numpy()\n\n            outputs = (outputs + outputs) / 2\n\n            image_ids.extend(_image_ids)\n            preds.append(outputs)\n\n    preds = np.concatenate(preds)\n\n    gc.collect()\n\n    if RESOURCE == \"GPU\":\n        torch.cuda.empty_cache()\n\n    return image_ids, preds\n</code></pre>\n<p>Thaks </p>",
  "messages": [
    {
      "id": "1230685",
      "postDate": "03/08/2021 11:05:09",
      "content": "<p>I joined this competition week ago, so I didn't still understand this competition well.</p>\n<p>I tried some submission, but I saw submission takes long time.</p>\n<p>Do you have idea speeding up inference time?</p>\n<p>Below is my inference code. model is one.<br>\nThis code takes more than one hour with 5 model.</p>\n<pre><code>def infer(model, transforms=None):\n\n    dataset_test = MyDataset(df=test_df, transform=transforms)\n    dataloader_test = DataLoader(dataset_test, batch_size=TEST_BATCH, num_workers=os.cpu_count(), shuffle=False, pin_memory=True)\n\n    image_ids = []\n    labels = []\n    preds = []\n\n    with torch.no_grad():\n        for step, batch in tqdm(enumerate(dataloader_test), total=len(dataloader_test)):\n\n            Xs = batch[0].to(device) # image\n            _image_ids = batch[1] # target\n\n            outputs = model(Xs)\n            outputs = outputs.sigmoid().to('cpu').numpy()\n\n            outputs2 = model(Xs.flip(-1))\n            outputs2 = outputs2.sigmoid().to('cpu').numpy()\n\n            outputs = (outputs + outputs) / 2\n\n            image_ids.extend(_image_ids)\n            preds.append(outputs)\n\n    preds = np.concatenate(preds)\n\n    gc.collect()\n\n    if RESOURCE == \"GPU\":\n        torch.cuda.empty_cache()\n\n    return image_ids, preds\n</code></pre>\n<p>Thaks </p>",
      "rawMarkdown": "I joined this competition week ago, so I didn't still understand this competition well.\n\nI tried some submission, but I saw submission takes long time.\n\nDo you have idea speeding up inference time?\n\nBelow is my inference code. model is one.\nThis code takes more than one hour with 5 model.\n\n```python\ndef infer(model, transforms=None):\n\n    dataset_test = MyDataset(df=test_df, transform=transforms)\n    dataloader_test = DataLoader(dataset_test, batch_size=TEST_BATCH, num_workers=os.cpu_count(), shuffle=False, pin_memory=True)\n            \n    image_ids = []\n    labels = []\n    preds = []\n\n    with torch.no_grad():\n        for step, batch in tqdm(enumerate(dataloader_test), total=len(dataloader_test)):\n\n            Xs = batch[0].to(device) # image\n            _image_ids = batch[1] # target\n            \n            outputs = model(Xs)\n            outputs = outputs.sigmoid().to('cpu').numpy()\n        \n            outputs2 = model(Xs.flip(-1))\n            outputs2 = outputs2.sigmoid().to('cpu').numpy()\n\n            outputs = (outputs + outputs) / 2\n            \n            image_ids.extend(_image_ids)\n            preds.append(outputs)\n            \n    preds = np.concatenate(preds)\n            \n    gc.collect()\n    \n    if RESOURCE == \"GPU\":\n        torch.cuda.empty_cache()\n            \n    return image_ids, preds\n```\n\nThaks",
      "votes": null
    },
    {
      "id": "1232409",
      "postDate": "03/09/2021 18:20:46",
      "content": "<p>The Time estimation i found in the inference time is <br>\n1 model takes 15 mins w/o TTA<br>\n1 model takes 18 min w/ TTA<br>\nif the no of Model increases in case of you its 5<br>\n5<em>15min = 75 min (1 hr and 15 min) w/o TTA\n5</em>18min = 90 min (1 hr and 30 min) w/ TTA</p>\n<p>you can just submit only in the Public LB to get fast inference result of Public LB (the private would be 0.000) . which takes only less than 2 min</p>",
      "rawMarkdown": "The Time estimation i found in the inference time is \n1 model takes 15 mins w/o TTA\n1 model takes 18 min w/ TTA\nif the no of Model increases in case of you its 5\n5*15min = 75 min (1 hr and 15 min) w/o TTA\n5*18min = 90 min (1 hr and 30 min) w/ TTA\n\nyou can just submit only in the Public LB to get fast inference result of Public LB (the private would be 0.000) . which takes only less than 2 min",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1232409,
      "author_name": "morizin",
      "author_url": "",
      "post_date": "03/09/2021 18:20:46",
      "content": "<p>The Time estimation i found in the inference time is <br>\n1 model takes 15 mins w/o TTA<br>\n1 model takes 18 min w/ TTA<br>\nif the no of Model increases in case of you its 5<br>\n5<em>15min = 75 min (1 hr and 15 min) w/o TTA\n5</em>18min = 90 min (1 hr and 30 min) w/ TTA</p>\n<p>you can just submit only in the Public LB to get fast inference result of Public LB (the private would be 0.000) . which takes only less than 2 min</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1230685": "I joined this competition week ago, so I didn't still understand this competition well.\n\nI tried some submission, but I saw submission takes long time.\n\nDo you have idea speeding up inference time?\n\nBelow is my inference code. model is one.\nThis code takes more than one hour with 5 model.\n\n```python\ndef infer(model, transforms=None):\n\n    dataset_test = MyDataset(df=test_df, transform=transforms)\n    dataloader_test = DataLoader(dataset_test, batch_size=TEST_BATCH, num_workers=os.cpu_count(), shuffle=False, pin_memory=True)\n            \n    image_ids = []\n    labels = []\n    preds = []\n\n    with torch.no_grad():\n        for step, batch in tqdm(enumerate(dataloader_test), total=len(dataloader_test)):\n\n            Xs = batch[0].to(device) # image\n            _image_ids = batch[1] # target\n            \n            outputs = model(Xs)\n            outputs = outputs.sigmoid().to('cpu').numpy()\n        \n            outputs2 = model(Xs.flip(-1))\n            outputs2 = outputs2.sigmoid().to('cpu').numpy()\n\n            outputs = (outputs + outputs) / 2\n            \n            image_ids.extend(_image_ids)\n            preds.append(outputs)\n            \n    preds = np.concatenate(preds)\n            \n    gc.collect()\n    \n    if RESOURCE == \"GPU\":\n        torch.cuda.empty_cache()\n            \n    return image_ids, preds\n```\n\nThaks",
    "1232409": "The Time estimation i found in the inference time is \n1 model takes 15 mins w/o TTA\n1 model takes 18 min w/ TTA\nif the no of Model increases in case of you its 5\n5*15min = 75 min (1 hr and 15 min) w/o TTA\n5*18min = 90 min (1 hr and 30 min) w/ TTA\n\nyou can just submit only in the Public LB to get fast inference result of Public LB (the private would be 0.000) . which takes only less than 2 min"
  },
  "source": "meta"
}