{
  "id": 174493,
  "title": "Processing time of Baseline submission",
  "url": "/competitions/landmark-recognition-2020/discussion/174493",
  "author_name": "",
  "post_date": "2020-08-13T19:04:03.347650100Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hey guys,</p>\n<p>I can't make sense of baseline submission in terms of performance. When running Host Kernel with GPU enabled it seems that processing 1.5m images for global features will take around 12 hours.</p>\n<p>Another thing is that calculation of cosine distances of even 100k test images against 1.5m training example will take another 6 hours.</p>\n<p>I am also experimenting at my local machine that is much faster and still cosine calculation seems to take very long time on CPU. </p>\n<p>I understand that one can use Faiss, LSH and other advanced matching approaches, but I just get it - how the baseline code is able to fit into 12 hours? What am I missing?</p>",
  "messages": [
    {
      "id": "969546",
      "postDate": "08/13/2020 19:04:03",
      "content": "<p>Hey guys,</p>\n<p>I can't make sense of baseline submission in terms of performance. When running Host Kernel with GPU enabled it seems that processing 1.5m images for global features will take around 12 hours.</p>\n<p>Another thing is that calculation of cosine distances of even 100k test images against 1.5m training example will take another 6 hours.</p>\n<p>I am also experimenting at my local machine that is much faster and still cosine calculation seems to take very long time on CPU. </p>\n<p>I understand that one can use Faiss, LSH and other advanced matching approaches, but I just get it - how the baseline code is able to fit into 12 hours? What am I missing?</p>",
      "rawMarkdown": "Hey guys,\n\nI can't make sense of baseline submission in terms of performance. When running Host Kernel with GPU enabled it seems that processing 1.5m images for global features will take around 12 hours.\n\nAnother thing is that calculation of cosine distances of even 100k test images against 1.5m training example will take another 6 hours.\n\nI am also experimenting at my local machine that is much faster and still cosine calculation seems to take very long time on CPU. \n\nI understand that one can use Faiss, LSH and other advanced matching approaches, but I just get it - how the baseline code is able to fit into 12 hours? What am I missing?",
      "votes": null
    },
    {
      "id": "969681",
      "postDate": "08/13/2020 21:05:06",
      "content": "<p>The train folder in the submission contains only 100K images, not 1.5M. So, this reduces the processing time in the submission.</p>",
      "rawMarkdown": "The train folder in the submission contains only 100K images, not 1.5M. So, this reduces the processing time in the submission.",
      "votes": null
    },
    {
      "id": "975164",
      "postDate": "08/18/2020 07:19:45",
      "content": "<p>You can do batched cosine distance on GPU </p>",
      "rawMarkdown": "You can do batched cosine distance on GPU",
      "votes": null
    },
    {
      "id": "982111",
      "postDate": "08/23/2020 04:37:12",
      "content": "<p>By the way, the baseline notebook took about 5.5 hours to complete.</p>",
      "rawMarkdown": "By the way, the baseline notebook took about 5.5 hours to complete.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 969681,
      "author_name": "tolgadincer",
      "author_url": "",
      "post_date": "08/13/2020 21:05:06",
      "content": "<p>The train folder in the submission contains only 100K images, not 1.5M. So, this reduces the processing time in the submission.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 975164,
      "author_name": "christofhenkel",
      "author_url": "",
      "post_date": "08/18/2020 07:19:45",
      "content": "<p>You can do batched cosine distance on GPU </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 982111,
      "author_name": "chankhavu",
      "author_url": "",
      "post_date": "08/23/2020 04:37:12",
      "content": "<p>By the way, the baseline notebook took about 5.5 hours to complete.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "969546": "Hey guys,\n\nI can't make sense of baseline submission in terms of performance. When running Host Kernel with GPU enabled it seems that processing 1.5m images for global features will take around 12 hours.\n\nAnother thing is that calculation of cosine distances of even 100k test images against 1.5m training example will take another 6 hours.\n\nI am also experimenting at my local machine that is much faster and still cosine calculation seems to take very long time on CPU. \n\nI understand that one can use Faiss, LSH and other advanced matching approaches, but I just get it - how the baseline code is able to fit into 12 hours? What am I missing?",
    "969681": "The train folder in the submission contains only 100K images, not 1.5M. So, this reduces the processing time in the submission.",
    "975164": "You can do batched cosine distance on GPU",
    "982111": "By the way, the baseline notebook took about 5.5 hours to complete."
  },
  "source": "meta"
}