{
  "id": 201203,
  "title": "The possibility of reducing the inference time [Edited]",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/201203",
  "author_name": "",
  "post_date": "2020-12-03T17:09:27.645883200Z",
  "votes": 10,
  "comment_count": 4,
  "views": 0,
  "content": "<p>In this competition is a notebook competition and there is a limit of 9 hours to inference.<br>\nMeanwhile fortunately we can find 5 images of the public test set.<br>\nIs it possible to pre-infer these 5 images, and program them to infer only the private test set in 9 hours?<br>\nSince 42% is a public test set, we can estimate that this will save about 42% of the time if this scheme is successful.<br>\nHas anyone tried this? And were you successful?</p>\n<p>[12/04 13:21 updated]<br>\nI have succeeded in inferencing about public and private separately.<br>\nAlthough the official statement is unclear, you can save time and take advantage by only inferring private test sets and not inferring public test sets. For example, you can increase the number of TTAs.<br>\nBy the way, the public test set took 3h30m for inference and 5h20m for submission(private test set only) in my notebook.</p>",
  "messages": [
    {
      "id": "1101156",
      "postDate": "12/03/2020 17:09:27",
      "content": "<p>In this competition is a notebook competition and there is a limit of 9 hours to inference.<br>\nMeanwhile fortunately we can find 5 images of the public test set.<br>\nIs it possible to pre-infer these 5 images, and program them to infer only the private test set in 9 hours?<br>\nSince 42% is a public test set, we can estimate that this will save about 42% of the time if this scheme is successful.<br>\nHas anyone tried this? And were you successful?</p>\n<p>[12/04 13:21 updated]<br>\nI have succeeded in inferencing about public and private separately.<br>\nAlthough the official statement is unclear, you can save time and take advantage by only inferring private test sets and not inferring public test sets. For example, you can increase the number of TTAs.<br>\nBy the way, the public test set took 3h30m for inference and 5h20m for submission(private test set only) in my notebook.</p>",
      "rawMarkdown": "In this competition is a notebook competition and there is a limit of 9 hours to inference.\nMeanwhile fortunately we can find 5 images of the public test set.\nIs it possible to pre-infer these 5 images, and program them to infer only the private test set in 9 hours?\nSince 42% is a public test set, we can estimate that this will save about 42% of the time if this scheme is successful.\nHas anyone tried this? And were you successful?\n\n[12/04 13:21 updated]\nI have succeeded in inferencing about public and private separately.\nAlthough the official statement is unclear, you can save time and take advantage by only inferring private test sets and not inferring public test sets. For example, you can increase the number of TTAs.\nBy the way, the public test set took 3h30m for inference and 5h20m for submission(private test set only) in my notebook.",
      "votes": null
    },
    {
      "id": "1101158",
      "postDate": "12/03/2020 17:19:22",
      "content": "<p>Yes, only predicting on the private test set is a good way to reduce your final submission time. I have done this in previous competitions successfully.</p>",
      "rawMarkdown": "Yes, only predicting on the private test set is a good way to reduce your final submission time. I have done this in previous competitions successfully.",
      "votes": null
    },
    {
      "id": "1101163",
      "postDate": "12/03/2020 17:23:19",
      "content": "<p>same discussion in previous competition<br>\n<a href=\"https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/113731\" target=\"_blank\">https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/113731</a></p>",
      "rawMarkdown": "same discussion in previous competition\nhttps://www.kaggle.com/c/severstal-steel-defect-detection/discussion/113731",
      "votes": null
    },
    {
      "id": "1101269",
      "postDate": "12/03/2020 18:57:48",
      "content": "<p>Still no official answer though ^^</p>",
      "rawMarkdown": "Still no official answer though ^^",
      "votes": null
    },
    {
      "id": "1106720",
      "postDate": "12/09/2020 04:13:49",
      "content": "<p>That was brave.<br>\nSo, just to confirm - your public score was 0 or some random with legit private score and still you were not excluded from the leaderboard ?</p>",
      "rawMarkdown": "That was brave.\nSo, just to confirm - your public score was 0 or some random with legit private score and still you were not excluded from the leaderboard ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1101158,
      "author_name": "matthewmasters",
      "author_url": "",
      "post_date": "12/03/2020 17:19:22",
      "content": "<p>Yes, only predicting on the private test set is a good way to reduce your final submission time. I have done this in previous competitions successfully.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1106720,
          "author_name": "isakev",
          "author_url": "",
          "post_date": "12/09/2020 04:13:49",
          "content": "<p>That was brave.<br>\nSo, just to confirm - your public score was 0 or some random with legit private score and still you were not excluded from the leaderboard ?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1101163,
      "author_name": "phalanx",
      "author_url": "",
      "post_date": "12/03/2020 17:23:19",
      "content": "<p>same discussion in previous competition<br>\n<a href=\"https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/113731\" target=\"_blank\">https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/113731</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1101269,
          "author_name": "theoviel",
          "author_url": "",
          "post_date": "12/03/2020 18:57:48",
          "content": "<p>Still no official answer though ^^</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1101156": "In this competition is a notebook competition and there is a limit of 9 hours to inference.\nMeanwhile fortunately we can find 5 images of the public test set.\nIs it possible to pre-infer these 5 images, and program them to infer only the private test set in 9 hours?\nSince 42% is a public test set, we can estimate that this will save about 42% of the time if this scheme is successful.\nHas anyone tried this? And were you successful?\n\n[12/04 13:21 updated]\nI have succeeded in inferencing about public and private separately.\nAlthough the official statement is unclear, you can save time and take advantage by only inferring private test sets and not inferring public test sets. For example, you can increase the number of TTAs.\nBy the way, the public test set took 3h30m for inference and 5h20m for submission(private test set only) in my notebook.",
    "1101158": "Yes, only predicting on the private test set is a good way to reduce your final submission time. I have done this in previous competitions successfully.",
    "1101163": "same discussion in previous competition\nhttps://www.kaggle.com/c/severstal-steel-defect-detection/discussion/113731",
    "1101269": "Still no official answer though ^^",
    "1106720": "That was brave.\nSo, just to confirm - your public score was 0 or some random with legit private score and still you were not excluded from the leaderboard ?"
  },
  "source": "meta"
}