{
  "id": 95014,
  "title": "How is people dealing with inference time?",
  "url": "/competitions/freesound-audio-tagging-2019/discussion/95014",
  "author_name": "saewonYang",
  "post_date": "2019-06-09T07:48:20.351000",
  "votes": 5,
  "comment_count": 11,
  "views": 0,
  "content": "<p>I'm trying to inference in 20 minutes but having a lot of difficulty with time and memory.</p>\n\n<p>ensembling gives me a lot of score boost but I'm hitting a time limit.</p>\n\n<p>whenever I infer it gives me different time. between 17 ~ 22 minutes (I don't know why but I think its because of kaggle server status, any idea?)</p>\n\n<p>I'm nervous because the host are not telling us the exact size of the private test size.</p>\n\n<p>how about guys in the high score? any comment would be appreciated. thanks!</p>",
  "messages": [
    {
      "id": 548331,
      "postDate": "2019-06-09T07:48:20.353Z",
      "content": "<p>I'm trying to inference in 20 minutes but having a lot of difficulty with time and memory.</p>\n\n<p>ensembling gives me a lot of score boost but I'm hitting a time limit.</p>\n\n<p>whenever I infer it gives me different time. between 17 ~ 22 minutes (I don't know why but I think its because of kaggle server status, any idea?)</p>\n\n<p>I'm nervous because the host are not telling us the exact size of the private test size.</p>\n\n<p>how about guys in the high score? any comment would be appreciated. thanks!</p>",
      "rawMarkdown": "I'm trying to inference in 20 minutes but having a lot of difficulty with time and memory.\n\nensembling gives me a lot of score boost but I'm hitting a time limit.\n\nwhenever I infer it gives me different time. between 17 ~ 22 minutes (I don't know why but I think its because of kaggle server status, any idea?)\n\nI'm nervous because the host are not telling us the exact size of the private test size.\n\nhow about guys in the high score? any comment would be appreciated. thanks!\n\n",
      "votes": 5
    },
    {
      "id": 548909,
      "postDate": "2019-06-10T05:02:56.613Z",
      "content": "<p>I wish the host tells us the exact size of the private test set :(</p>",
      "rawMarkdown": "I wish the host tells us the exact size of the private test set :("
    },
    {
      "id": 548891,
      "postDate": "2019-06-10T04:22:47.373Z",
      "content": "<p>I am testing my submission code with test data duplicated 3 (or 4) times in order to test time limit exceeded and out of memory.\nI am also afraid with OOM, because In iMet 2019 competition, which is a kernel competition too,  some people suffered OOM error.</p>",
      "rawMarkdown": "I am testing my submission code with test data duplicated 3 (or 4) times in order to test time limit exceeded and out of memory.\nI am also afraid with OOM, because In iMet 2019 competition, which is a kernel competition too,  some people suffered OOM error.\n",
      "replies": [
        {
          "id": 548895,
          "postDate": "2019-06-10T04:27:50.807Z",
          "content": "<p>testing with 3 times test data seems to be perfect tip! yeah right I participated with imet there was same issue too and some people got dropped because of kernel inference restrictions if I remember it right :(</p>",
          "rawMarkdown": "testing with 3 times test data seems to be perfect tip! yeah right I participated with imet there was same issue too and some people got dropped because of kernel inference restrictions if I remember it right :("
        }
      ]
    },
    {
      "id": 548667,
      "postDate": "2019-06-09T17:22:52.150Z",
      "content": "<p><a href=\"/yangsaewon\">@yangsaewon</a> In some threads the organizers have mentioned that stage2 test dataset will be approx. 3 times the current test dataset. On inference time, my model was also taking lot of time as by default GPU was disabled in the kernel. Once I enabled it, the processing became fast.</p>",
      "rawMarkdown": "@yangsaewon In some threads the organizers have mentioned that stage2 test dataset will be approx. 3 times the current test dataset. On inference time, my model was also taking lot of time as by default GPU was disabled in the kernel. Once I enabled it, the processing became fast.",
      "replies": [
        {
          "id": 548829,
          "postDate": "2019-06-10T01:34:05.370Z",
          "content": "<p>yeah of course I checked gpu status, thanks for the advice!</p>",
          "rawMarkdown": "yeah of course I checked gpu status, thanks for the advice!"
        }
      ]
    },
    {
      "id": 548345,
      "postDate": "2019-06-09T08:17:21.770Z",
      "content": "<p>I'm trying to keep the inference time under 15 minutes for the current test set. I think this should be enough to process the private part in one hour. 20 minutes seem to be a \"danger\" zone for me.</p>",
      "rawMarkdown": "I'm trying to keep the inference time under 15 minutes for the current test set. I think this should be enough to process the private part in one hour. 20 minutes seem to be a \"danger\" zone for me.",
      "replies": [
        {
          "id": 548348,
          "postDate": "2019-06-09T08:22:00.560Z",
          "content": "<p>but can you ensemble models within 15 minutes which includes preprocessing time?</p>",
          "rawMarkdown": "but can you ensemble models within 15 minutes which includes preprocessing time?"
        },
        {
          "id": 548358,
          "postDate": "2019-06-09T08:56:18.447Z",
          "content": "<p>Sure. A single model (5 folds) takes only ~1 minute to process the current test set (preprocessing time included), so making ensemble is not a problem.</p>",
          "rawMarkdown": "Sure. A single model (5 folds) takes only ~1 minute to process the current test set (preprocessing time included), so making ensemble is not a problem."
        },
        {
          "id": 548399,
          "postDate": "2019-06-09T10:34:57.280Z",
          "content": "<p>I see, seems like your model is much smaller than mine. I hope lot of participants are well noticed about the inference time :) thanks for the comment!</p>",
          "rawMarkdown": "I see, seems like your model is much smaller than mine. I hope lot of participants are well noticed about the inference time :) thanks for the comment!"
        },
        {
          "id": 548551,
          "postDate": "2019-06-09T15:06:25.717Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 548830,
          "postDate": "2019-06-10T01:35:11.407Z",
          "content": "<p>oh you put a time for it, I wish I could see the code after the deadline :)</p>",
          "rawMarkdown": "oh you put a time for it, I wish I could see the code after the deadline :)"
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 548909,
      "author_name": "chrisyang7",
      "author_url": "",
      "post_date": "2019-06-10T05:02:56.613000",
      "content": "<p>I wish the host tells us the exact size of the private test set :(</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 548891,
      "author_name": "mrkmakr",
      "author_url": "",
      "post_date": "2019-06-10T04:22:47.373000",
      "content": "<p>I am testing my submission code with test data duplicated 3 (or 4) times in order to test time limit exceeded and out of memory.\nI am also afraid with OOM, because In iMet 2019 competition, which is a kernel competition too,  some people suffered OOM error.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 548895,
          "author_name": "saewonYang",
          "author_url": "",
          "post_date": "2019-06-10T04:27:50.807000",
          "content": "<p>testing with 3 times test data seems to be perfect tip! yeah right I participated with imet there was same issue too and some people got dropped because of kernel inference restrictions if I remember it right :(</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 548667,
      "author_name": "Deepak Singh Rawat",
      "author_url": "",
      "post_date": "2019-06-09T17:22:52.150000",
      "content": "<p><a href=\"/yangsaewon\">@yangsaewon</a> In some threads the organizers have mentioned that stage2 test dataset will be approx. 3 times the current test dataset. On inference time, my model was also taking lot of time as by default GPU was disabled in the kernel. Once I enabled it, the processing became fast.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 548829,
          "author_name": "saewonYang",
          "author_url": "",
          "post_date": "2019-06-10T01:34:05.370000",
          "content": "<p>yeah of course I checked gpu status, thanks for the advice!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 548345,
      "author_name": "Dmytro Danevskyi",
      "author_url": "",
      "post_date": "2019-06-09T08:17:21.770000",
      "content": "<p>I'm trying to keep the inference time under 15 minutes for the current test set. I think this should be enough to process the private part in one hour. 20 minutes seem to be a \"danger\" zone for me.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 548348,
          "author_name": "saewonYang",
          "author_url": "",
          "post_date": "2019-06-09T08:22:00.560000",
          "content": "<p>but can you ensemble models within 15 minutes which includes preprocessing time?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 548358,
          "author_name": "Dmytro Danevskyi",
          "author_url": "",
          "post_date": "2019-06-09T08:56:18.447000",
          "content": "<p>Sure. A single model (5 folds) takes only ~1 minute to process the current test set (preprocessing time included), so making ensemble is not a problem.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 548399,
          "author_name": "saewonYang",
          "author_url": "",
          "post_date": "2019-06-09T10:34:57.280000",
          "content": "<p>I see, seems like your model is much smaller than mine. I hope lot of participants are well noticed about the inference time :) thanks for the comment!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 548551,
          "author_name": "",
          "author_url": "",
          "post_date": "2019-06-09T15:06:25.717000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 548830,
          "author_name": "saewonYang",
          "author_url": "",
          "post_date": "2019-06-10T01:35:11.407000",
          "content": "<p>oh you put a time for it, I wish I could see the code after the deadline :)</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "548331": "I'm trying to inference in 20 minutes but having a lot of difficulty with time and memory.\n\nensembling gives me a lot of score boost but I'm hitting a time limit.\n\nwhenever I infer it gives me different time. between 17 ~ 22 minutes (I don't know why but I think its because of kaggle server status, any idea?)\n\nI'm nervous because the host are not telling us the exact size of the private test size.\n\nhow about guys in the high score? any comment would be appreciated. thanks!\n\n",
    "548909": "I wish the host tells us the exact size of the private test set :(",
    "548891": "I am testing my submission code with test data duplicated 3 (or 4) times in order to test time limit exceeded and out of memory.\nI am also afraid with OOM, because In iMet 2019 competition, which is a kernel competition too,  some people suffered OOM error.\n",
    "548667": "@yangsaewon In some threads the organizers have mentioned that stage2 test dataset will be approx. 3 times the current test dataset. On inference time, my model was also taking lot of time as by default GPU was disabled in the kernel. Once I enabled it, the processing became fast.",
    "548345": "I'm trying to keep the inference time under 15 minutes for the current test set. I think this should be enough to process the private part in one hour. 20 minutes seem to be a \"danger\" zone for me."
  }
}