{
  "id": 88070,
  "title": "Is a stopped-after-9-hours model eligible for stage 2 leaderboard? (having it a proper submission.csv output)",
  "url": "/competitions/imet-2019-fgvc6/discussion/88070",
  "author_name": "",
  "post_date": "2019-04-05T14:23:31.929891300Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I'm doing models that are being stoped after 9 hours:</p>\n\n<pre><code>Failed. Exited with code 137\n</code></pre>\n\n<p>They are marked with a failure symbol Ⓧ. But these models contain a submission.csv inside.</p>\n\n<p>The idea is to have an strategy based on adding enhancements to the solution during the 9 hours.</p>\n\n<p>And, no matters if it were stopped during the enhancement N+1, because it would have the submissions.csv made after the iteration N</p>\n\n<p>Thanks in advance</p>",
  "messages": [
    {
      "id": "508017",
      "postDate": "04/05/2019 14:23:31",
      "content": "<p>I'm doing models that are being stoped after 9 hours:</p>\n\n<pre><code>Failed. Exited with code 137\n</code></pre>\n\n<p>They are marked with a failure symbol Ⓧ. But these models contain a submission.csv inside.</p>\n\n<p>The idea is to have an strategy based on adding enhancements to the solution during the 9 hours.</p>\n\n<p>And, no matters if it were stopped during the enhancement N+1, because it would have the submissions.csv made after the iteration N</p>\n\n<p>Thanks in advance</p>",
      "rawMarkdown": "I'm doing models that are being stoped after 9 hours:\n\n\tFailed. Exited with code 137\n\nThey are marked with a failure symbol Ⓧ. But these models contain a submission.csv inside.\n\nThe idea is to have an strategy based on adding enhancements to the solution during the 9 hours.\n\nAnd, no matters if it were stopped during the enhancement N+1, because it would have the submissions.csv made after the iteration N\n\nThanks in advance",
      "votes": null
    },
    {
      "id": "518705",
      "postDate": "04/17/2019 16:33:29",
      "content": "<p>I think submission.csv file won't help in this case since your kernel will be re-runned with new test set which is 5 times bigger comparing to current one. \nSo if your kernel can't finish within 9 hours with current test set, it won't be able to finish with stage 2 test set also.</p>",
      "rawMarkdown": "I think submission.csv file won't help in this case since your kernel will be re-runned with new test set which is 5 times bigger comparing to current one. \nSo if your kernel can't finish within 9 hours with current test set, it won't be able to finish with stage 2 test set also.",
      "votes": null
    },
    {
      "id": "524275",
      "postDate": "04/28/2019 11:45:36",
      "content": "<p>thanks <a href=\"/demonplus\">@demonplus</a> </p>\n\n<p>yes, it could be a probem if my kernel wasn't able to run for stage 1 in 9 hours!</p>\n\n<p>I have planned to run a load test by replicating each stage-1 image five times.\nIn this way, I could test not only that it runs in 9 hours, but also that I'm not going to have a out of memory error.</p>\n\n<p>I think I didn't explain my self.  I was asking about this strategy:</p>\n\n<pre><code>- model 1 inference --&gt; submission.csv\n- model 2 inference\n- model 1 and model 2 ensemble --&gt; submission.csv\n- model 3 inference\n- model 1,2 and 3 ensemble --&gt; submission.csv\n\n... until N models.\n</code></pre>\n\n<p>In this way, the process could be greedy and use a number N of models to be very close to the 9 hours.\nBut, if it exceeded the 9 hours limit during the round i, would have the submission.csv created during the round i-1</p>\n\n<p>Anyway, I think I could add a timer or something to break the process close to the 9 hours limit.</p>\n\n<p>Also, I'm not very sure if I'm going to be able to get good results enough to implement such a complex pipeline</p>",
      "rawMarkdown": "thanks @demonplus \n\nyes, it could be a probem if my kernel wasn't able to run for stage 1 in 9 hours!\n\nI have planned to run a load test by replicating each stage-1 image five times.\nIn this way, I could test not only that it runs in 9 hours, but also that I'm not going to have a out of memory error.\n\nI think I didn't explain my self.  I was asking about this strategy:\n\n\t- model 1 inference --&gt; submission.csv\n\t- model 2 inference\n\t- model 1 and model 2 ensemble --&gt; submission.csv\n\t- model 3 inference\n\t- model 1,2 and 3 ensemble --&gt; submission.csv\n\n\t... until N models.\n\nIn this way, the process could be greedy and use a number N of models to be very close to the 9 hours.\nBut, if it exceeded the 9 hours limit during the round i, would have the submission.csv created during the round i-1\n\nAnyway, I think I could add a timer or something to break the process close to the 9 hours limit.\n\nAlso, I'm not very sure if I'm going to be able to get good results enough to implement such a complex pipeline",
      "votes": null
    },
    {
      "id": "527600",
      "postDate": "05/05/2019 22:56:50",
      "content": "<p>it seems that a timer in kernel script can fulfill your purpose</p>",
      "rawMarkdown": "it seems that a timer in kernel script can fulfill your purpose",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 518705,
      "author_name": "demonplus",
      "author_url": "",
      "post_date": "04/17/2019 16:33:29",
      "content": "<p>I think submission.csv file won't help in this case since your kernel will be re-runned with new test set which is 5 times bigger comparing to current one. \nSo if your kernel can't finish within 9 hours with current test set, it won't be able to finish with stage 2 test set also.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 524275,
      "author_name": "virilo",
      "author_url": "",
      "post_date": "04/28/2019 11:45:36",
      "content": "<p>thanks <a href=\"/demonplus\">@demonplus</a> </p>\n\n<p>yes, it could be a probem if my kernel wasn't able to run for stage 1 in 9 hours!</p>\n\n<p>I have planned to run a load test by replicating each stage-1 image five times.\nIn this way, I could test not only that it runs in 9 hours, but also that I'm not going to have a out of memory error.</p>\n\n<p>I think I didn't explain my self.  I was asking about this strategy:</p>\n\n<pre><code>- model 1 inference --&gt; submission.csv\n- model 2 inference\n- model 1 and model 2 ensemble --&gt; submission.csv\n- model 3 inference\n- model 1,2 and 3 ensemble --&gt; submission.csv\n\n... until N models.\n</code></pre>\n\n<p>In this way, the process could be greedy and use a number N of models to be very close to the 9 hours.\nBut, if it exceeded the 9 hours limit during the round i, would have the submission.csv created during the round i-1</p>\n\n<p>Anyway, I think I could add a timer or something to break the process close to the 9 hours limit.</p>\n\n<p>Also, I'm not very sure if I'm going to be able to get good results enough to implement such a complex pipeline</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 527600,
      "author_name": "weizhezhang",
      "author_url": "",
      "post_date": "05/05/2019 22:56:50",
      "content": "<p>it seems that a timer in kernel script can fulfill your purpose</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "508017": "I'm doing models that are being stoped after 9 hours:\n\n\tFailed. Exited with code 137\n\nThey are marked with a failure symbol Ⓧ. But these models contain a submission.csv inside.\n\nThe idea is to have an strategy based on adding enhancements to the solution during the 9 hours.\n\nAnd, no matters if it were stopped during the enhancement N+1, because it would have the submissions.csv made after the iteration N\n\nThanks in advance",
    "518705": "I think submission.csv file won't help in this case since your kernel will be re-runned with new test set which is 5 times bigger comparing to current one. \nSo if your kernel can't finish within 9 hours with current test set, it won't be able to finish with stage 2 test set also.",
    "524275": "thanks @demonplus \n\nyes, it could be a probem if my kernel wasn't able to run for stage 1 in 9 hours!\n\nI have planned to run a load test by replicating each stage-1 image five times.\nIn this way, I could test not only that it runs in 9 hours, but also that I'm not going to have a out of memory error.\n\nI think I didn't explain my self.  I was asking about this strategy:\n\n\t- model 1 inference --&gt; submission.csv\n\t- model 2 inference\n\t- model 1 and model 2 ensemble --&gt; submission.csv\n\t- model 3 inference\n\t- model 1,2 and 3 ensemble --&gt; submission.csv\n\n\t... until N models.\n\nIn this way, the process could be greedy and use a number N of models to be very close to the 9 hours.\nBut, if it exceeded the 9 hours limit during the round i, would have the submission.csv created during the round i-1\n\nAnyway, I think I could add a timer or something to break the process close to the 9 hours limit.\n\nAlso, I'm not very sure if I'm going to be able to get good results enough to implement such a complex pipeline",
    "527600": "it seems that a timer in kernel script can fulfill your purpose"
  },
  "source": "meta"
}