{
  "id": 21428,
  "title": "Question about printing validation performance ",
  "url": "/competitions/expedia-hotel-recommendations/discussion/21428",
  "author_name": "",
  "post_date": "2016-06-04T15:48:11.417Z",
  "votes": null,
  "comment_count": 5,
  "views": 524,
  "content": "<pre><code>[30]    eval-MAP@5:0.273960 train-MAP@5:0.288924\n[31]    eval-MAP@5:0.276115 train-MAP@5:0.291002\n[32]    eval-MAP@5:0.278767 train-MAP@5:0.294094\n[33]    eval-MAP@5:0.280590 train-MAP@5:0.296954\n[34]    eval-MAP@5:0.281628 train-MAP@5:0.297273\n[35]    eval-MAP@5:0.284637 train-MAP@5:0.301252\n[36]    eval-MAP@5:0.286232 train-MAP@5:0.302846\n[37]    eval-MAP@5:0.287402 train-MAP@5:0.303931\n</code></pre>\n\n<p>I went to sleep leaving my code running last night. But this morning I could just see 37 rows of validation information printed in my  Jupyter but the terminal and kernel  is still running. </p>\n\n<p>How could I solve this problem? Thank you.</p>",
  "messages": [
    {
      "id": "122494",
      "postDate": "06/04/2016 15:48:11",
      "content": "<pre><code>[30]    eval-MAP@5:0.273960 train-MAP@5:0.288924\n[31]    eval-MAP@5:0.276115 train-MAP@5:0.291002\n[32]    eval-MAP@5:0.278767 train-MAP@5:0.294094\n[33]    eval-MAP@5:0.280590 train-MAP@5:0.296954\n[34]    eval-MAP@5:0.281628 train-MAP@5:0.297273\n[35]    eval-MAP@5:0.284637 train-MAP@5:0.301252\n[36]    eval-MAP@5:0.286232 train-MAP@5:0.302846\n[37]    eval-MAP@5:0.287402 train-MAP@5:0.303931\n</code></pre>\n\n<p>I went to sleep leaving my code running last night. But this morning I could just see 37 rows of validation information printed in my  Jupyter but the terminal and kernel  is still running. </p>\n\n<p>How could I solve this problem? Thank you.</p>",
      "rawMarkdown": "[30]\teval-MAP@5:0.273960\ttrain-MAP@5:0.288924\r\n    [31]\teval-MAP@5:0.276115\ttrain-MAP@5:0.291002\r\n    [32]\teval-MAP@5:0.278767\ttrain-MAP@5:0.294094\r\n    [33]\teval-MAP@5:0.280590\ttrain-MAP@5:0.296954\r\n    [34]\teval-MAP@5:0.281628\ttrain-MAP@5:0.297273\r\n    [35]\teval-MAP@5:0.284637\ttrain-MAP@5:0.301252\r\n    [36]\teval-MAP@5:0.286232\ttrain-MAP@5:0.302846\r\n    [37]\teval-MAP@5:0.287402\ttrain-MAP@5:0.303931\r\n\r\nI went to sleep leaving my code running last night. But this morning I could just see 37 rows of validation information printed in my  Jupyter but the terminal and kernel  is still running. \r\n\r\nHow could I solve this problem? Thank you.",
      "votes": null
    },
    {
      "id": "122508",
      "postDate": "06/04/2016 18:03:52",
      "content": "<p>Maybe there is no problem, and you code is working iteration 38....</p>",
      "rawMarkdown": "Maybe there is no problem, and you code is working iteration 38....",
      "votes": null
    },
    {
      "id": "122510",
      "postDate": "06/04/2016 18:05:53",
      "content": "<p>The gap between each iteration should be approximately same. I have waited for more than 2hours....</p>",
      "rawMarkdown": "The gap between each iteration should be approximately same. I have waited for more than 2hours....",
      "votes": null
    },
    {
      "id": "122515",
      "postDate": "06/04/2016 18:45:10",
      "content": "<p>Try to make reconnect. Do not restart, namely reconnect. Maybe the browser has lost communication with the kernel simply does not accept messages from the kernel. Reconnect in the menu item</p>\n\n<p>If this does not help, I do not know, then the task can continue to run.</p>",
      "rawMarkdown": "Try to make reconnect. Do not restart, namely reconnect. Maybe the browser has lost communication with the kernel simply does not accept messages from the kernel. Reconnect in the menu item\r\n\r\nIf this does not help, I do not know, then the task can continue to run.",
      "votes": null
    },
    {
      "id": "122518",
      "postDate": "06/04/2016 19:13:59",
      "content": "<p>Sometimes the Python / R kernel loses the communication with xgboost due to threading violations. I had &quot;xgboost not responding&quot; (kernel not communicating with xgboost anymore although xgboost keeps computing) on very specific data sets (at very specific moments), especially if a thread (from any process) violates xgboost process (collision). In Windows it should be safeguarded against that, but it does happens in Windows and I also found these issues sometimes happens in Linux. Doing actions while xgboost is computing increases the (infinitesimally low) odd of xgboost not communicating anymore with the R / Python kernel.</p>\n\n<p>In Windows, I also found out setting the kernel running thread on specific cores only delays the &quot;communication loss&quot; on very special <em>&quot;crashing&quot;</em> data sets.</p>\n\n<p>The best scenario is you used the parameter to dump the model periodically (if you were to use train and not CV). Otherwise, you are gone having to restart from scratch (I found no other solution myself to this issue when facing it, even if it incurred a 1-day computing lost).</p>\n\n<p>It might be also possible that you just lost connection with the kernel and you would just have to reconnect, but I doubt it is the case. You can always try this solution, but as @Vladimir Sorokin said, do <strong>not restart</strong> but reconnect only.</p>",
      "rawMarkdown": "Sometimes the Python / R kernel loses the communication with xgboost due to threading violations. I had \"xgboost not responding\" (kernel not communicating with xgboost anymore although xgboost keeps computing) on very specific data sets (at very specific moments), especially if a thread (from any process) violates xgboost process (collision). In Windows it should be safeguarded against that, but it does happens in Windows and I also found these issues sometimes happens in Linux. Doing actions while xgboost is computing increases the (infinitesimally low) odd of xgboost not communicating anymore with the R / Python kernel.\r\n\r\nIn Windows, I also found out setting the kernel running thread on specific cores only delays the \"communication loss\" on very special *\"crashing\"* data sets.\r\n\r\nThe best scenario is you used the parameter to dump the model periodically (if you were to use train and not CV). Otherwise, you are gone having to restart from scratch (I found no other solution myself to this issue when facing it, even if it incurred a 1-day computing lost).\r\n\r\nIt might be also possible that you just lost connection with the kernel and you would just have to reconnect, but I doubt it is the case. You can always try this solution, but as @Vladimir Sorokin said, do **not restart** but reconnect only.",
      "votes": null
    },
    {
      "id": "122530",
      "postDate": "06/04/2016 21:10:59",
      "content": "<p>Thank you very much. </p>",
      "rawMarkdown": "Thank you very much.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 122508,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "06/04/2016 18:03:52",
      "content": "<p>Maybe there is no problem, and you code is working iteration 38....</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 122510,
      "author_name": "beedata",
      "author_url": "",
      "post_date": "06/04/2016 18:05:53",
      "content": "<p>The gap between each iteration should be approximately same. I have waited for more than 2hours....</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 122515,
      "author_name": "sorokinv",
      "author_url": "",
      "post_date": "06/04/2016 18:45:10",
      "content": "<p>Try to make reconnect. Do not restart, namely reconnect. Maybe the browser has lost communication with the kernel simply does not accept messages from the kernel. Reconnect in the menu item</p>\n\n<p>If this does not help, I do not know, then the task can continue to run.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 122518,
      "author_name": "laurae2",
      "author_url": "",
      "post_date": "06/04/2016 19:13:59",
      "content": "<p>Sometimes the Python / R kernel loses the communication with xgboost due to threading violations. I had &quot;xgboost not responding&quot; (kernel not communicating with xgboost anymore although xgboost keeps computing) on very specific data sets (at very specific moments), especially if a thread (from any process) violates xgboost process (collision). In Windows it should be safeguarded against that, but it does happens in Windows and I also found these issues sometimes happens in Linux. Doing actions while xgboost is computing increases the (infinitesimally low) odd of xgboost not communicating anymore with the R / Python kernel.</p>\n\n<p>In Windows, I also found out setting the kernel running thread on specific cores only delays the &quot;communication loss&quot; on very special <em>&quot;crashing&quot;</em> data sets.</p>\n\n<p>The best scenario is you used the parameter to dump the model periodically (if you were to use train and not CV). Otherwise, you are gone having to restart from scratch (I found no other solution myself to this issue when facing it, even if it incurred a 1-day computing lost).</p>\n\n<p>It might be also possible that you just lost connection with the kernel and you would just have to reconnect, but I doubt it is the case. You can always try this solution, but as @Vladimir Sorokin said, do <strong>not restart</strong> but reconnect only.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 122530,
      "author_name": "beedata",
      "author_url": "",
      "post_date": "06/04/2016 21:10:59",
      "content": "<p>Thank you very much. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "122494": "[30]\teval-MAP@5:0.273960\ttrain-MAP@5:0.288924\r\n    [31]\teval-MAP@5:0.276115\ttrain-MAP@5:0.291002\r\n    [32]\teval-MAP@5:0.278767\ttrain-MAP@5:0.294094\r\n    [33]\teval-MAP@5:0.280590\ttrain-MAP@5:0.296954\r\n    [34]\teval-MAP@5:0.281628\ttrain-MAP@5:0.297273\r\n    [35]\teval-MAP@5:0.284637\ttrain-MAP@5:0.301252\r\n    [36]\teval-MAP@5:0.286232\ttrain-MAP@5:0.302846\r\n    [37]\teval-MAP@5:0.287402\ttrain-MAP@5:0.303931\r\n\r\nI went to sleep leaving my code running last night. But this morning I could just see 37 rows of validation information printed in my  Jupyter but the terminal and kernel  is still running. \r\n\r\nHow could I solve this problem? Thank you.",
    "122508": "Maybe there is no problem, and you code is working iteration 38....",
    "122510": "The gap between each iteration should be approximately same. I have waited for more than 2hours....",
    "122515": "Try to make reconnect. Do not restart, namely reconnect. Maybe the browser has lost communication with the kernel simply does not accept messages from the kernel. Reconnect in the menu item\r\n\r\nIf this does not help, I do not know, then the task can continue to run.",
    "122518": "Sometimes the Python / R kernel loses the communication with xgboost due to threading violations. I had \"xgboost not responding\" (kernel not communicating with xgboost anymore although xgboost keeps computing) on very specific data sets (at very specific moments), especially if a thread (from any process) violates xgboost process (collision). In Windows it should be safeguarded against that, but it does happens in Windows and I also found these issues sometimes happens in Linux. Doing actions while xgboost is computing increases the (infinitesimally low) odd of xgboost not communicating anymore with the R / Python kernel.\r\n\r\nIn Windows, I also found out setting the kernel running thread on specific cores only delays the \"communication loss\" on very special *\"crashing\"* data sets.\r\n\r\nThe best scenario is you used the parameter to dump the model periodically (if you were to use train and not CV). Otherwise, you are gone having to restart from scratch (I found no other solution myself to this issue when facing it, even if it incurred a 1-day computing lost).\r\n\r\nIt might be also possible that you just lost connection with the kernel and you would just have to reconnect, but I doubt it is the case. You can always try this solution, but as @Vladimir Sorokin said, do **not restart** but reconnect only.",
    "122530": "Thank you very much."
  },
  "source": "meta"
}