{
  "id": 18309,
  "title": "Recover terminated aws spot instance",
  "url": "/competitions/second-annual-data-science-bowl/discussion/18309",
  "author_name": "",
  "post_date": "2016-01-08T20:36:40.890Z",
  "votes": null,
  "comment_count": 3,
  "views": 909,
  "content": "<p>Does anyone know if it is possible to recover an aws spot instance image that was terminated because of bid price? I have been training for a while and all that might be lost.</p>",
  "messages": [
    {
      "id": "104061",
      "postDate": "01/08/2016 20:36:40",
      "content": "<p>Does anyone know if it is possible to recover an aws spot instance image that was terminated because of bid price? I have been training for a while and all that might be lost.</p>",
      "rawMarkdown": "Does anyone know if it is possible to recover an aws spot instance image that was terminated because of bid price? I have been training for a while and all that might be lost.",
      "votes": null
    },
    {
      "id": "104067",
      "postDate": "01/08/2016 23:00:27",
      "content": "<p>If it was stopped, you can restart it, but if it's been terminated the instance data is gone and cannot be recovered.</p>\n\n<p><a href=\"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html\">http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html</a></p>",
      "rawMarkdown": "If it was stopped, you can restart it, but if it's been terminated the instance data is gone and cannot be recovered.\r\n\r\nhttp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html",
      "votes": null
    },
    {
      "id": "104147",
      "postDate": "01/09/2016 17:18:40",
      "content": "<p>you might want to save epochs on local disk (EBS or S3, MXnet supports both as I remember). Spot instance would be gone forever when it is turned off by the 'price cut'. One can also save the pre-processed data to disk, as well as the configured code, so one can quickly create another spot instance when the last one was killed.</p>",
      "rawMarkdown": "you might want to save epochs on local disk (EBS or S3, MXnet supports both as I remember). Spot instance would be gone forever when it is turned off by the 'price cut'. One can also save the pre-processed data to disk, as well as the configured code, so one can quickly create another spot instance when the last one was killed.",
      "votes": null
    },
    {
      "id": "104151",
      "postDate": "01/09/2016 18:07:56",
      "content": "<p>Thanks for the response, it would have been nice if amazon kept the spot instance image for a day or two so users can salvage any work. I lost multiple hours of processing over a few cents price change, once bitten twice shy.</p>",
      "rawMarkdown": "Thanks for the response, it would have been nice if amazon kept the spot instance image for a day or two so users can salvage any work. I lost multiple hours of processing over a few cents price change, once bitten twice shy.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 104067,
      "author_name": "midikiman",
      "author_url": "",
      "post_date": "01/08/2016 23:00:27",
      "content": "<p>If it was stopped, you can restart it, but if it's been terminated the instance data is gone and cannot be recovered.</p>\n\n<p><a href=\"http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html\">http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 104147,
      "author_name": "phunter",
      "author_url": "",
      "post_date": "01/09/2016 17:18:40",
      "content": "<p>you might want to save epochs on local disk (EBS or S3, MXnet supports both as I remember). Spot instance would be gone forever when it is turned off by the 'price cut'. One can also save the pre-processed data to disk, as well as the configured code, so one can quickly create another spot instance when the last one was killed.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 104151,
      "author_name": "godaibo",
      "author_url": "",
      "post_date": "01/09/2016 18:07:56",
      "content": "<p>Thanks for the response, it would have been nice if amazon kept the spot instance image for a day or two so users can salvage any work. I lost multiple hours of processing over a few cents price change, once bitten twice shy.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "104061": "Does anyone know if it is possible to recover an aws spot instance image that was terminated because of bid price? I have been training for a while and all that might be lost.",
    "104067": "If it was stopped, you can restart it, but if it's been terminated the instance data is gone and cannot be recovered.\r\n\r\nhttp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/terminating-instances.html",
    "104147": "you might want to save epochs on local disk (EBS or S3, MXnet supports both as I remember). Spot instance would be gone forever when it is turned off by the 'price cut'. One can also save the pre-processed data to disk, as well as the configured code, so one can quickly create another spot instance when the last one was killed.",
    "104151": "Thanks for the response, it would have been nice if amazon kept the spot instance image for a day or two so users can salvage any work. I lost multiple hours of processing over a few cents price change, once bitten twice shy."
  },
  "source": "meta"
}