{
  "id": 21238,
  "title": "Keras starter: Queries",
  "url": "/competitions/state-farm-distracted-driver-detection/discussion/21238",
  "author_name": "",
  "post_date": "2016-05-26T09:38:06.107Z",
  "votes": 1,
  "comment_count": 1,
  "views": 558,
  "content": "<p>Hi,</p>\n\n<p>I am trying to learn from ZFTurbo's Keras starter script. I am a non-CS background DS. I just don't get why the <strong>cache_data</strong>, <strong>restore_data</strong>, <strong>save_model</strong>, <strong>restore_model</strong> functions are there for.</p>\n\n<p>I am going sequentially understanding each and every line of code from the top but I am stuck at these 4 functions. Can't make sense of them at all. Specially when <strong>cache_path</strong> and <strong>if not</strong> condition is used in the function <strong>read_and_normalize_train_data</strong></p>\n\n<p>Sorry if the answer to this one is too trivial, but I need to know this. Can anybody explain their role in the script? And if possible, give an analogy of this script with the traditional non deep learning scripts too.</p>\n\n<p>Thanks,</p>\n\n<p>-CG</p>",
  "messages": [
    {
      "id": "121438",
      "postDate": "05/26/2016 09:38:06",
      "content": "<p>Hi,</p>\n\n<p>I am trying to learn from ZFTurbo's Keras starter script. I am a non-CS background DS. I just don't get why the <strong>cache_data</strong>, <strong>restore_data</strong>, <strong>save_model</strong>, <strong>restore_model</strong> functions are there for.</p>\n\n<p>I am going sequentially understanding each and every line of code from the top but I am stuck at these 4 functions. Can't make sense of them at all. Specially when <strong>cache_path</strong> and <strong>if not</strong> condition is used in the function <strong>read_and_normalize_train_data</strong></p>\n\n<p>Sorry if the answer to this one is too trivial, but I need to know this. Can anybody explain their role in the script? And if possible, give an analogy of this script with the traditional non deep learning scripts too.</p>\n\n<p>Thanks,</p>\n\n<p>-CG</p>",
      "rawMarkdown": "Hi,\r\n\r\nI am trying to learn from ZFTurbo's Keras starter script. I am a non-CS background DS. I just don't get why the **cache_data**, **restore_data**, **save_model**, **restore_model** functions are there for.\r\n\r\nI am going sequentially understanding each and every line of code from the top but I am stuck at these 4 functions. Can't make sense of them at all. Specially when **cache_path** and **if not** condition is used in the function **read_and_normalize_train_data**\r\n\r\nSorry if the answer to this one is too trivial, but I need to know this. Can anybody explain their role in the script? And if possible, give an analogy of this script with the traditional non deep learning scripts too.\r\n\r\nThanks,\r\n\r\n-CG",
      "votes": null
    },
    {
      "id": "121475",
      "postDate": "05/26/2016 18:25:23",
      "content": "<p>Caching and restoring data is there because it takes a long time to read and process the entire training and test dataset.  One doesn't want to have to re-do that every time one makes a model change or encounters a small bug or typo.  Not only is it faster and more efficient to read from one binary file for a hard disk, but we're resizing the images to a smaller size.</p>\n\n<pre><code>if not os.path.isfile(cache_path)\n</code></pre>\n\n<p>is checking if the cache_path is a file.  If it's not a file, then you can't load from the cache and need to create it.  </p>\n\n<p>save_model and restore_model are there since you don't want to keep all the models you train in memory. Say, for example, running inference off of a bunch of models you've trained on the past and ensembling them, or fine tuning them.</p>",
      "rawMarkdown": "Caching and restoring data is there because it takes a long time to read and process the entire training and test dataset.  One doesn't want to have to re-do that every time one makes a model change or encounters a small bug or typo.  Not only is it faster and more efficient to read from one binary file for a hard disk, but we're resizing the images to a smaller size.\r\n\r\n    if not os.path.isfile(cache_path)\r\n\r\nis checking if the cache_path is a file.  If it's not a file, then you can't load from the cache and need to create it.  \r\n\r\nsave_model and restore_model are there since you don't want to keep all the models you train in memory. Say, for example, running inference off of a bunch of models you've trained on the past and ensembling them, or fine tuning them.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 121475,
      "author_name": "jonathankchang",
      "author_url": "",
      "post_date": "05/26/2016 18:25:23",
      "content": "<p>Caching and restoring data is there because it takes a long time to read and process the entire training and test dataset.  One doesn't want to have to re-do that every time one makes a model change or encounters a small bug or typo.  Not only is it faster and more efficient to read from one binary file for a hard disk, but we're resizing the images to a smaller size.</p>\n\n<pre><code>if not os.path.isfile(cache_path)\n</code></pre>\n\n<p>is checking if the cache_path is a file.  If it's not a file, then you can't load from the cache and need to create it.  </p>\n\n<p>save_model and restore_model are there since you don't want to keep all the models you train in memory. Say, for example, running inference off of a bunch of models you've trained on the past and ensembling them, or fine tuning them.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "121438": "Hi,\r\n\r\nI am trying to learn from ZFTurbo's Keras starter script. I am a non-CS background DS. I just don't get why the **cache_data**, **restore_data**, **save_model**, **restore_model** functions are there for.\r\n\r\nI am going sequentially understanding each and every line of code from the top but I am stuck at these 4 functions. Can't make sense of them at all. Specially when **cache_path** and **if not** condition is used in the function **read_and_normalize_train_data**\r\n\r\nSorry if the answer to this one is too trivial, but I need to know this. Can anybody explain their role in the script? And if possible, give an analogy of this script with the traditional non deep learning scripts too.\r\n\r\nThanks,\r\n\r\n-CG",
    "121475": "Caching and restoring data is there because it takes a long time to read and process the entire training and test dataset.  One doesn't want to have to re-do that every time one makes a model change or encounters a small bug or typo.  Not only is it faster and more efficient to read from one binary file for a hard disk, but we're resizing the images to a smaller size.\r\n\r\n    if not os.path.isfile(cache_path)\r\n\r\nis checking if the cache_path is a file.  If it's not a file, then you can't load from the cache and need to create it.  \r\n\r\nsave_model and restore_model are there since you don't want to keep all the models you train in memory. Say, for example, running inference off of a bunch of models you've trained on the past and ensembling them, or fine tuning them."
  },
  "source": "meta"
}