{
  "id": 46880,
  "title": "Recipe for running on Google Cloud Platform / ML Engine",
  "url": "/competitions/tensorflow-speech-recognition-challenge/discussion/46880",
  "author_name": "Andre Holzner",
  "post_date": "2018-01-04T12:42:11.395000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hello,</p>\n\n<p>a happy new year to all !</p>\n\n<p>Thanks to the credit provided @Julia Elliot and Google Cloud Platform, I tried to get @Alex Ozerin's kernel  (<a href=\"https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72\">https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72</a>) working on Google's ML Engine.</p>\n\n<p>For those who want to try it, the recipe is here: <a href=\"https://www.kaggle.com/holzner/recipe-for-training-on-google-ml-engine\">https://www.kaggle.com/holzner/recipe-for-training-on-google-ml-engine</a> (use at your own risk).</p>\n\n<p>Admittedly, the recipe looks quite lengthy. Part of the complication comes from the fact that I decided to pack the entire train and test dataset into single npy files (according to my experience,  training is about a factor 30 faster when reading from a few files only from Google cloud storage buckets rather than reading from ~ 65k wav files directly). </p>\n\n<p>Another source of slight complication comes from the fact that I did not want to rely on uploading the train and test from my local machine back to the Google cloud via my 1 MBit/s upload line.</p>\n\n<p>I typically see the train job exectue on a K80 GPU. With the default parameters from Alex' kernel training and test set prediction takes about 41 minutes of running time (after a few minutes waiting for the job to be dispatched).</p>\n\n<p>Let me know if you find any problems or mistakes in the recipe.</p>",
  "messages": [
    {
      "id": 265036,
      "postDate": "2018-01-04T12:42:11.397Z",
      "content": "<p>Hello,</p>\n\n<p>a happy new year to all !</p>\n\n<p>Thanks to the credit provided @Julia Elliot and Google Cloud Platform, I tried to get @Alex Ozerin's kernel  (<a href=\"https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72\">https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72</a>) working on Google's ML Engine.</p>\n\n<p>For those who want to try it, the recipe is here: <a href=\"https://www.kaggle.com/holzner/recipe-for-training-on-google-ml-engine\">https://www.kaggle.com/holzner/recipe-for-training-on-google-ml-engine</a> (use at your own risk).</p>\n\n<p>Admittedly, the recipe looks quite lengthy. Part of the complication comes from the fact that I decided to pack the entire train and test dataset into single npy files (according to my experience,  training is about a factor 30 faster when reading from a few files only from Google cloud storage buckets rather than reading from ~ 65k wav files directly). </p>\n\n<p>Another source of slight complication comes from the fact that I did not want to rely on uploading the train and test from my local machine back to the Google cloud via my 1 MBit/s upload line.</p>\n\n<p>I typically see the train job exectue on a K80 GPU. With the default parameters from Alex' kernel training and test set prediction takes about 41 minutes of running time (after a few minutes waiting for the job to be dispatched).</p>\n\n<p>Let me know if you find any problems or mistakes in the recipe.</p>",
      "rawMarkdown": "Hello,\n\na happy new year to all !\n\nThanks to the credit provided @Julia Elliot and Google Cloud Platform, I tried to get @Alex Ozerin's kernel  (https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72) working on Google's ML Engine.\n\nFor those who want to try it, the recipe is here: https://www.kaggle.com/holzner/recipe-for-training-on-google-ml-engine (use at your own risk).\n\nAdmittedly, the recipe looks quite lengthy. Part of the complication comes from the fact that I decided to pack the entire train and test dataset into single npy files (according to my experience,  training is about a factor 30 faster when reading from a few files only from Google cloud storage buckets rather than reading from ~ 65k wav files directly). \n\nAnother source of slight complication comes from the fact that I did not want to rely on uploading the train and test from my local machine back to the Google cloud via my 1 MBit/s upload line.\n\nI typically see the train job exectue on a K80 GPU. With the default parameters from Alex' kernel training and test set prediction takes about 41 minutes of running time (after a few minutes waiting for the job to be dispatched).\n\nLet me know if you find any problems or mistakes in the recipe.",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "265036": "Hello,\n\na happy new year to all !\n\nThanks to the credit provided @Julia Elliot and Google Cloud Platform, I tried to get @Alex Ozerin's kernel  (https://www.kaggle.com/alexozerin/end-to-end-baseline-tf-estimator-lb-0-72) working on Google's ML Engine.\n\nFor those who want to try it, the recipe is here: https://www.kaggle.com/holzner/recipe-for-training-on-google-ml-engine (use at your own risk).\n\nAdmittedly, the recipe looks quite lengthy. Part of the complication comes from the fact that I decided to pack the entire train and test dataset into single npy files (according to my experience,  training is about a factor 30 faster when reading from a few files only from Google cloud storage buckets rather than reading from ~ 65k wav files directly). \n\nAnother source of slight complication comes from the fact that I did not want to rely on uploading the train and test from my local machine back to the Google cloud via my 1 MBit/s upload line.\n\nI typically see the train job exectue on a K80 GPU. With the default parameters from Alex' kernel training and test set prediction takes about 41 minutes of running time (after a few minutes waiting for the job to be dispatched).\n\nLet me know if you find any problems or mistakes in the recipe."
  }
}