{
  "id": 43520,
  "title": "Hi from resident Speechgler!",
  "url": "/competitions/tensorflow-speech-recognition-challenge/discussion/43520",
  "author_name": "",
  "post_date": "2017-11-15T21:02:43.508883200Z",
  "votes": 18,
  "comment_count": 10,
  "views": 0,
  "content": "<p>First of all thanks everyone participating in this challenge!</p>\n\n<p>I'm a member of Google's Speech group developing \"ok/hey Google\" across devices, so I'm very excited on what the community will be able to develop for this challenge. I put together a simple and relatively small neural network (SVDF) on Pete's audio recognition tutorial (<a href=\"https://www.tensorflow.org/tutorials/audio_recognition\">https://www.tensorflow.org/tutorials/audio_recognition</a>) and I'm sure you'll be able to do much better than that!</p>\n\n<p>In general I work with a lot of machine learning on very constrained mobile and embedded devices, and as an area that is experiencing an explosion of interest I want to encourage you to think about efficiency as a significant variable when thinking of ML applications.</p>\n\n<p>I'll be around to answer your questions, and see your progress. Good luck all!</p>",
  "messages": [
    {
      "id": "244227",
      "postDate": "11/15/2017 21:02:43",
      "content": "<p>First of all thanks everyone participating in this challenge!</p>\n\n<p>I'm a member of Google's Speech group developing \"ok/hey Google\" across devices, so I'm very excited on what the community will be able to develop for this challenge. I put together a simple and relatively small neural network (SVDF) on Pete's audio recognition tutorial (<a href=\"https://www.tensorflow.org/tutorials/audio_recognition\">https://www.tensorflow.org/tutorials/audio_recognition</a>) and I'm sure you'll be able to do much better than that!</p>\n\n<p>In general I work with a lot of machine learning on very constrained mobile and embedded devices, and as an area that is experiencing an explosion of interest I want to encourage you to think about efficiency as a significant variable when thinking of ML applications.</p>\n\n<p>I'll be around to answer your questions, and see your progress. Good luck all!</p>",
      "rawMarkdown": "First of all thanks everyone participating in this challenge!\n\nI'm a member of Google's Speech group developing \"ok/hey Google\" across devices, so I'm very excited on what the community will be able to develop for this challenge. I put together a simple and relatively small neural network (SVDF) on Pete's audio recognition tutorial (https://www.tensorflow.org/tutorials/audio_recognition) and I'm sure you'll be able to do much better than that!\n\nIn general I work with a lot of machine learning on very constrained mobile and embedded devices, and as an area that is experiencing an explosion of interest I want to encourage you to think about efficiency as a significant variable when thinking of ML applications.\n\nI'll be around to answer your questions, and see your progress. Good luck all!",
      "votes": null
    },
    {
      "id": "244869",
      "postDate": "11/17/2017 03:49:02",
      "content": "<p>Awesome thx!, I am already using it!. \nNot a long time ago, I had less than 4GB of RAM to do all my ML so I am all for efficiency.</p>\n\n<p>However... the way Kaggle works, does not leave much room for efficiency optimization, a single super fast model scoring 0.98 is useless here if a 500+ model ensemble score 0.981.</p>\n\n<p>Hopefully one day Kaggle consider(hard task) model complexity/efficiency vs prediction performance </p>",
      "rawMarkdown": "Awesome thx!, I am already using it!. \nNot a long time ago, I had less than 4GB of RAM to do all my ML so I am all for efficiency.\n\nHowever... the way Kaggle works, does not leave much room for efficiency optimization, a single super fast model scoring 0.98 is useless here if a 500+ model ensemble score 0.981.\n\nHopefully one day Kaggle consider(hard task) model complexity/efficiency vs prediction performance",
      "votes": null
    },
    {
      "id": "244879",
      "postDate": "11/17/2017 04:18:33",
      "content": "<blockquote>\n  <p>Hopefully one day Kaggle consider(hard task) model\n  complexity/efficiency vs prediction performance</p>\n</blockquote>\n\n<p>This is actually what I'm hoping the Raspberry Pi special prize will do. As you say, it's a hard task and I'm not sure we've got it perfected yet, but I'm hoping that the requirements on latency and file size will give an advantage to efficient models. Is that the sort of thing you were thinking of?</p>",
      "rawMarkdown": "&gt; Hopefully one day Kaggle consider(hard task) model\n&gt; complexity/efficiency vs prediction performance\n\nThis is actually what I'm hoping the Raspberry Pi special prize will do. As you say, it's a hard task and I'm not sure we've got it perfected yet, but I'm hoping that the requirements on latency and file size will give an advantage to efficient models. Is that the sort of thing you were thinking of?",
      "votes": null
    },
    {
      "id": "244887",
      "postDate": "11/17/2017 04:40:54",
      "content": "<p>That is a great progress towards an overall solution!.</p>\n\n<p>And yes indeed, that was exactly my thinking, setting real-life constraints to solutions would end with the fight for 0.000001 log-loss (overfit) improvements and move effort for things that really matter.</p>",
      "rawMarkdown": "That is a great progress towards an overall solution!.\n\nAnd yes indeed, that was exactly my thinking, setting real-life constraints to solutions would end with the fight for 0.000001 log-loss (overfit) improvements and move effort for things that really matter.",
      "votes": null
    },
    {
      "id": "244889",
      "postDate": "11/17/2017 04:42:32",
      "content": "<p>On the Kaggle side - I'll note that we're excited about competitions like this for the same reason! We get requests for efficiency-constrained competitions often, but it's not something we fully have the ability to support at the moment. But we're glad we're able to make advances! </p>",
      "rawMarkdown": "On the Kaggle side - I'll note that we're excited about competitions like this for the same reason! We get requests for efficiency-constrained competitions often, but it's not something we fully have the ability to support at the moment. But we're glad we're able to make advances!",
      "votes": null
    },
    {
      "id": "244924",
      "postDate": "11/17/2017 06:51:57",
      "content": "<p>Speaking of, any plans to do another kernel competition soon?</p>",
      "rawMarkdown": "Speaking of, any plans to do another kernel competition soon?",
      "votes": null
    },
    {
      "id": "244926",
      "postDate": "11/17/2017 06:54:18",
      "content": "<p>Keep your eyes peeled within the next few days ;)</p>",
      "rawMarkdown": "Keep your eyes peeled within the next few days ;)",
      "votes": null
    },
    {
      "id": "246058",
      "postDate": "11/20/2017 13:19:10",
      "content": "<p>I have a Kaggle competition idea. A Kaggle competition to identify most efficient solution for Kaggle competition. :)</p>",
      "rawMarkdown": "I have a Kaggle competition idea. A Kaggle competition to identify most efficient solution for Kaggle competition. :)",
      "votes": null
    },
    {
      "id": "247273",
      "postDate": "11/22/2017 17:45:17",
      "content": "<p>Hey Raziel,</p>\n\n<p>I am interested in learning more about machine learning in constrained environments in general. Are there any papers or resources you would recommend for exploring this?</p>",
      "rawMarkdown": "Hey Raziel,\n\nI am interested in learning more about machine learning in constrained environments in general. Are there any papers or resources you would recommend for exploring this?",
      "votes": null
    },
    {
      "id": "247292",
      "postDate": "11/22/2017 18:28:09",
      "content": "<p>Hi Tyler.\nIn general anything that aims to reduce parameter count, or the bit-size of those parameters (e.g. quantization from floating point to lower integer representations) are good ways to start, and there's plenty of papers being constantly published. I think our paper of an entirely embedded speech recognizer gives a good idea of the challenges and some solutions (<a href=\"https://arxiv.org/abs/1603.03185\">https://arxiv.org/abs/1603.03185</a>, <a href=\"https://arxiv.org/abs/1607.04683\">https://arxiv.org/abs/1607.04683</a>). There's also going to be an specialized workshop at this years NIPS (<a href=\"https://nips.cc/Conferences/2017/Schedule?showEvent=8791\">https://nips.cc/Conferences/2017/Schedule?showEvent=8791</a>), and there's been some related ones in other past conferences --the field is picking up a lot of momentum.</p>",
      "rawMarkdown": "Hi Tyler.\nIn general anything that aims to reduce parameter count, or the bit-size of those parameters (e.g. quantization from floating point to lower integer representations) are good ways to start, and there's plenty of papers being constantly published. I think our paper of an entirely embedded speech recognizer gives a good idea of the challenges and some solutions (https://arxiv.org/abs/1603.03185, https://arxiv.org/abs/1607.04683). There's also going to be an specialized workshop at this years NIPS (https://nips.cc/Conferences/2017/Schedule?showEvent=8791), and there's been some related ones in other past conferences --the field is picking up a lot of momentum.",
      "votes": null
    },
    {
      "id": "256084",
      "postDate": "12/11/2017 03:50:28",
      "content": "<p>How about a competition to predict the final rank of users based on how their rank goes up/down?</p>",
      "rawMarkdown": "How about a competition to predict the final rank of users based on how their rank goes up/down?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 244869,
      "author_name": "carloshuertas",
      "author_url": "",
      "post_date": "11/17/2017 03:49:02",
      "content": "<p>Awesome thx!, I am already using it!. \nNot a long time ago, I had less than 4GB of RAM to do all my ML so I am all for efficiency.</p>\n\n<p>However... the way Kaggle works, does not leave much room for efficiency optimization, a single super fast model scoring 0.98 is useless here if a 500+ model ensemble score 0.981.</p>\n\n<p>Hopefully one day Kaggle consider(hard task) model complexity/efficiency vs prediction performance </p>",
      "votes": null,
      "replies": [
        {
          "id": 244879,
          "author_name": "petewarden",
          "author_url": "",
          "post_date": "11/17/2017 04:18:33",
          "content": "<blockquote>\n  <p>Hopefully one day Kaggle consider(hard task) model\n  complexity/efficiency vs prediction performance</p>\n</blockquote>\n\n<p>This is actually what I'm hoping the Raspberry Pi special prize will do. As you say, it's a hard task and I'm not sure we've got it perfected yet, but I'm hoping that the requirements on latency and file size will give an advantage to efficient models. Is that the sort of thing you were thinking of?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 244887,
          "author_name": "carloshuertas",
          "author_url": "",
          "post_date": "11/17/2017 04:40:54",
          "content": "<p>That is a great progress towards an overall solution!.</p>\n\n<p>And yes indeed, that was exactly my thinking, setting real-life constraints to solutions would end with the fight for 0.000001 log-loss (overfit) improvements and move effort for things that really matter.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 244889,
          "author_name": "addisonhoward",
          "author_url": "",
          "post_date": "11/17/2017 04:42:32",
          "content": "<p>On the Kaggle side - I'll note that we're excited about competitions like this for the same reason! We get requests for efficiency-constrained competitions often, but it's not something we fully have the ability to support at the moment. But we're glad we're able to make advances! </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 244924,
          "author_name": "happycube",
          "author_url": "",
          "post_date": "11/17/2017 06:51:57",
          "content": "<p>Speaking of, any plans to do another kernel competition soon?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 244926,
          "author_name": "addisonhoward",
          "author_url": "",
          "post_date": "11/17/2017 06:54:18",
          "content": "<p>Keep your eyes peeled within the next few days ;)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 246058,
      "author_name": "bsp2020",
      "author_url": "",
      "post_date": "11/20/2017 13:19:10",
      "content": "<p>I have a Kaggle competition idea. A Kaggle competition to identify most efficient solution for Kaggle competition. :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 256084,
          "author_name": "parthsuresh",
          "author_url": "",
          "post_date": "12/11/2017 03:50:28",
          "content": "<p>How about a competition to predict the final rank of users based on how their rank goes up/down?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 247273,
      "author_name": "trome17",
      "author_url": "",
      "post_date": "11/22/2017 17:45:17",
      "content": "<p>Hey Raziel,</p>\n\n<p>I am interested in learning more about machine learning in constrained environments in general. Are there any papers or resources you would recommend for exploring this?</p>",
      "votes": null,
      "replies": [
        {
          "id": 247292,
          "author_name": "razielalvarez",
          "author_url": "",
          "post_date": "11/22/2017 18:28:09",
          "content": "<p>Hi Tyler.\nIn general anything that aims to reduce parameter count, or the bit-size of those parameters (e.g. quantization from floating point to lower integer representations) are good ways to start, and there's plenty of papers being constantly published. I think our paper of an entirely embedded speech recognizer gives a good idea of the challenges and some solutions (<a href=\"https://arxiv.org/abs/1603.03185\">https://arxiv.org/abs/1603.03185</a>, <a href=\"https://arxiv.org/abs/1607.04683\">https://arxiv.org/abs/1607.04683</a>). There's also going to be an specialized workshop at this years NIPS (<a href=\"https://nips.cc/Conferences/2017/Schedule?showEvent=8791\">https://nips.cc/Conferences/2017/Schedule?showEvent=8791</a>), and there's been some related ones in other past conferences --the field is picking up a lot of momentum.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "244227": "First of all thanks everyone participating in this challenge!\n\nI'm a member of Google's Speech group developing \"ok/hey Google\" across devices, so I'm very excited on what the community will be able to develop for this challenge. I put together a simple and relatively small neural network (SVDF) on Pete's audio recognition tutorial (https://www.tensorflow.org/tutorials/audio_recognition) and I'm sure you'll be able to do much better than that!\n\nIn general I work with a lot of machine learning on very constrained mobile and embedded devices, and as an area that is experiencing an explosion of interest I want to encourage you to think about efficiency as a significant variable when thinking of ML applications.\n\nI'll be around to answer your questions, and see your progress. Good luck all!",
    "244869": "Awesome thx!, I am already using it!. \nNot a long time ago, I had less than 4GB of RAM to do all my ML so I am all for efficiency.\n\nHowever... the way Kaggle works, does not leave much room for efficiency optimization, a single super fast model scoring 0.98 is useless here if a 500+ model ensemble score 0.981.\n\nHopefully one day Kaggle consider(hard task) model complexity/efficiency vs prediction performance",
    "244879": "&gt; Hopefully one day Kaggle consider(hard task) model\n&gt; complexity/efficiency vs prediction performance\n\nThis is actually what I'm hoping the Raspberry Pi special prize will do. As you say, it's a hard task and I'm not sure we've got it perfected yet, but I'm hoping that the requirements on latency and file size will give an advantage to efficient models. Is that the sort of thing you were thinking of?",
    "244887": "That is a great progress towards an overall solution!.\n\nAnd yes indeed, that was exactly my thinking, setting real-life constraints to solutions would end with the fight for 0.000001 log-loss (overfit) improvements and move effort for things that really matter.",
    "244889": "On the Kaggle side - I'll note that we're excited about competitions like this for the same reason! We get requests for efficiency-constrained competitions often, but it's not something we fully have the ability to support at the moment. But we're glad we're able to make advances!",
    "244924": "Speaking of, any plans to do another kernel competition soon?",
    "244926": "Keep your eyes peeled within the next few days ;)",
    "246058": "I have a Kaggle competition idea. A Kaggle competition to identify most efficient solution for Kaggle competition. :)",
    "247273": "Hey Raziel,\n\nI am interested in learning more about machine learning in constrained environments in general. Are there any papers or resources you would recommend for exploring this?",
    "247292": "Hi Tyler.\nIn general anything that aims to reduce parameter count, or the bit-size of those parameters (e.g. quantization from floating point to lower integer representations) are good ways to start, and there's plenty of papers being constantly published. I think our paper of an entirely embedded speech recognizer gives a good idea of the challenges and some solutions (https://arxiv.org/abs/1603.03185, https://arxiv.org/abs/1607.04683). There's also going to be an specialized workshop at this years NIPS (https://nips.cc/Conferences/2017/Schedule?showEvent=8791), and there's been some related ones in other past conferences --the field is picking up a lot of momentum.",
    "256084": "How about a competition to predict the final rank of users based on how their rank goes up/down?"
  },
  "source": "meta"
}