Welcome and thanks to everyone who's looking into this competition! I'm a long-time Kaggle fan, and actually ran a contest six years ago that I really learned a lot from, so I'm glad to have another chance to work with the awesome people here: https://www.kaggle.com/c/PhotoQualityPrediction#description
\n\nI'm also the tech lead of the mobile and embedded side of TensorFlow, and I'm particularly excited because I'm hoping the results will directly feed back into the community through TensorFlow open source contributions, and help people hack Raspberry Pis to do even more interesting things.
\n\nIf you'd like a quick way to get started, I have a tutorial on how to use TensorFlow to train audio models on the training set here: https://www.tensorflow.org/tutorials/audio_recognition
\n\nMy best model there achieves around 88% accuracy (using the script's metrics), but I'm confident that the community here can do much better. I will be hanging out here on the forums to try to answer questions, though I do have a vacation planned over Thanksgiving (next week and the week after), so my responses may be a bit delayed then. I'm also active on Twitter at https://twitter.com/petewarden, though these forums are probably the best place to ask questions so the knowledge is easy to find for everyone. Please do ping me with anything you'd like more information on.
\n\nAs a final note, we're still actively trying to build out the underlying open data set, so please do go to https://aiyprojects.withgoogle.com/open_speech_recording if you can and share your voice for future releases!
", "messages": [ { "id": 244194, "postDate": "2017-11-15T19:20:52.427Z", "content": "Welcome and thanks to everyone who's looking into this competition! I'm a long-time Kaggle fan, and actually ran a contest six years ago that I really learned a lot from, so I'm glad to have another chance to work with the awesome people here: https://www.kaggle.com/c/PhotoQualityPrediction#description
\n\nI'm also the tech lead of the mobile and embedded side of TensorFlow, and I'm particularly excited because I'm hoping the results will directly feed back into the community through TensorFlow open source contributions, and help people hack Raspberry Pis to do even more interesting things.
\n\nIf you'd like a quick way to get started, I have a tutorial on how to use TensorFlow to train audio models on the training set here: https://www.tensorflow.org/tutorials/audio_recognition
\n\nMy best model there achieves around 88% accuracy (using the script's metrics), but I'm confident that the community here can do much better. I will be hanging out here on the forums to try to answer questions, though I do have a vacation planned over Thanksgiving (next week and the week after), so my responses may be a bit delayed then. I'm also active on Twitter at https://twitter.com/petewarden, though these forums are probably the best place to ask questions so the knowledge is easy to find for everyone. Please do ping me with anything you'd like more information on.
\n\nAs a final note, we're still actively trying to build out the underlying open data set, so please do go to https://aiyprojects.withgoogle.com/open_speech_recording if you can and share your voice for future releases!
", "rawMarkdown": "Welcome and thanks to everyone who's looking into this competition! I'm a long-time Kaggle fan, and actually ran a contest six years ago that I really learned a lot from, so I'm glad to have another chance to work with the awesome people here: https://www.kaggle.com/c/PhotoQualityPrediction#description\n\nI'm also the tech lead of the mobile and embedded side of TensorFlow, and I'm particularly excited because I'm hoping the results will directly feed back into the community through TensorFlow open source contributions, and help people hack Raspberry Pis to do even more interesting things. \n\nIf you'd like a quick way to get started, I have a tutorial on how to use TensorFlow to train audio models on the training set here: https://www.tensorflow.org/tutorials/audio_recognition\n\nMy best model there achieves around 88% accuracy (using the script's metrics), but I'm confident that the community here can do much better. I will be hanging out here on the forums to try to answer questions, though I do have a vacation planned over Thanksgiving (next week and the week after), so my responses may be a bit delayed then. I'm also active on Twitter at https://twitter.com/petewarden, though these forums are probably the best place to ask questions so the knowledge is easy to find for everyone. Please do ping me with anything you'd like more information on.\n\nAs a final note, we're still actively trying to build out the underlying open data set, so please do go to https://aiyprojects.withgoogle.com/open_speech_recording if you can and share your voice for future releases!", "votes": 51 }, { "id": 253197, "postDate": "2017-12-04T15:49:49.897Z", "content": "Hi @Pete. Have you tried your solution on the test set?
\n\nIf this is your model, I used the example scripts and got ~ 89% of validation accuracy after training, but only 78% accuracy on the Kaggle LB. So it's probably overfitting the training/validation data.
\n\nI'm posting this because we are seeing such great solutions on the leaderboard and your initial post saying that you had an ~ 88% accuracy model could confuse someone.
", "rawMarkdown": "Hi @Pete. Have you tried your solution on the test set?\n\nIf this is your [model][1], I used the example scripts and got ~ 89% of validation accuracy after training, but only 78% accuracy on the Kaggle LB. So it's probably overfitting the training/validation data.\n\nI'm posting this because we are seeing such great solutions on the leaderboard and your initial post saying that you had an ~ 88% accuracy model could confuse someone.\n\n [1]: https://github.com/tensorflow/tensorflow/tree/v1.4.0/tensorflow/examples/speech_commands", "votes": 3, "replies": [ { "id": 253776, "postDate": "2017-12-05T16:36:28.553Z", "content": "That is a good point Rafael, thanks for the question. I do see the same results using the pretrained model (89% on the training scripts, 78% on the leaderboard). I'm hopeful that this is due to the different distributions of categories of data that are measured to produce the average accuracy, rather than overfitting, but I haven't investigated any deeper.
\n\nHopefully that's enough to reassure anybody using the example as a starting point that they're not doing anything wrong though? I'm excited to see we have big improvements over that on the LB already!
", "rawMarkdown": "That is a good point Rafael, thanks for the question. I do see the same results using the pretrained model (89% on the training scripts, 78% on the leaderboard). I'm hopeful that this is due to the different distributions of categories of data that are measured to produce the average accuracy, rather than overfitting, but I haven't investigated any deeper. \n\nHopefully that's enough to reassure anybody using the example as a starting point that they're not doing anything wrong though? I'm excited to see we have big improvements over that on the LB already!", "votes": 2 }, { "id": 253794, "postDate": "2017-12-05T17:00:23.457Z", "content": "Your model is definitely very useful for anyone working in this competition.
\n\nI've run some tests and I believe the example doesn't generalise well to the test set because the data used by the example model (~ 65k samples) probably have a different distribution regarding the real test set of this competition (~ 150k samples). That would explain why the example is not overfitting in the validation set, but doesn't achieve a higher accuracy in the competition.
\n\nI'll start to work on data augmentation (like pitch shifting) because this is probably going to lead to better results than tweaking the network architecture.
\n\nUsing Squeezenet, I got ~ 83% on Kaggle Leaderboard after ~ 300 epochs.\nUsing Inception Resnet V2, I got ~ 84% accuracy on Kaggle Leaderboard after ~ 10 epochs.
\n\nGood luck for everyone.
", "rawMarkdown": "Your model is definitely very useful for anyone working in this competition.\n\nI've run some tests and I believe the example doesn't generalise well to the test set because the data used by the example model (~ 65k samples) probably have a different distribution regarding the real test set of this competition (~ 150k samples). That would explain why the example is not overfitting in the validation set, but doesn't achieve a higher accuracy in the competition.\n\nI'll start to work on data augmentation (like pitch shifting) because this is probably going to lead to better results than tweaking the network architecture.\n\nUsing Squeezenet, I got ~ 83% on Kaggle Leaderboard after ~ 300 epochs.\nUsing Inception Resnet V2, I got ~ 84% accuracy on Kaggle Leaderboard after ~ 10 epochs.\n\nGood luck for everyone.", "votes": 5 } ] }, { "id": 244802, "postDate": "2017-11-17T00:41:23.840Z", "content": "Hi Pete,
\n\nI have a quick question: is the use of pre-trained models such as VGGish:https://github.com/tensorflow/models/tree/master/research/audioset allowed? or is this considered using external data?
", "rawMarkdown": "Hi Pete,\n\nI have a quick question: is the use of pre-trained models such as VGGish:https://github.com/tensorflow/models/tree/master/research/audioset allowed? or is this considered using external data?", "votes": 1, "replies": [ { "id": 244812, "postDate": "2017-11-17T01:36:03.953Z", "content": "There's another thread on this that goes into more detail, but the summary is they are considered external data (since they're trained on data outside of the released set).
", "rawMarkdown": "There's another thread on this that goes into more detail, but the summary is they are considered external data (since they're trained on data outside of the released set).", "votes": 3 } ] }, { "id": 244295, "postDate": "2017-11-16T00:17:57.090Z", "content": "Hello Pete,
\n\nFrom what I understood the \"./benchmark_model\" should be used on a raspberry, right? Is there away for people to test runtime without owning the device - perhaps some kind of an environment?
", "rawMarkdown": "Hello Pete,\n\nFrom what I understood the \"./benchmark_model\" should be used on a raspberry, right? Is there away for people to test runtime without owning the device - perhaps some kind of an environment? ", "votes": 1, "replies": [ { "id": 244310, "postDate": "2017-11-16T00:47:49.553Z", "content": "There isn't a good way to estimate the speed on a Pi without access to one unfortunately. It is possible to run the same command on a desktop x86 machine if you've created a model and get a rough idea of relative speeds (for example model v2 is twice as slow as model v1), but even that may not map accurately over to an ARM device.
", "rawMarkdown": "There isn't a good way to estimate the speed on a Pi without access to one unfortunately. It is possible to run the same command on a desktop x86 machine if you've created a model and get a rough idea of relative speeds (for example model v2 is twice as slow as model v1), but even that may not map accurately over to an ARM device.", "votes": 1 }, { "id": 244311, "postDate": "2017-11-16T00:49:25.387Z", "content": "And just to clarify, you'll have to build and run the x86 version of tensorflow/tools/benchmark:benchmark_model to run it on your desktop machine, the curl commands to download the Pi version won't work.
", "rawMarkdown": "And just to clarify, you'll have to build and run the x86 version of tensorflow/tools/benchmark:benchmark_model to run it on your desktop machine, the curl commands to download the Pi version won't work." }, { "id": 266752, "postDate": "2018-01-09T16:32:09.693Z", "content": "Hi Pete, thanks so much for hosting this competition, I'm having issues with trying to use the provided benchmark script to benchmark a model frozen using Tensorflow 1.4 and strided_slice
The error I'm getting is:
\n\nCreate kernel failed: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=<unknown>; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=\"\"; is_stateful=true>; NodeDef: cond/map/TensorArray = TensorArrayV3[clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=<unknown>, identical_element_shapes=true, tensor_array_name=\"\", _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](cond/map/strided_slice). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
Other models that I tried worked fine. Is there something simple I'm doing wrong, or is it possible the benchmark script is out of date, and if so could we get a version for TF 1.4? (having a heck of a time trying to figure out how to cross-compile the benchmark tool for the Pi)
", "rawMarkdown": "Hi Pete, thanks so much for hosting this competition, I'm having issues with trying to use the provided benchmark script to benchmark a model frozen using Tensorflow 1.4 and ````strided_slice````\n\nThe error I'm getting is:\n\n````Create kernel failed: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op", "votes": 1 }, { "id": 267627, "postDate": "2018-01-11T22:35:34.277Z", "content": "Kindly disregard, I was able to get it to run using tensorflow==1.4.0
", "rawMarkdown": "Kindly disregard, I was able to get it to run using tensorflow==1.4.0" }, { "id": 267639, "postDate": "2018-01-11T23:27:06.573Z", "content": "Thanks for the update, I'm glad you're able to run now!
", "rawMarkdown": "Thanks for the update, I'm glad you're able to run now!" }, { "id": 276322, "postDate": "2018-01-31T07:14:50.437Z", "content": "Hi Thomas, I recently got the same issue but not able to resolve it. I have trained the model in tensorflow 1.5.0-dev20171023 and when exporting the model i used 1.5.0, I am getting same error as you. How's the error got resolved ?
\n\nAny kind of help is appreciated.
", "rawMarkdown": "Hi Thomas, I recently got the same issue but not able to resolve it. I have trained the model in tensorflow 1.5.0-dev20171023 and when exporting the model i used 1.5.0, I am getting same error as you. How's the error got resolved ?\n\nAny kind of help is appreciated." } ] }, { "id": 244282, "postDate": "2017-11-15T23:35:21.720Z", "content": "thanks for this competition. good chance to learn pure Tensorflow although I am used to stick with Keras
", "rawMarkdown": "thanks for this competition. good chance to learn pure Tensorflow although I am used to stick with Keras", "votes": 1, "replies": [ { "id": 244287, "postDate": "2017-11-16T00:04:18.727Z", "content": "No problem, thanks for jumping in! I'm a fan of Keras too, though I haven't used it for audio myself.
", "rawMarkdown": "No problem, thanks for jumping in! I'm a fan of Keras too, though I haven't used it for audio myself.", "votes": 1 } ] }, { "id": 250492, "postDate": "2017-11-30T03:55:06.177Z", "content": "Raspbian has recently been updated to Debian 9 (Stretch). https://www.raspberrypi.org/downloads/raspbian/
\n\nThe rules state Raspbian 8 (Jessie) shall be used in the evaluation. Is there a link to the image that will be used for testing?
", "rawMarkdown": "Raspbian has recently been updated to Debian 9 (Stretch). https://www.raspberrypi.org/downloads/raspbian/\n\nThe rules state Raspbian 8 (Jessie) shall be used in the evaluation. Is there a link to the image that will be used for testing?", "votes": 2 }, { "id": 244216, "postDate": "2017-11-15T20:18:46.930Z", "content": "Hi Pete, thanks for hosting this competition. Sorry to ask a silly question: is the competition related to the newly released TensorFlow Lite? Didn't have much time to read that.
", "rawMarkdown": "Hi Pete, thanks for hosting this competition. Sorry to ask a silly question: is the competition related to the newly released TensorFlow Lite? Didn't have much time to read that.", "votes": 2, "replies": [ { "id": 244219, "postDate": "2017-11-15T20:35:44.910Z", "content": "Good question! I am hoping to use the resulting model in TF Lite, since we're aiming that at devices like the Pi, but that's still a little way down the road, and it isn't supported there yet.
", "rawMarkdown": "Good question! I am hoping to use the resulting model in TF Lite, since we're aiming that at devices like the Pi, but that's still a little way down the road, and it isn't supported there yet.", "votes": 3 }, { "id": 244221, "postDate": "2017-11-15T20:42:08.440Z", "content": "Good to know that!
", "rawMarkdown": "Good to know that!" } ] }, { "id": 247230, "postDate": "2017-11-22T16:28:14.937Z", "content": "hi Pete,please forgive me if these are very dump questions and correct me if i'm wrong anywhere.\nquestion1\nwhen i was installing tensorflow from source using this repotensorflow on raspberry pi 3 .i was confused on one question which is [Do you wish to build TensorFlow with GPU support? [y/N]]\nif i'm correct Tensorflow leverages Nvidia drivers to power Nvidia GPUs and Raspberry Pi 3 model Bdoes not have Nvidia hardware. it has GPU: Broadcom VideoCore IV.so technically i should say \"no\" to this question ,please give me your insights on it\nquestion2\n i'm unable to find wheel for tensorflow 1.4 version ,python 3.4 could you please guide me how i should tackle this problem
", "rawMarkdown": "hi Pete,please forgive me if these are very dump questions and correct me if i'm wrong anywhere.\nquestion1\nwhen i was installing tensorflow from source using this repo[tensorflow on raspberry pi 3][1] .i was confused on one question which is [Do you wish to build TensorFlow with GPU support? [y/N]]\nif i'm correct Tensorflow leverages Nvidia drivers to power Nvidia GPUs and Raspberry Pi 3 model Bdoes not have Nvidia hardware. it has GPU: Broadcom VideoCore IV.so technically i should say \"no\" to this question ,please give me your insights on it\nquestion2\n i'm unable to find wheel for tensorflow 1.4 version ,python 3.4 could you please guide me how i should tackle this problem\n\n\n \n\n\n [1]: https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/244bf9c48d81105b7b97b539448f2b818dfa9d91/GUIDE.md", "votes": -1 }, { "id": 261665, "postDate": "2017-12-23T13:41:19.950Z", "content": "Hi @Pete. Just a quick question. There's been a lot of mentioning in regards to using the example audio speech code as a starting point but there is a problem with that. The audio_ops are missing and this is still not fixed as far as I know. In the github issues people were talking about v1.5 of tensorflow that might resolve. I don't know if other people had the same issues and how they resolved it. Any suggestions?
", "rawMarkdown": "Hi @Pete. Just a quick question. There's been a lot of mentioning in regards to using the example audio speech code as a starting point but there is a problem with that. The audio_ops are missing and this is still not fixed as far as I know. In the github issues people were talking about v1.5 of tensorflow that might resolve. I don't know if other people had the same issues and how they resolved it. Any suggestions?", "replies": [ { "id": 267233, "postDate": "2018-01-10T22:16:53.117Z", "content": "The audio_ops worked for me. I compiled Tensorflow from source, so that may have something to do with it.
", "rawMarkdown": "The audio_ops worked for me. I compiled Tensorflow from source, so that may have something to do with it." } ] }, { "id": 261282, "postDate": "2017-12-22T06:33:57.247Z", "content": "Hi @Pete Could you please tell about VAD model https://pypi.python.org/pypi/webrtcvad. Is it allowed or not? Thanks in advance.
", "rawMarkdown": "Hi @Pete Could you please tell about VAD model https://pypi.python.org/pypi/webrtcvad. Is it allowed or not? Thanks in advance." }, { "id": 260889, "postDate": "2017-12-21T05:39:27.487Z", "content": "Hi, I have a problem with the model for Raspberry Pi 3. I can not create the inputs have the names:
\n\n\n\n\ndecoded_sample_data:0, taking a [16000, 1] float tensor as input,\n representing the audio PCM-encoded data.
\n \ndecoded_sample_data:1, taking a scalar [] int32 tensor as input,\n representing the sample rate, which must be the value 16000.
\n
I created 2 tensors as follows:
\n\n\n\n\nsample_placeholder = tf.placeholder(dtype=tf.float32,\n shape=[16000,1],name='decoded_sample_data') sr_placeholder =\n tf.placeholder(dtype=tf.int32, name='decoded_sample_data')
\n
But their name are different. How can I do it? Thank
", "rawMarkdown": "Hi, I have a problem with the model for Raspberry Pi 3. I can not create the inputs have the names:\n\n> decoded_sample_data:0, taking a [16000, 1] float tensor as input,\n> representing the audio PCM-encoded data.\n> \n> decoded_sample_data:1, taking a scalar [] int32 tensor as input,\n> representing the sample rate, which must be the value 16000.\n\nI created 2 tensors as follows:\n\n> sample_placeholder = tf.placeholder(dtype=tf.float32,\n> shape=[16000,1],name='decoded_sample_data') sr_placeholder =\n> tf.placeholder(dtype=tf.int32, name='decoded_sample_data')\n\nBut their name are different. How can I do it? Thank" }, { "id": 260485, "postDate": "2017-12-20T09:40:28.653Z", "content": "@Pete I'm no longer seeing checkpoint file (conv.ckpt-18000) being created which is needed for freeze.py . I've created an issue @ https://github.com/tensorflow/tensorflow/issues/15505, would love to hear any suggestions you may have.
Armen, the freeze script uses the prefix by convention. I see your conv.ckpt-18000 files in your directory list output posted to the GitHub issue, so the freeze script will work without any problems if you simply run \"python freeze.py --start_checkpoint=path/to/conv.ckpt-18000 --model_architecture conv --output_file=frozen_graph.pb\"
", "rawMarkdown": "Armen, the freeze script uses the prefix by convention. I see your conv.ckpt-18000 files in your directory list output posted to the GitHub issue, so the freeze script will work without any problems if you simply run \"python freeze.py --start_checkpoint=path/to/conv.ckpt-18000 --model_architecture conv --output_file=frozen_graph.pb\"", "votes": 1 }, { "id": 260679, "postDate": "2017-12-20T17:50:56.967Z", "content": "Regarding the label_wav.py, did you have to create an additional function inside the label_wav.py so you can run it within a directory and save it to CSV. For example:\npython tensorflow/examples/speech_commands/label_wav.py \\\n--graph=/tmp/my_frozen_graph.pb \\\n--labels=/tmp/speech_commands_train/conv_labels.txt \\\n--wav=/tmp/speech_dataset/*/*wav
", "rawMarkdown": "Regarding the label_wav.py, did you have to create an additional function inside the label_wav.py so you can run it within a directory and save it to CSV. For example:\npython tensorflow/examples/speech_commands/label_wav.py \\\n--graph=/tmp/my_frozen_graph.pb \\\n--labels=/tmp/speech_commands_train/conv_labels.txt \\\n--wav=/tmp/speech_dataset/*/*wav " }, { "id": 260726, "postDate": "2017-12-20T19:19:40.773Z", "content": "Tasos, that is one option, but I imported the functions into a different script in order to use the Pandas library to plot some visualizations of the predictions before saving them to CSV. The label_wav.py script is very slow because it reloads your model for one wav file. It's better to load your model once and then run a batch of predictions.
", "rawMarkdown": "Tasos, that is one option, but I imported the functions into a different script in order to use the Pandas library to plot some visualizations of the predictions before saving them to CSV. The label_wav.py script is very slow because it reloads your model for one wav file. It's better to load your model once and then run a batch of predictions." }, { "id": 260758, "postDate": "2017-12-20T21:11:52.823Z", "content": "@Tasos yes, I added a function to output a submission file as well as predict probabilities for all classes to be used later for stacking.
\n\nAdditionally, I've added a new clause to create_model in models.py, looking for the name of my architecture and then calling a model creation function. I've found this demo app to be very useful in terms of a pluggable pipeline to experiment with ctc, lstm, and plain ol 2d cnns.
", "rawMarkdown": "@Tasos yes, I added a function to output a submission file as well as predict probabilities for all classes to be used later for stacking.\n\nAdditionally, I've added a new clause to create_model in models.py, looking for the name of my architecture and then calling a model creation function. I've found this demo app to be very useful in terms of a pluggable pipeline to experiment with ctc, lstm, and plain ol 2d cnns." } ] }, { "id": 258076, "postDate": "2017-12-15T11:57:41.870Z", "content": "Hi Pete
\n\nThanks for the competition, it is my first kaggle project and also my first ML project. I got a 0.84, and I still want to move on. But I have a little confusion.
\n\nI found that the test audios may contain words like \"learn\" and \"visual\", which are predicted by my model as \"left\" and \"zero\". I'm not sure if it is my mistake, or these words do exist and should be predicted as \"unknown\".
\n\nExpecting a reply, so I can find the right direction to optimize my model.
", "rawMarkdown": "Hi Pete\n\nThanks for the competition, it is my first kaggle project and also my first ML project. I got a 0.84, and I still want to move on. But I have a little confusion.\n\nI found that the test audios may contain words like \"learn\" and \"visual\", which are predicted by my model as \"left\" and \"zero\". I'm not sure if it is my mistake, or these words do exist and should be predicted as \"unknown\".\n\n Expecting a reply, so I can find the right direction to optimize my model.\n", "replies": [ { "id": 258141, "postDate": "2017-12-15T15:40:50.073Z", "content": "The words you mention do exist in the test set and should be predicted as unknown.
", "rawMarkdown": "The words you mention do exist in the test set and should be predicted as unknown." }, { "id": 258388, "postDate": "2017-12-16T02:42:15.410Z", "content": "Thank you, James.
\n\nSo the range of the test words is bigger than the train words. I think I can conclude that a model trained by the train data only may not get much higher score because an over-fitting model cannot classify the new words in the test set properly, let alone an under-fitting model.
\n\nHere is my way, I'm going to use 12 categories rather than 31 to get a more compatible \"unknown\" category and use some of the test data with higher probability like 98% predicted by former model to train the next model. Hoping for a progress.
", "rawMarkdown": "Thank you, James. \n\nSo the range of the test words is bigger than the train words. I think I can conclude that a model trained by the train data only may not get much higher score because an over-fitting model cannot classify the new words in the test set properly, let alone an under-fitting model. \n\nHere is my way, I'm going to use 12 categories rather than 31 to get a more compatible \"unknown\" category and use some of the test data with higher probability like 98% predicted by former model to train the next model. Hoping for a progress." } ] }, { "id": 256970, "postDate": "2017-12-13T02:19:10.880Z", "content": "In my case, the example scripts got ~ 90% of validation accuracy after training, but only 63% accuracy on the Kaggle LB.\nI checked the distribution of selected words in the submitted file and got a plot below.\nSomehow, it could not catch the voices correctly, I guess.
", "rawMarkdown": "In my case, the example scripts got ~ 90% of validation accuracy after training, but only 63% accuracy on the Kaggle LB.\nI checked the distribution of selected words in the submitted file and got a plot below.\nSomehow, it could not catch the voices correctly, I guess.\n" }, { "id": 253894, "postDate": "2017-12-05T20:09:05.600Z", "content": "@Pete, \nI looked at your code on GitHub where you have slightly mentioned about Kaldi for advanced speech recognition systems. Can we use Kaldi for the competition as well or do we have to stick to Keras or Tensorflow?\nKindly suggest.
", "rawMarkdown": "@Pete, \nI looked at your code on GitHub where you have slightly mentioned about Kaldi for advanced speech recognition systems. Can we use Kaldi for the competition as well or do we have to stick to Keras or Tensorflow?\nKindly suggest. " }, { "id": 252296, "postDate": "2017-12-02T17:53:10.733Z", "content": "I'm using Mathematica to build and test my network. If I want to submit an entry what would you need to run it?\nThanks for any guidance you may have.
", "rawMarkdown": "I'm using Mathematica to build and test my network. If I want to submit an entry what would you need to run it?\nThanks for any guidance you may have.", "replies": [ { "id": 252351, "postDate": "2017-12-02T19:36:49.090Z", "content": "You need to apply you mathematica model on the test set and produce something like \"sample_submission.csv\" but with your predicted classes from your network and then upload the results to kaggle. The evaluation is done on kaggle's side since they have the ground truth for the test set.
", "rawMarkdown": "You need to apply you mathematica model on the test set and produce something like \"sample_submission.csv\" but with your predicted classes from your network and then upload the results to kaggle. The evaluation is done on kaggle's side since they have the ground truth for the test set." }, { "id": 252417, "postDate": "2017-12-02T21:56:17.960Z", "content": "Thanks for the clarification. I can do that!
", "rawMarkdown": "Thanks for the clarification. I can do that!" } ] }, { "id": 251868, "postDate": "2017-12-01T20:29:07.123Z", "content": "Thank you very much for organizing this competition.
\n\nI just have a comment on the size of the test data. It is extremely inconvenient to have a test data this size, it takes ages to load and to score because not all of us Kagglers have multiple 64gb ram 1 To SSD machines. What bothers me more is that a part of the testing set won't even be used for evaluation and is there just to discourage hand labeling. I wish we could find a better way against cheating without wasting all this time and kwh scoring useless samples.
", "rawMarkdown": "Thank you very much for organizing this competition.\n\nI just have a comment on the size of the test data. It is extremely inconvenient to have a test data this size, it takes ages to load and to score because not all of us Kagglers have multiple 64gb ram 1 To SSD machines. What bothers me more is that a part of the testing set won't even be used for evaluation and is there just to discourage hand labeling. I wish we could find a better way against cheating without wasting all this time and kwh scoring useless samples." }, { "id": 248272, "postDate": "2017-11-25T13:15:48.970Z", "content": "Hi, thankyou for challenge 😊
", "rawMarkdown": "Hi, thankyou for challenge 😊" }, { "id": 247477, "postDate": "2017-11-23T06:27:36.890Z", "content": "May I build a model without TensorFlow for this competition?
", "rawMarkdown": "May I build a model without TensorFlow for this competition?", "replies": [ { "id": 247765, "postDate": "2017-11-23T19:45:39.020Z", "content": "Yes. That was answered here: https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/43685
", "rawMarkdown": "Yes. That was answered here: https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/43685" } ] }, { "id": 247444, "postDate": "2017-11-23T03:47:02.290Z", "content": "Thanks for hosting the competition Pete! :)
", "rawMarkdown": "Thanks for hosting the competition Pete! :)" }, { "id": 246182, "postDate": "2017-11-20T16:58:48.540Z", "content": "Hi Pete,
\n\nIt is really excited to see a speech recognition test appeared on Kaggle. I have a question about the entry of competition. I work in a speech company and want to use our code to commit the result. The models will be the traditional DNN+HMM+WFST. I may not use tensor flow and share codes to you and also not expect to get any prize from the competition, while I definite only train the models using provided training data. Am I allowed to submit the result? I really want to see the difference performance between traditional decoder and an end-to-end on recognizing command.
", "rawMarkdown": "Hi Pete,\n\nIt is really excited to see a speech recognition test appeared on Kaggle. I have a question about the entry of competition. I work in a speech company and want to use our code to commit the result. The models will be the traditional DNN+HMM+WFST. I may not use tensor flow and share codes to you and also not expect to get any prize from the competition, while I definite only train the models using provided training data. Am I allowed to submit the result? I really want to see the difference performance between traditional decoder and an end-to-end on recognizing command.\n" }, { "id": 245481, "postDate": "2017-11-18T15:51:04.333Z", "content": "Thanks for hosting this, it’s an interesting competition. I have some questions about the special prize.
\n\nHi Pete, the data page has some styling issues under Partitioning section. Please have a look.
Sorry about that! @inversion, are you able to take a look at the Python markdown issues there?
", "rawMarkdown": "Sorry about that! @inversion, are you able to take a look at the Python markdown issues there?" } ] }, { "id": 244946, "postDate": "2017-11-17T08:21:56.943Z", "content": "Pete, a while back you did a lot of cool work using the Pi's GPU to implement GEMM.
\n\nHas there been any progress in exposing it as a TF op?
", "rawMarkdown": "Pete, a while back you did a lot of cool work using the [Pi's GPU to implement GEMM][1]. \n\nHas there been any progress in exposing it as a TF op?\n\n [1]: https://github.com/jetpacapp/pi-gemm", "replies": [ { "id": 245291, "postDate": "2017-11-18T00:14:42.690Z", "content": "Thanks for remembering that! I haven't had a chance to return to it, and the newer Pi models have CPUs that are much more capable than the old ARMv6 CPU that didn't have NEON, so it's not as big an advantage as it was, except on the Zero. I did have a lot of fun working on that though, so I hope I can do some more there eventually.
", "rawMarkdown": "Thanks for remembering that! I haven't had a chance to return to it, and the newer Pi models have CPUs that are much more capable than the old ARMv6 CPU that didn't have NEON, so it's not as big an advantage as it was, except on the Zero. I did have a lot of fun working on that though, so I hope I can do some more there eventually." } ] }, { "id": 244329, "postDate": "2017-11-16T02:17:39.913Z", "content": "Hi Pete,
\n\nExcited to be in this competition - my first ever opportunity to do something with speech data.
\n\nQuick question: would it be possible to build/prototype speech recognition models with the recently introduced TF’s Eager execution?
", "rawMarkdown": "Hi Pete,\n\nExcited to be in this competition - my first ever opportunity to do something with speech data.\n\nQuick question: would it be possible to build/prototype speech recognition models with the recently introduced TF’s Eager execution?", "replies": [ { "id": 244335, "postDate": "2017-11-16T02:36:07.923Z", "content": "I haven't had a chance to play with Eager much myself yet, but all of the ops used in the tutorial are normal TensorFlow ones, so I believe it should be possible. I would be interested to hear how you get on, it would be great to update the tutorial with Eager directions too!
", "rawMarkdown": "I haven't had a chance to play with Eager much myself yet, but all of the ops used in the tutorial are normal TensorFlow ones, so I believe it should be possible. I would be interested to hear how you get on, it would be great to update the tutorial with Eager directions too!", "votes": 3 } ] }, { "id": 323230, "postDate": "2018-05-04T16:47:30.390Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 244842, "postDate": "2017-11-17T02:14:36.153Z", "content": "Thanks for competition !
", "rawMarkdown": "Thanks for competition !", "votes": 1 }, { "id": 248468, "postDate": "2017-11-26T07:18:45.897Z", "content": "Thanks for the competition.
", "rawMarkdown": "Thanks for the competition." }, { "id": 247025, "postDate": "2017-11-22T06:02:17.030Z", "content": "Thanks for this competition.
", "rawMarkdown": "Thanks for this competition." } ], "comments": [ { "id": 253197, "author_name": "Rafael Barbolo", "author_url": "", "post_date": "2017-12-04T15:49:49.897000", "content": "Hi @Pete. Have you tried your solution on the test set?
\n\nIf this is your model, I used the example scripts and got ~ 89% of validation accuracy after training, but only 78% accuracy on the Kaggle LB. So it's probably overfitting the training/validation data.
\n\nI'm posting this because we are seeing such great solutions on the leaderboard and your initial post saying that you had an ~ 88% accuracy model could confuse someone.
", "votes": 3, "replies": [ { "id": 253776, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-12-05T16:36:28.553000", "content": "That is a good point Rafael, thanks for the question. I do see the same results using the pretrained model (89% on the training scripts, 78% on the leaderboard). I'm hopeful that this is due to the different distributions of categories of data that are measured to produce the average accuracy, rather than overfitting, but I haven't investigated any deeper.
\n\nHopefully that's enough to reassure anybody using the example as a starting point that they're not doing anything wrong though? I'm excited to see we have big improvements over that on the LB already!
", "votes": 2, "replies": [] }, { "id": 253794, "author_name": "Rafael Barbolo", "author_url": "", "post_date": "2017-12-05T17:00:23.457000", "content": "Your model is definitely very useful for anyone working in this competition.
\n\nI've run some tests and I believe the example doesn't generalise well to the test set because the data used by the example model (~ 65k samples) probably have a different distribution regarding the real test set of this competition (~ 150k samples). That would explain why the example is not overfitting in the validation set, but doesn't achieve a higher accuracy in the competition.
\n\nI'll start to work on data augmentation (like pitch shifting) because this is probably going to lead to better results than tweaking the network architecture.
\n\nUsing Squeezenet, I got ~ 83% on Kaggle Leaderboard after ~ 300 epochs.\nUsing Inception Resnet V2, I got ~ 84% accuracy on Kaggle Leaderboard after ~ 10 epochs.
\n\nGood luck for everyone.
", "votes": 5, "replies": [] } ] }, { "id": 244802, "author_name": "Lukas Grasse", "author_url": "", "post_date": "2017-11-17T00:41:23.840000", "content": "Hi Pete,
\n\nI have a quick question: is the use of pre-trained models such as VGGish:https://github.com/tensorflow/models/tree/master/research/audioset allowed? or is this considered using external data?
", "votes": 1, "replies": [ { "id": 244812, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-17T01:36:03.953000", "content": "There's another thread on this that goes into more detail, but the summary is they are considered external data (since they're trained on data outside of the released set).
", "votes": 3, "replies": [] } ] }, { "id": 244295, "author_name": "Miha Skalic", "author_url": "", "post_date": "2017-11-16T00:17:57.090000", "content": "Hello Pete,
\n\nFrom what I understood the \"./benchmark_model\" should be used on a raspberry, right? Is there away for people to test runtime without owning the device - perhaps some kind of an environment?
", "votes": 1, "replies": [ { "id": 244310, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-16T00:47:49.553000", "content": "There isn't a good way to estimate the speed on a Pi without access to one unfortunately. It is possible to run the same command on a desktop x86 machine if you've created a model and get a rough idea of relative speeds (for example model v2 is twice as slow as model v1), but even that may not map accurately over to an ARM device.
", "votes": 1, "replies": [] }, { "id": 244311, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-16T00:49:25.387000", "content": "And just to clarify, you'll have to build and run the x86 version of tensorflow/tools/benchmark:benchmark_model to run it on your desktop machine, the curl commands to download the Pi version won't work.
", "votes": 0, "replies": [] }, { "id": 266752, "author_name": "Thomas O'Malley", "author_url": "", "post_date": "2018-01-09T16:32:09.693000", "content": "Hi Pete, thanks so much for hosting this competition, I'm having issues with trying to use the provided benchmark script to benchmark a model frozen using Tensorflow 1.4 and strided_slice
The error I'm getting is:
\n\nCreate kernel failed: Invalid argument: NodeDef mentions attr 'identical_element_shapes' not in Op<name=TensorArrayV3; signature=size:int32 -> handle:resource, flow:float; attr=dtype:type; attr=element_shape:shape,default=<unknown>; attr=dynamic_size:bool,default=false; attr=clear_after_read:bool,default=true; attr=tensor_array_name:string,default=\"\"; is_stateful=true>; NodeDef: cond/map/TensorArray = TensorArrayV3[clear_after_read=true, dtype=DT_FLOAT, dynamic_size=false, element_shape=<unknown>, identical_element_shapes=true, tensor_array_name=\"\", _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](cond/map/strided_slice). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
Other models that I tried worked fine. Is there something simple I'm doing wrong, or is it possible the benchmark script is out of date, and if so could we get a version for TF 1.4? (having a heck of a time trying to figure out how to cross-compile the benchmark tool for the Pi)
", "votes": 1, "replies": [] }, { "id": 267627, "author_name": "Thomas O'Malley", "author_url": "", "post_date": "2018-01-11T22:35:34.277000", "content": "Kindly disregard, I was able to get it to run using tensorflow==1.4.0
", "votes": 0, "replies": [] }, { "id": 267639, "author_name": "Pete Warden", "author_url": "", "post_date": "2018-01-11T23:27:06.573000", "content": "Thanks for the update, I'm glad you're able to run now!
", "votes": 0, "replies": [] }, { "id": 276322, "author_name": "kushwanth goutham", "author_url": "", "post_date": "2018-01-31T07:14:50.437000", "content": "Hi Thomas, I recently got the same issue but not able to resolve it. I have trained the model in tensorflow 1.5.0-dev20171023 and when exporting the model i used 1.5.0, I am getting same error as you. How's the error got resolved ?
\n\nAny kind of help is appreciated.
", "votes": 0, "replies": [] } ] }, { "id": 244282, "author_name": "yuerlong", "author_url": "", "post_date": "2017-11-15T23:35:21.720000", "content": "thanks for this competition. good chance to learn pure Tensorflow although I am used to stick with Keras
", "votes": 1, "replies": [ { "id": 244287, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-16T00:04:18.727000", "content": "No problem, thanks for jumping in! I'm a fan of Keras too, though I haven't used it for audio myself.
", "votes": 1, "replies": [] } ] }, { "id": 250492, "author_name": "oursland", "author_url": "", "post_date": "2017-11-30T03:55:06.177000", "content": "Raspbian has recently been updated to Debian 9 (Stretch). https://www.raspberrypi.org/downloads/raspbian/
\n\nThe rules state Raspbian 8 (Jessie) shall be used in the evaluation. Is there a link to the image that will be used for testing?
", "votes": 2, "replies": [] }, { "id": 244216, "author_name": "Shujian Liu", "author_url": "", "post_date": "2017-11-15T20:18:46.930000", "content": "Hi Pete, thanks for hosting this competition. Sorry to ask a silly question: is the competition related to the newly released TensorFlow Lite? Didn't have much time to read that.
", "votes": 2, "replies": [ { "id": 244219, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-15T20:35:44.910000", "content": "Good question! I am hoping to use the resulting model in TF Lite, since we're aiming that at devices like the Pi, but that's still a little way down the road, and it isn't supported there yet.
", "votes": 3, "replies": [] }, { "id": 244221, "author_name": "Shujian Liu", "author_url": "", "post_date": "2017-11-15T20:42:08.440000", "content": "Good to know that!
", "votes": 0, "replies": [] } ] }, { "id": 247230, "author_name": "NaveenManwani", "author_url": "", "post_date": "2017-11-22T16:28:14.937000", "content": "hi Pete,please forgive me if these are very dump questions and correct me if i'm wrong anywhere.\nquestion1\nwhen i was installing tensorflow from source using this repotensorflow on raspberry pi 3 .i was confused on one question which is [Do you wish to build TensorFlow with GPU support? [y/N]]\nif i'm correct Tensorflow leverages Nvidia drivers to power Nvidia GPUs and Raspberry Pi 3 model Bdoes not have Nvidia hardware. it has GPU: Broadcom VideoCore IV.so technically i should say \"no\" to this question ,please give me your insights on it\nquestion2\n i'm unable to find wheel for tensorflow 1.4 version ,python 3.4 could you please guide me how i should tackle this problem
", "votes": -1, "replies": [] }, { "id": 261665, "author_name": "kirk", "author_url": "", "post_date": "2017-12-23T13:41:19.950000", "content": "Hi @Pete. Just a quick question. There's been a lot of mentioning in regards to using the example audio speech code as a starting point but there is a problem with that. The audio_ops are missing and this is still not fixed as far as I know. In the github issues people were talking about v1.5 of tensorflow that might resolve. I don't know if other people had the same issues and how they resolved it. Any suggestions?
", "votes": 0, "replies": [ { "id": 267233, "author_name": "Jeff", "author_url": "", "post_date": "2018-01-10T22:16:53.117000", "content": "The audio_ops worked for me. I compiled Tensorflow from source, so that may have something to do with it.
", "votes": 0, "replies": [] } ] }, { "id": 261282, "author_name": "Mikhail Pavlov", "author_url": "", "post_date": "2017-12-22T06:33:57.247000", "content": "Hi @Pete Could you please tell about VAD model https://pypi.python.org/pypi/webrtcvad. Is it allowed or not? Thanks in advance.
", "votes": 0, "replies": [] }, { "id": 260889, "author_name": "Huynh", "author_url": "", "post_date": "2017-12-21T05:39:27.487000", "content": "Hi, I have a problem with the model for Raspberry Pi 3. I can not create the inputs have the names:
\n\n\n\n\ndecoded_sample_data:0, taking a [16000, 1] float tensor as input,\n representing the audio PCM-encoded data.
\n \ndecoded_sample_data:1, taking a scalar [] int32 tensor as input,\n representing the sample rate, which must be the value 16000.
\n
I created 2 tensors as follows:
\n\n\n\n\nsample_placeholder = tf.placeholder(dtype=tf.float32,\n shape=[16000,1],name='decoded_sample_data') sr_placeholder =\n tf.placeholder(dtype=tf.int32, name='decoded_sample_data')
\n
But their name are different. How can I do it? Thank
", "votes": 0, "replies": [] }, { "id": 260485, "author_name": "Armen Donigian", "author_url": "", "post_date": "2017-12-20T09:40:28.653000", "content": "@Pete I'm no longer seeing checkpoint file (conv.ckpt-18000) being created which is needed for freeze.py . I've created an issue @ https://github.com/tensorflow/tensorflow/issues/15505, would love to hear any suggestions you may have.
Armen, the freeze script uses the prefix by convention. I see your conv.ckpt-18000 files in your directory list output posted to the GitHub issue, so the freeze script will work without any problems if you simply run \"python freeze.py --start_checkpoint=path/to/conv.ckpt-18000 --model_architecture conv --output_file=frozen_graph.pb\"
", "votes": 1, "replies": [] }, { "id": 260679, "author_name": "Tasos Vafeiadis", "author_url": "", "post_date": "2017-12-20T17:50:56.967000", "content": "Regarding the label_wav.py, did you have to create an additional function inside the label_wav.py so you can run it within a directory and save it to CSV. For example:\npython tensorflow/examples/speech_commands/label_wav.py \\\n--graph=/tmp/my_frozen_graph.pb \\\n--labels=/tmp/speech_commands_train/conv_labels.txt \\\n--wav=/tmp/speech_dataset/*/*wav
", "votes": 0, "replies": [] }, { "id": 260726, "author_name": "Jeff", "author_url": "", "post_date": "2017-12-20T19:19:40.773000", "content": "Tasos, that is one option, but I imported the functions into a different script in order to use the Pandas library to plot some visualizations of the predictions before saving them to CSV. The label_wav.py script is very slow because it reloads your model for one wav file. It's better to load your model once and then run a batch of predictions.
", "votes": 0, "replies": [] }, { "id": 260758, "author_name": "Armen Donigian", "author_url": "", "post_date": "2017-12-20T21:11:52.823000", "content": "@Tasos yes, I added a function to output a submission file as well as predict probabilities for all classes to be used later for stacking.
\n\nAdditionally, I've added a new clause to create_model in models.py, looking for the name of my architecture and then calling a model creation function. I've found this demo app to be very useful in terms of a pluggable pipeline to experiment with ctc, lstm, and plain ol 2d cnns.
", "votes": 0, "replies": [] } ] }, { "id": 258076, "author_name": " Ferdinand Zhen", "author_url": "", "post_date": "2017-12-15T11:57:41.870000", "content": "Hi Pete
\n\nThanks for the competition, it is my first kaggle project and also my first ML project. I got a 0.84, and I still want to move on. But I have a little confusion.
\n\nI found that the test audios may contain words like \"learn\" and \"visual\", which are predicted by my model as \"left\" and \"zero\". I'm not sure if it is my mistake, or these words do exist and should be predicted as \"unknown\".
\n\nExpecting a reply, so I can find the right direction to optimize my model.
", "votes": 0, "replies": [ { "id": 258141, "author_name": "James King", "author_url": "", "post_date": "2017-12-15T15:40:50.073000", "content": "The words you mention do exist in the test set and should be predicted as unknown.
", "votes": 0, "replies": [] }, { "id": 258388, "author_name": " Ferdinand Zhen", "author_url": "", "post_date": "2017-12-16T02:42:15.410000", "content": "Thank you, James.
\n\nSo the range of the test words is bigger than the train words. I think I can conclude that a model trained by the train data only may not get much higher score because an over-fitting model cannot classify the new words in the test set properly, let alone an under-fitting model.
\n\nHere is my way, I'm going to use 12 categories rather than 31 to get a more compatible \"unknown\" category and use some of the test data with higher probability like 98% predicted by former model to train the next model. Hoping for a progress.
", "votes": 0, "replies": [] } ] }, { "id": 256970, "author_name": "Wataru Yasuda", "author_url": "", "post_date": "2017-12-13T02:19:10.880000", "content": "In my case, the example scripts got ~ 90% of validation accuracy after training, but only 63% accuracy on the Kaggle LB.\nI checked the distribution of selected words in the submitted file and got a plot below.\nSomehow, it could not catch the voices correctly, I guess.
", "votes": 0, "replies": [] }, { "id": 253894, "author_name": "ManiKhanuja", "author_url": "", "post_date": "2017-12-05T20:09:05.600000", "content": "@Pete, \nI looked at your code on GitHub where you have slightly mentioned about Kaldi for advanced speech recognition systems. Can we use Kaldi for the competition as well or do we have to stick to Keras or Tensorflow?\nKindly suggest.
", "votes": 0, "replies": [] }, { "id": 252296, "author_name": "Steve F", "author_url": "", "post_date": "2017-12-02T17:53:10.733000", "content": "I'm using Mathematica to build and test my network. If I want to submit an entry what would you need to run it?\nThanks for any guidance you may have.
", "votes": 0, "replies": [ { "id": 252351, "author_name": "CVxTz", "author_url": "", "post_date": "2017-12-02T19:36:49.090000", "content": "You need to apply you mathematica model on the test set and produce something like \"sample_submission.csv\" but with your predicted classes from your network and then upload the results to kaggle. The evaluation is done on kaggle's side since they have the ground truth for the test set.
", "votes": 0, "replies": [] }, { "id": 252417, "author_name": "Steve F", "author_url": "", "post_date": "2017-12-02T21:56:17.960000", "content": "Thanks for the clarification. I can do that!
", "votes": 0, "replies": [] } ] }, { "id": 251868, "author_name": "CVxTz", "author_url": "", "post_date": "2017-12-01T20:29:07.123000", "content": "Thank you very much for organizing this competition.
\n\nI just have a comment on the size of the test data. It is extremely inconvenient to have a test data this size, it takes ages to load and to score because not all of us Kagglers have multiple 64gb ram 1 To SSD machines. What bothers me more is that a part of the testing set won't even be used for evaluation and is there just to discourage hand labeling. I wish we could find a better way against cheating without wasting all this time and kwh scoring useless samples.
", "votes": 0, "replies": [] }, { "id": 248272, "author_name": "Rikson Gultom", "author_url": "", "post_date": "2017-11-25T13:15:48.970000", "content": "Hi, thankyou for challenge 😊
", "votes": 0, "replies": [] }, { "id": 247477, "author_name": "Ben Lai", "author_url": "", "post_date": "2017-11-23T06:27:36.890000", "content": "May I build a model without TensorFlow for this competition?
", "votes": 0, "replies": [ { "id": 247765, "author_name": "Batangas", "author_url": "", "post_date": "2017-11-23T19:45:39.020000", "content": "Yes. That was answered here: https://www.kaggle.com/c/tensorflow-speech-recognition-challenge/discussion/43685
", "votes": 0, "replies": [] } ] }, { "id": 247444, "author_name": "Ankit Goila", "author_url": "", "post_date": "2017-11-23T03:47:02.290000", "content": "Thanks for hosting the competition Pete! :)
", "votes": 0, "replies": [] }, { "id": 246182, "author_name": "Yu Yang", "author_url": "", "post_date": "2017-11-20T16:58:48.540000", "content": "Hi Pete,
\n\nIt is really excited to see a speech recognition test appeared on Kaggle. I have a question about the entry of competition. I work in a speech company and want to use our code to commit the result. The models will be the traditional DNN+HMM+WFST. I may not use tensor flow and share codes to you and also not expect to get any prize from the competition, while I definite only train the models using provided training data. Am I allowed to submit the result? I really want to see the difference performance between traditional decoder and an end-to-end on recognizing command.
", "votes": 0, "replies": [] }, { "id": 245481, "author_name": "Tim Hoolihan", "author_url": "", "post_date": "2017-11-18T15:51:04.333000", "content": "Thanks for hosting this, it’s an interesting competition. I have some questions about the special prize.
\n\nHi Pete, the data page has some styling issues under Partitioning section. Please have a look.
Sorry about that! @inversion, are you able to take a look at the Python markdown issues there?
", "votes": 0, "replies": [] } ] }, { "id": 244946, "author_name": "Andrés Miguel Torrubia Sáez", "author_url": "", "post_date": "2017-11-17T08:21:56.943000", "content": "Pete, a while back you did a lot of cool work using the Pi's GPU to implement GEMM.
\n\nHas there been any progress in exposing it as a TF op?
", "votes": 0, "replies": [ { "id": 245291, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-18T00:14:42.690000", "content": "Thanks for remembering that! I haven't had a chance to return to it, and the newer Pi models have CPUs that are much more capable than the old ARMv6 CPU that didn't have NEON, so it's not as big an advantage as it was, except on the Zero. I did have a lot of fun working on that though, so I hope I can do some more there eventually.
", "votes": 0, "replies": [] } ] }, { "id": 244329, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2017-11-16T02:17:39.913000", "content": "Hi Pete,
\n\nExcited to be in this competition - my first ever opportunity to do something with speech data.
\n\nQuick question: would it be possible to build/prototype speech recognition models with the recently introduced TF’s Eager execution?
", "votes": 0, "replies": [ { "id": 244335, "author_name": "Pete Warden", "author_url": "", "post_date": "2017-11-16T02:36:07.923000", "content": "I haven't had a chance to play with Eager much myself yet, but all of the ops used in the tutorial are normal TensorFlow ones, so I believe it should be possible. I would be interested to hear how you get on, it would be great to update the tutorial with Eager directions too!
", "votes": 3, "replies": [] } ] }, { "id": 323230, "author_name": "", "author_url": "", "post_date": "2018-05-04T16:47:30.390000", "content": "", "votes": 2, "replies": [] }, { "id": 244842, "author_name": "quang vu", "author_url": "", "post_date": "2017-11-17T02:14:36.153000", "content": "Thanks for competition !
", "votes": 1, "replies": [] }, { "id": 248468, "author_name": "Amanpreet Singh", "author_url": "", "post_date": "2017-11-26T07:18:45.897000", "content": "Thanks for the competition.
", "votes": 0, "replies": [] }, { "id": 247025, "author_name": "Ugur Uresin", "author_url": "", "post_date": "2017-11-22T06:02:17.030000", "content": "Thanks for this competition.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "244194": "Welcome and thanks to everyone who's looking into this competition! I'm a long-time Kaggle fan, and actually ran a contest six years ago that I really learned a lot from, so I'm glad to have another chance to work with the awesome people here: https://www.kaggle.com/c/PhotoQualityPrediction#description\n\nI'm also the tech lead of the mobile and embedded side of TensorFlow, and I'm particularly excited because I'm hoping the results will directly feed back into the community through TensorFlow open source contributions, and help people hack Raspberry Pis to do even more interesting things. \n\nIf you'd like a quick way to get started, I have a tutorial on how to use TensorFlow to train audio models on the training set here: https://www.tensorflow.org/tutorials/audio_recognition\n\nMy best model there achieves around 88% accuracy (using the script's metrics), but I'm confident that the community here can do much better. I will be hanging out here on the forums to try to answer questions, though I do have a vacation planned over Thanksgiving (next week and the week after), so my responses may be a bit delayed then. I'm also active on Twitter at https://twitter.com/petewarden, though these forums are probably the best place to ask questions so the knowledge is easy to find for everyone. Please do ping me with anything you'd like more information on.\n\nAs a final note, we're still actively trying to build out the underlying open data set, so please do go to https://aiyprojects.withgoogle.com/open_speech_recording if you can and share your voice for future releases!", "253197": "Hi @Pete. Have you tried your solution on the test set?\n\nIf this is your [model][1], I used the example scripts and got ~ 89% of validation accuracy after training, but only 78% accuracy on the Kaggle LB. So it's probably overfitting the training/validation data.\n\nI'm posting this because we are seeing such great solutions on the leaderboard and your initial post saying that you had an ~ 88% accuracy model could confuse someone.\n\n [1]: https://github.com/tensorflow/tensorflow/tree/v1.4.0/tensorflow/examples/speech_commands", "244802": "Hi Pete,\n\nI have a quick question: is the use of pre-trained models such as VGGish:https://github.com/tensorflow/models/tree/master/research/audioset allowed? or is this considered using external data?", "244295": "Hello Pete,\n\nFrom what I understood the \"./benchmark_model\" should be used on a raspberry, right? Is there away for people to test runtime without owning the device - perhaps some kind of an environment? ", "244282": "thanks for this competition. good chance to learn pure Tensorflow although I am used to stick with Keras", "250492": "Raspbian has recently been updated to Debian 9 (Stretch). https://www.raspberrypi.org/downloads/raspbian/\n\nThe rules state Raspbian 8 (Jessie) shall be used in the evaluation. Is there a link to the image that will be used for testing?", "244216": "Hi Pete, thanks for hosting this competition. Sorry to ask a silly question: is the competition related to the newly released TensorFlow Lite? Didn't have much time to read that.", "247230": "hi Pete,please forgive me if these are very dump questions and correct me if i'm wrong anywhere.\nquestion1\nwhen i was installing tensorflow from source using this repo[tensorflow on raspberry pi 3][1] .i was confused on one question which is [Do you wish to build TensorFlow with GPU support? [y/N]]\nif i'm correct Tensorflow leverages Nvidia drivers to power Nvidia GPUs and Raspberry Pi 3 model Bdoes not have Nvidia hardware. it has GPU: Broadcom VideoCore IV.so technically i should say \"no\" to this question ,please give me your insights on it\nquestion2\n i'm unable to find wheel for tensorflow 1.4 version ,python 3.4 could you please guide me how i should tackle this problem\n\n\n \n\n\n [1]: https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/244bf9c48d81105b7b97b539448f2b818dfa9d91/GUIDE.md", "261665": "Hi @Pete. Just a quick question. There's been a lot of mentioning in regards to using the example audio speech code as a starting point but there is a problem with that. The audio_ops are missing and this is still not fixed as far as I know. In the github issues people were talking about v1.5 of tensorflow that might resolve. I don't know if other people had the same issues and how they resolved it. Any suggestions?", "261282": "Hi @Pete Could you please tell about VAD model https://pypi.python.org/pypi/webrtcvad. Is it allowed or not? Thanks in advance.", "260889": "Hi, I have a problem with the model for Raspberry Pi 3. I can not create the inputs have the names:\n\n> decoded_sample_data:0, taking a [16000, 1] float tensor as input,\n> representing the audio PCM-encoded data.\n> \n> decoded_sample_data:1, taking a scalar [] int32 tensor as input,\n> representing the sample rate, which must be the value 16000.\n\nI created 2 tensors as follows:\n\n> sample_placeholder = tf.placeholder(dtype=tf.float32,\n> shape=[16000,1],name='decoded_sample_data') sr_placeholder =\n> tf.placeholder(dtype=tf.int32, name='decoded_sample_data')\n\nBut their name are different. How can I do it? Thank", "260485": "@Pete I'm no longer seeing checkpoint file (conv.ckpt-18000) being created which is needed for `freeze.py` . I've created an issue @ https://github.com/tensorflow/tensorflow/issues/15505, would love to hear any suggestions you may have.", "258076": "Hi Pete\n\nThanks for the competition, it is my first kaggle project and also my first ML project. I got a 0.84, and I still want to move on. But I have a little confusion.\n\nI found that the test audios may contain words like \"learn\" and \"visual\", which are predicted by my model as \"left\" and \"zero\". I'm not sure if it is my mistake, or these words do exist and should be predicted as \"unknown\".\n\n Expecting a reply, so I can find the right direction to optimize my model.\n", "256970": "In my case, the example scripts got ~ 90% of validation accuracy after training, but only 63% accuracy on the Kaggle LB.\nI checked the distribution of selected words in the submitted file and got a plot below.\nSomehow, it could not catch the voices correctly, I guess.\n", "253894": "@Pete, \nI looked at your code on GitHub where you have slightly mentioned about Kaldi for advanced speech recognition systems. Can we use Kaldi for the competition as well or do we have to stick to Keras or Tensorflow?\nKindly suggest. ", "252296": "I'm using Mathematica to build and test my network. If I want to submit an entry what would you need to run it?\nThanks for any guidance you may have.", "251868": "Thank you very much for organizing this competition.\n\nI just have a comment on the size of the test data. It is extremely inconvenient to have a test data this size, it takes ages to load and to score because not all of us Kagglers have multiple 64gb ram 1 To SSD machines. What bothers me more is that a part of the testing set won't even be used for evaluation and is there just to discourage hand labeling. I wish we could find a better way against cheating without wasting all this time and kwh scoring useless samples.", "248272": "Hi, thankyou for challenge 😊", "247477": "May I build a model without TensorFlow for this competition?", "247444": "Thanks for hosting the competition Pete! :)", "246182": "Hi Pete,\n\nIt is really excited to see a speech recognition test appeared on Kaggle. I have a question about the entry of competition. I work in a speech company and want to use our code to commit the result. The models will be the traditional DNN+HMM+WFST. I may not use tensor flow and share codes to you and also not expect to get any prize from the competition, while I definite only train the models using provided training data. Am I allowed to submit the result? I really want to see the difference performance between traditional decoder and an end-to-end on recognizing command.\n", "245481": "Thanks for hosting this, it’s an interesting competition. I have some questions about the special prize. \n\n* is there a score threshold to be eligible to self select for the pi competition?\n* could a team submit with both prizes in mind? ie A Keira’s model aiming at first prize and a different network aimed at pi prize?\n* since we don’t know who will self select, there will essentially be no public leaderboard for that prize, correct?", "244997": "Hi Pete, the `data` page has some styling issues under `Partitioning` section. Please have a look.", "244946": "Pete, a while back you did a lot of cool work using the [Pi's GPU to implement GEMM][1]. \n\nHas there been any progress in exposing it as a TF op?\n\n [1]: https://github.com/jetpacapp/pi-gemm", "244329": "Hi Pete,\n\nExcited to be in this competition - my first ever opportunity to do something with speech data.\n\nQuick question: would it be possible to build/prototype speech recognition models with the recently introduced TF’s Eager execution?", "323230": "", "244842": "Thanks for competition !", "248468": "Thanks for the competition.", "247025": "Thanks for this competition." } }