I do a fair amount of exploratory analysis on datasets to avoid making unnecessary assumptions. I try to find problem data early, which can help later.
\n\nSo, I looked at the train and test wav files to make sure they were all consistent having the same sampling rate, number of channels, format, and bits per sample. All train and test file were consistent in the wav attributes. That's very good!
\n\n90% of the train files had exactly 16000 samples (1 sec), but about 10% were shorter. So, any processing of the train files cannot assume constant length. However, 100% of the test files had exactly 16000 samples, so those will be easier to process.
\n\nI looked at the number utterances by word. One speaker uttered \"six\" 12 times, but few words were uttered more than 3 or 4 times. [See summary table, Number of train utterances by word, in File-Inventory .html or Jupyter notebook files below.]
\n\nLastly, I computed md5sums for all train and test files to look for duplicates.
\n\nTrain file count: 64,721\nTest file count: 158,538\nAll: 223,259\n\nDuplicates found using md5sums:\n 1 2 3 4 5 6 3710 Duplicates\n215795 1557 172 27 2 1 1 Frequency\nThat is,\n 215,795 files were unique;\n 1557 files appear 2 times;\n 172 files appear 3 times;\n . . .\n 1 file appears 3710 times!
\n\nMost of the duplicates appeared in test or training but not both.
\n\nA few duplicates appeared in both test and training, but there’s one serious problem:\nOne file was repeated 3710 times, appearing once in training and 3709 times in test.
\n\nThe train file was:
\n\ntrain/audio/bird/3e7124ba_nohash_0.wav \nSome of the identical test files (same md5sums) included:
\n\ntest/audio/clip_00293950f.wav \ntest/audio/clip_0064f7bae.wav \ntest/audio/clip_006ee8636.wav \n+ 3706 other identical files\nSo, getting 3709 predictions right should be easy!
\n\nThe Jupyter notebook below using R reads both the train and test files, so I did not attempt to post it as a kernel (only the train files are there, right?). Three csv output files are also included.
", "messages": [ { "id": "251419", "postDate": "12/01/2017 06:36:00", "content": "I do a fair amount of exploratory analysis on datasets to avoid making unnecessary assumptions. I try to find problem data early, which can help later.
\n\nSo, I looked at the train and test wav files to make sure they were all consistent having the same sampling rate, number of channels, format, and bits per sample. All train and test file were consistent in the wav attributes. That's very good!
\n\n90% of the train files had exactly 16000 samples (1 sec), but about 10% were shorter. So, any processing of the train files cannot assume constant length. However, 100% of the test files had exactly 16000 samples, so those will be easier to process.
\n\nI looked at the number utterances by word. One speaker uttered \"six\" 12 times, but few words were uttered more than 3 or 4 times. [See summary table, Number of train utterances by word, in File-Inventory .html or Jupyter notebook files below.]
\n\nLastly, I computed md5sums for all train and test files to look for duplicates.
\n\nTrain file count: 64,721\nTest file count: 158,538\nAll: 223,259\n\nDuplicates found using md5sums:\n 1 2 3 4 5 6 3710 Duplicates\n215795 1557 172 27 2 1 1 Frequency\nThat is,\n 215,795 files were unique;\n 1557 files appear 2 times;\n 172 files appear 3 times;\n . . .\n 1 file appears 3710 times!
\n\nMost of the duplicates appeared in test or training but not both.
\n\nA few duplicates appeared in both test and training, but there’s one serious problem:\nOne file was repeated 3710 times, appearing once in training and 3709 times in test.
\n\nThe train file was:
\n\ntrain/audio/bird/3e7124ba_nohash_0.wav \nSome of the identical test files (same md5sums) included:
\n\ntest/audio/clip_00293950f.wav \ntest/audio/clip_0064f7bae.wav \ntest/audio/clip_006ee8636.wav \n+ 3706 other identical files\nSo, getting 3709 predictions right should be easy!
\n\nThe Jupyter notebook below using R reads both the train and test files, so I did not attempt to post it as a kernel (only the train files are there, right?). Three csv output files are also included.
", "rawMarkdown": "I do a fair amount of exploratory analysis on datasets to avoid making unnecessary assumptions. I try to find problem data early, which can help later.\n\nSo, I looked at the train and test wav files to make sure they were all consistent having the same sampling rate, number of channels, format, and bits per sample. All train and test file were consistent in the wav attributes. That's very good!\n\n90% of the train files had exactly 16000 samples (1 sec), but about 10% were shorter. So, any processing of the train files cannot assume constant length. However, 100% of the test files had exactly 16000 samples, so those will be easier to process.\n\nI looked at the number utterances by word. One speaker uttered \"six\" 12 times, but few words were uttered more than 3 or 4 times. [See summary table, *Number of train utterances by word,* in File-Inventory .html or Jupyter notebook files below.]\n\nLastly, I computed md5sums for all train and test files to look for duplicates.\n\n Train file count: 64,721\n Test file count: 158,538\n All: 223,259\n\n Duplicates found using md5sums:\n 1 2 3 4 5 6 3710 Duplicates\n 215795 1557 172 27 2 1 1 Frequency\n\nThat is,\n 215,795 files were unique;\n 1557 files appear 2 times;\n 172 files appear 3 times;\n . . .\n **1 file appears 3710 times!**\n\nMost of the duplicates appeared in test or training but not both. \n\nA few duplicates appeared in both test and training, but there’s one serious problem:\nOne file was repeated 3710 times, appearing once in training and 3709 times in test.\n\nThe train file was:\n\n train/audio/bird/3e7124ba_nohash_0.wav \n\nSome of the identical test files (same md5sums) included:\n\n test/audio/clip_00293950f.wav \n test/audio/clip_0064f7bae.wav \n test/audio/clip_006ee8636.wav \n + 3706 other identical files\n\nSo, getting 3709 predictions right should be easy!\n\nThe Jupyter notebook below using R reads both the train and test files, so I did not attempt to post it as a kernel (only the train files are there, right?). Three csv output files are also included.", "votes": null }, { "id": "251442", "postDate": "12/01/2017 07:41:10", "content": "Quite interesting. If u see in data page its written \"Not all of the files are evaluated for the leaderboard score.\". Could be a case of increasing test size to discourage manual labeling
", "rawMarkdown": "Quite interesting. If u see in data page its written \"Not all of the files are evaluated for the leaderboard score.\". Could be a case of increasing test size to discourage manual labeling", "votes": null }, { "id": "251526", "postDate": "12/01/2017 10:23:41", "content": "All these 3710 files seem to be all zero files (silence)
", "rawMarkdown": "All these 3710 files seem to be all zero files (silence)", "votes": null }, { "id": "255796", "postDate": "12/10/2017 05:46:10", "content": "Thanks for the feedback, Stas SI, about those 3710 duplicate files being all zeros.
\n\nTo study files with small amplitude changes, like all the zero files, I added additional wav file metadata to the train and test \"inventory\" csv files, including amplitude quantiles [Q0 (min), Q10, Q25, Q50 (median), Q75, Q90, Q100 (max)], amplitude range, mean, sd. All the updates are now on GitHub (where I should have put the original files).
\n\nThe density plots of the amplitude ranges for the train and test sets are not the same. The test set has more files at the extremes, including the 3709 test files that are all zeros, and a number of files with the full amplitude range for 16-bit sound (-32768 to 32767 = 65535). See attached density plots.
", "rawMarkdown": "Thanks for the feedback, Stas SI, about those 3710 duplicate files being all zeros. \n\nTo study files with small amplitude changes, like all the zero files, I added additional wav file metadata to the train and test \"inventory\" csv files, including amplitude quantiles [Q0 (min), Q10, Q25, Q50 (median), Q75, Q90, Q100 (max)], amplitude range, mean, sd. All the [updates are now on GitHub][1] (where I should have put the original files).\n\nThe density plots of the amplitude ranges for the train and test sets are not the same. The test set has more files at the extremes, including the 3709 test files that are all zeros, and a number of files with the full amplitude range for 16-bit sound (-32768 to 32767 = 65535). See attached density plots.\n\n [1]: https://github.com/EarlGlynn/kaggle-speech-recognition/tree/master/Jupyter/00-WAV-File-Inventory-Duplicates", "votes": null } ], "comments": [ { "id": 251442, "author_name": "", "author_url": "", "post_date": "12/01/2017 07:41:10", "content": "Quite interesting. If u see in data page its written \"Not all of the files are evaluated for the leaderboard score.\". Could be a case of increasing test size to discourage manual labeling
", "votes": null, "replies": [] }, { "id": 251526, "author_name": "stassl", "author_url": "", "post_date": "12/01/2017 10:23:41", "content": "All these 3710 files seem to be all zero files (silence)
", "votes": null, "replies": [] }, { "id": 255796, "author_name": "efglynn", "author_url": "", "post_date": "12/10/2017 05:46:10", "content": "Thanks for the feedback, Stas SI, about those 3710 duplicate files being all zeros.
\n\nTo study files with small amplitude changes, like all the zero files, I added additional wav file metadata to the train and test \"inventory\" csv files, including amplitude quantiles [Q0 (min), Q10, Q25, Q50 (median), Q75, Q90, Q100 (max)], amplitude range, mean, sd. All the updates are now on GitHub (where I should have put the original files).
\n\nThe density plots of the amplitude ranges for the train and test sets are not the same. The test set has more files at the extremes, including the 3709 test files that are all zeros, and a number of files with the full amplitude range for 16-bit sound (-32768 to 32767 = 65535). See attached density plots.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "251419": "I do a fair amount of exploratory analysis on datasets to avoid making unnecessary assumptions. I try to find problem data early, which can help later.\n\nSo, I looked at the train and test wav files to make sure they were all consistent having the same sampling rate, number of channels, format, and bits per sample. All train and test file were consistent in the wav attributes. That's very good!\n\n90% of the train files had exactly 16000 samples (1 sec), but about 10% were shorter. So, any processing of the train files cannot assume constant length. However, 100% of the test files had exactly 16000 samples, so those will be easier to process.\n\nI looked at the number utterances by word. One speaker uttered \"six\" 12 times, but few words were uttered more than 3 or 4 times. [See summary table, *Number of train utterances by word,* in File-Inventory .html or Jupyter notebook files below.]\n\nLastly, I computed md5sums for all train and test files to look for duplicates.\n\n Train file count: 64,721\n Test file count: 158,538\n All: 223,259\n\n Duplicates found using md5sums:\n 1 2 3 4 5 6 3710 Duplicates\n 215795 1557 172 27 2 1 1 Frequency\n\nThat is,\n 215,795 files were unique;\n 1557 files appear 2 times;\n 172 files appear 3 times;\n . . .\n **1 file appears 3710 times!**\n\nMost of the duplicates appeared in test or training but not both. \n\nA few duplicates appeared in both test and training, but there’s one serious problem:\nOne file was repeated 3710 times, appearing once in training and 3709 times in test.\n\nThe train file was:\n\n train/audio/bird/3e7124ba_nohash_0.wav \n\nSome of the identical test files (same md5sums) included:\n\n test/audio/clip_00293950f.wav \n test/audio/clip_0064f7bae.wav \n test/audio/clip_006ee8636.wav \n + 3706 other identical files\n\nSo, getting 3709 predictions right should be easy!\n\nThe Jupyter notebook below using R reads both the train and test files, so I did not attempt to post it as a kernel (only the train files are there, right?). Three csv output files are also included.", "251442": "Quite interesting. If u see in data page its written \"Not all of the files are evaluated for the leaderboard score.\". Could be a case of increasing test size to discourage manual labeling", "251526": "All these 3710 files seem to be all zero files (silence)", "255796": "Thanks for the feedback, Stas SI, about those 3710 duplicate files being all zeros. \n\nTo study files with small amplitude changes, like all the zero files, I added additional wav file metadata to the train and test \"inventory\" csv files, including amplitude quantiles [Q0 (min), Q10, Q25, Q50 (median), Q75, Q90, Q100 (max)], amplitude range, mean, sd. All the [updates are now on GitHub][1] (where I should have put the original files).\n\nThe density plots of the amplitude ranges for the train and test sets are not the same. The test set has more files at the extremes, including the 3709 test files that are all zeros, and a number of files with the full amplitude range for 16-bit sound (-32768 to 32767 = 65535). See attached density plots.\n\n [1]: https://github.com/EarlGlynn/kaggle-speech-recognition/tree/master/Jupyter/00-WAV-File-Inventory-Duplicates" }, "source": "meta" }