![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F5f4aa57d966bcfdcfd5ae476e0de552d%2FSelection_999(2807).png?generation=1690314890709034&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F76106326d9984e670608619c544173b9%2FSelection_999(2806).png?generation=1690314900191892&alt=media\")
\n \n\n
In @reasat bhai's best scoring Notebook, the audios in a batch were padded up to the maximum audio length of the batch during inference.
\n ():\n \n \n lengths = torch.tensor([ t.shape[] t batch ])\n \n batch = [ torch.Tensor(t) t batch ]\n batch = torch.nn.utils.rnn.pad_sequence(batch)\n \n mask = (batch != )\n batch, lengths, mask\nI wasn't sure whether this was the right idea since the audio length might vary a lot in batches (consider the case when a 1s audio is in the batch with a 10s audio). So I ran two experiments.
\npaths = (os(TEST_DIRECTORY,))\n\ntmp_df = pd({:paths.:paths})\ntmp_df = tmp_df(lambda x: (AudioSegment(x)))\npaths = tmp_df()()\nThus audios having same duration were batched together, so less padding was used comparatively than before. This increased the performance by the same amount, achieving LB 0.506
\nI wasn't able to check the CV score since the inference time is higher in both notebooks.
\nNow my question is, does zero padding actually decrease the inference performance of Wav2Vec2? Any ideas/comments?
", "messages": [ { "id": "2357910", "postDate": "07/25/2023 07:48:12", "content": "\n \n\n
In @reasat bhai's best scoring Notebook, the audios in a batch were padded up to the maximum audio length of the batch during inference.
\n ():\n \n \n lengths = torch.tensor([ t.shape[] t batch ])\n \n batch = [ torch.Tensor(t) t batch ]\n batch = torch.nn.utils.rnn.pad_sequence(batch)\n \n mask = (batch != )\n batch, lengths, mask\nI wasn't sure whether this was the right idea since the audio length might vary a lot in batches (consider the case when a 1s audio is in the batch with a 10s audio). So I ran two experiments.
\npaths = (os(TEST_DIRECTORY,))\n\ntmp_df = pd({:paths.:paths})\ntmp_df = tmp_df(lambda x: (AudioSegment(x)))\npaths = tmp_df()()\nThus audios having same duration were batched together, so less padding was used comparatively than before. This increased the performance by the same amount, achieving LB 0.506
\nI wasn't able to check the CV score since the inference time is higher in both notebooks.
\nNow my question is, does zero padding actually decrease the inference performance of Wav2Vec2? Any ideas/comments?
", "rawMarkdown": ">[Notebook with LB score 0.506](https://www.kaggle.com/code/mbmmurad/lb-0-506-inference-w-previous-comp-winner-s-model?scriptVersionId=137803947)\n\n\nIn @reasat bhai's best scoring [Notebook](https://www.kaggle.com/code/reasat/yellowking-dlsprint-inference), the audios in a batch were padded up to the maximum audio length of the batch during inference. \n\n```\ndef collate_fn_padd(batch):\n '''\n Padds batch of variable length\n\n note: it converts things ToTensor manually here since the ToTensor transform\n assume it takes in images rather than arbitrary tensors.\n '''\n ## get sequence lengths\n lengths = torch.tensor([ t.shape[0] for t in batch ])\n ## padd\n batch = [ torch.Tensor(t) for t in batch ]\n batch = torch.nn.utils.rnn.pad_sequence(batch)\n ## compute mask\n mask = (batch != 0)\n return batch, lengths, mask\n```\n\nI wasn't sure whether this was the right idea since the audio length might vary a lot in batches (consider the case when a 1s audio is in the batch with a 10s audio). So I ran two experiments. \n\n1. In [my notebook](https://www.kaggle.com/code/mbmmurad/lb-0-506-inference-w-previous-comp-winner-s-model?scriptVersionId=137803947) I passed each of the audios separately into the pipeline, thus not requiring to pad the audio, and the LB score improved to **0.506!**\n2. I sorted the audio paths by the duration of the audio files. \n```\npaths = glob(os.path.join(TEST_DIRECTORY,'*.mp3'))\n\ntmp_df = pd.DataFrame({\"id\":paths.\"duration\":paths})\ntmp_df[\"duration\"] = tmp_df[\"duration\"].apply(lambda x: len(AudioSegment.from_file(x)))\npaths = tmp_df.sort_values('duration')['id'].tolist()\n```\n\nThus audios having same duration were batched together, so less padding was used comparatively than before. This increased the performance by the same amount, achieving **LB 0.506**\n\nI wasn't able to check the CV score since the inference time is higher in both notebooks. \n\nNow my question is, does zero padding actually decrease the inference performance of Wav2Vec2? Any ideas/comments?", "votes": null }, { "id": "2358129", "postDate": "07/25/2023 10:00:20", "content": "batching results is almost always worse than single inference.
\nthis is also proven in local validation.
the reason is prediction of noise token at the end of wave (i.e. padding region)
\nfor example:
\nwith out pad:\npredict : \n\n\nwith pad\npredict : \n__\n____x___\n\nx \ni observe this when i debug my code. to verify if my padding code is correct or not, i compare results with and without batching.
", "rawMarkdown": "batching results is almost always worse than single inference.\nthis is also proven in local validation.\n\nthe reason is prediction of noise token at the end of wave (i.e. padding region)\n\nfor example:\n```\nwith out pad:\npredict : \n@mbmmurad
\nremove this to get 0.505:
submit_df.loc[:,'sentence'] = submit_df.sentence.apply(lambda x:dari(x))
", "rawMarkdown": "mbmmurad \nremove this to get 0.505:\n\nsubmit_df.loc[:,'sentence'] = submit_df.sentence.apply(lambda x:dari(x))", "votes": null }, { "id": "2358817", "postDate": "07/25/2023 18:41:37", "content": "Thanks for the insight. I think It'll be a dilemma between sacrifing a bit LB score vs Fast inference.
", "rawMarkdown": "Thanks for the insight. I think It'll be a dilemma between sacrifing a bit LB score vs Fast inference.", "votes": null }, { "id": "2358868", "postDate": "07/25/2023 19:56:04", "content": "i manage to find my old results and i will show it here
\n
\n
two noise predictions from batch inference. these are absent for single input inference.
", "rawMarkdown": "i manage to find my old results and i will show it here\n\n.png?generation=1690314890709034&alt=media)\n.png?generation=1690314900191892&alt=media)\n\ntwo noise predictions from batch inference. these are absent for single input inference.", "votes": null }, { "id": "2359889", "postDate": "07/26/2023 12:18:50", "content": "\nI checked in local which results are:
\nI reviewed your notebooks and find out that you also process .wav files in test folder. Other public notebooks only process .mp3 files in test folder. So, I think the reason that your notebook has score of 0.506 is that you process .wav files in test folder.
", "rawMarkdown": "mbmmurad \n\nI checked in local which results are:\n+ Single infer is worse than batch infer\n\nI reviewed your notebooks and find out that you also process .wav files in test folder. Other public notebooks only process .mp3 files in test folder. So, I think the reason that your notebook has score of 0.506 is that you process .wav files in test folder.", "votes": null } ], "comments": [ { "id": 2358129, "author_name": "hengck23", "author_url": "", "post_date": "07/25/2023 10:00:20", "content": "batching results is almost always worse than single inference.
\nthis is also proven in local validation.
the reason is prediction of noise token at the end of wave (i.e. padding region)
\nfor example:
\nwith out pad:\npredict : \n\n\nwith pad\npredict : \n__\n____x___\n\nx \ni observe this when i debug my code. to verify if my padding code is correct or not, i compare results with and without batching.
", "votes": null, "replies": [ { "id": 2358817, "author_name": "mbmmurad", "author_url": "", "post_date": "07/25/2023 18:41:37", "content": "Thanks for the insight. I think It'll be a dilemma between sacrifing a bit LB score vs Fast inference.
", "votes": null, "replies": [] }, { "id": 2358868, "author_name": "hengck23", "author_url": "", "post_date": "07/25/2023 19:56:04", "content": "i manage to find my old results and i will show it here
\n
\n
two noise predictions from batch inference. these are absent for single input inference.
", "votes": null, "replies": [] } ] }, { "id": 2358234, "author_name": "hengck23", "author_url": "", "post_date": "07/25/2023 11:27:31", "content": "@mbmmurad
\nremove this to get 0.505:
submit_df.loc[:,'sentence'] = submit_df.sentence.apply(lambda x:dari(x))
", "votes": null, "replies": [] }, { "id": 2359889, "author_name": "texopher", "author_url": "", "post_date": "07/26/2023 12:18:50", "content": "\nI checked in local which results are:
\nI reviewed your notebooks and find out that you also process .wav files in test folder. Other public notebooks only process .mp3 files in test folder. So, I think the reason that your notebook has score of 0.506 is that you process .wav files in test folder.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2357910": ">[Notebook with LB score 0.506](https://www.kaggle.com/code/mbmmurad/lb-0-506-inference-w-previous-comp-winner-s-model?scriptVersionId=137803947)\n\n\nIn @reasat bhai's best scoring [Notebook](https://www.kaggle.com/code/reasat/yellowking-dlsprint-inference), the audios in a batch were padded up to the maximum audio length of the batch during inference. \n\n```\ndef collate_fn_padd(batch):\n '''\n Padds batch of variable length\n\n note: it converts things ToTensor manually here since the ToTensor transform\n assume it takes in images rather than arbitrary tensors.\n '''\n ## get sequence lengths\n lengths = torch.tensor([ t.shape[0] for t in batch ])\n ## padd\n batch = [ torch.Tensor(t) for t in batch ]\n batch = torch.nn.utils.rnn.pad_sequence(batch)\n ## compute mask\n mask = (batch != 0)\n return batch, lengths, mask\n```\n\nI wasn't sure whether this was the right idea since the audio length might vary a lot in batches (consider the case when a 1s audio is in the batch with a 10s audio). So I ran two experiments. \n\n1. In [my notebook](https://www.kaggle.com/code/mbmmurad/lb-0-506-inference-w-previous-comp-winner-s-model?scriptVersionId=137803947) I passed each of the audios separately into the pipeline, thus not requiring to pad the audio, and the LB score improved to **0.506!**\n2. I sorted the audio paths by the duration of the audio files. \n```\npaths = glob(os.path.join(TEST_DIRECTORY,'*.mp3'))\n\ntmp_df = pd.DataFrame({\"id\":paths.\"duration\":paths})\ntmp_df[\"duration\"] = tmp_df[\"duration\"].apply(lambda x: len(AudioSegment.from_file(x)))\npaths = tmp_df.sort_values('duration')['id'].tolist()\n```\n\nThus audios having same duration were batched together, so less padding was used comparatively than before. This increased the performance by the same amount, achieving **LB 0.506**\n\nI wasn't able to check the CV score since the inference time is higher in both notebooks. \n\nNow my question is, does zero padding actually decrease the inference performance of Wav2Vec2? Any ideas/comments?", "2358129": "batching results is almost always worse than single inference.\nthis is also proven in local validation.\n\nthe reason is prediction of noise token at the end of wave (i.e. padding region)\n\nfor example:\n```\nwith out pad:\npredict : \n