When I submit an audio classifier, the Submission Scoring Error returned for many times...
So I use simply use ffmpeg to extract audios on test set. But Submission Scoring Error still exist...
There is my code, I cant find anywhere cause the 'Submission Scoring Error'
\n\n```python\nimport soundfile as sf\nimport torch\nimport math\nimport numpy as np\nimport os\nfrom pathlib import Path\nimport subprocess\nimport pandas as pd
\n\noutput_dir = Path(f\"wavs\")\nPath(output_dir).mkdir(exist_ok=True, parents=True)\nsubmit_ = []\nfor vi in os.listdir('/kaggle/input/deepfake-detection-challenge/test_videos'):\n sub_list = [vi,0.5]\n video_path = os.path.join(\"/kaggle/input/deepfake-detection-challenge/test_videos\", vi)\n command = f\"../working/ffmpeg-git-20191209-amd64-static/ffmpeg -i {video_path} -ab 192000 -ac 2 -ar 44100 -vn {output_dir/vi[:-4]}.wav\"\n subprocess.call(command, shell=True)\n if not os.path.exists(os.path.join(output_dir, vi[:-4] + '.wav')):\n submit_.append(sub_list)\n continue\n submit_.append(sub_list)
\n\nsubmission = pd.DataFrame(submit_, columns=['filename', 'label'])\nsubmission.sort_values('filename').to_csv('submission.csv', index=False)\n```
\n\nWish you can give me some advices.
", "messages": [ { "id": "710721", "postDate": "01/05/2020 05:58:43", "content": "When I submit an audio classifier, the Submission Scoring Error returned for many times...
So I use simply use ffmpeg to extract audios on test set. But Submission Scoring Error still exist...
There is my code, I cant find anywhere cause the 'Submission Scoring Error'
\n\n```python\nimport soundfile as sf\nimport torch\nimport math\nimport numpy as np\nimport os\nfrom pathlib import Path\nimport subprocess\nimport pandas as pd
\n\noutput_dir = Path(f\"wavs\")\nPath(output_dir).mkdir(exist_ok=True, parents=True)\nsubmit_ = []\nfor vi in os.listdir('/kaggle/input/deepfake-detection-challenge/test_videos'):\n sub_list = [vi,0.5]\n video_path = os.path.join(\"/kaggle/input/deepfake-detection-challenge/test_videos\", vi)\n command = f\"../working/ffmpeg-git-20191209-amd64-static/ffmpeg -i {video_path} -ab 192000 -ac 2 -ar 44100 -vn {output_dir/vi[:-4]}.wav\"\n subprocess.call(command, shell=True)\n if not os.path.exists(os.path.join(output_dir, vi[:-4] + '.wav')):\n submit_.append(sub_list)\n continue\n submit_.append(sub_list)
\n\nsubmission = pd.DataFrame(submit_, columns=['filename', 'label'])\nsubmission.sort_values('filename').to_csv('submission.csv', index=False)\n```
\n\nWish you can give me some advices.
", "rawMarkdown": "When I submit an audio classifier, the `Submission Scoring Error` returned for many times...\n\nSo I use simply use `ffmpeg` to extract audios on test set. But `Submission Scoring Error` still exist...\n\nThere is my code, I cant find anywhere cause the 'Submission Scoring Error'\n\n```python\nimport soundfile as sf\nimport torch\nimport math\nimport numpy as np\nimport os\nfrom pathlib import Path\nimport subprocess\nimport pandas as pd\n\noutput_dir = Path(f\"wavs\")\nPath(output_dir).mkdir(exist_ok=True, parents=True)\nsubmit_ = []\nfor vi in os.listdir('/kaggle/input/deepfake-detection-challenge/test_videos'):\n sub_list = [vi,0.5]\n video_path = os.path.join(\"/kaggle/input/deepfake-detection-challenge/test_videos\", vi)\n command = f\"../working/ffmpeg-git-20191209-amd64-static/ffmpeg -i {video_path} -ab 192000 -ac 2 -ar 44100 -vn {output_dir/vi[:-4]}.wav\"\n subprocess.call(command, shell=True)\n if not os.path.exists(os.path.join(output_dir, vi[:-4] + '.wav')):\n submit_.append(sub_list)\n continue\n submit_.append(sub_list)\n\nsubmission = pd.DataFrame(submit_, columns=['filename', 'label'])\nsubmission.sort_values('filename').to_csv('submission.csv', index=False)\n```\n\nWish you can give me some advices.", "votes": null }, { "id": "710748", "postDate": "01/05/2020 06:56:56", "content": "did you load ffmpg and just not show us that in your code?
\n\nEasier for folks to help if you have a public kernel with all the code so I don't have to ask what might be this dumb question
\n\n! tar xvf ../input/ffmpeg-static-build/ffmpeg-git-amd64-static.tar.xz
", "rawMarkdown": "did you load ffmpg and just not show us that in your code?\n\nEasier for folks to help if you have a public kernel with all the code so I don't have to ask what might be this dumb question\n\n! tar xvf ../input/ffmpeg-static-build/ffmpeg-git-amd64-static.tar.xz", "votes": null }, { "id": "710757", "postDate": "01/05/2020 07:12:13", "content": "public kernel is at: https://www.kaggle.com/xiaofengmao/kernel553c006564/notebook\nAs my code shows, I just define the model, and not use any model to give a prediction. Instead I use 0.5 as my submission score.\nSo this report error Submission Scoring Error puzzled me a lot. I cannot find anywhere to report this error.
Hmmm - can't run a fork to try as the first three cells are inputs that don't exist on my fork. Don't see any obvious.
", "rawMarkdown": "Hmmm - can't run a fork to try as the first three cells are inputs that don't exist on my fork. Don't see any obvious.", "votes": null }, { "id": "710763", "postDate": "01/05/2020 07:30:38", "content": "Sorry that they are from my private dataset. \nTry this clean kernel:\nhttps://www.kaggle.com/xiaofengmao/kernel553c006564\nI delete these dataset here.
", "rawMarkdown": "Sorry that they are from my private dataset. \nTry this clean kernel:\nhttps://www.kaggle.com/xiaofengmao/kernel553c006564\nI delete these dataset here.", "votes": null }, { "id": "711268", "postDate": "01/05/2020 21:32:55", "content": "OK - it ran ok on the commit. Just submitted now - guessing it might be an hour or so before I know if success. One observation that I want to check out - a lot of files get written during the process - I wonder if the 10X increase in the files being evaluated during the submission leads to using more HDD than we are allowed.
", "rawMarkdown": "OK - it ran ok on the commit. Just submitted now - guessing it might be an hour or so before I know if success. One observation that I want to check out - a lot of files get written during the process - I wonder if the 10X increase in the files being evaluated during the submission leads to using more HDD than we are allowed.", "votes": null }, { "id": "711269", "postDate": "01/05/2020 21:34:49", "content": "Got your same error - looks like error occurred about 6 minutes in.
", "rawMarkdown": "Got your same error - looks like error occurred about 6 minutes in.", "votes": null }, { "id": "711271", "postDate": "01/05/2020 21:41:49", "content": "Size of written 158 mb - so 10X might be 1580 mb - think we are allowed 5GB during runs so this not it.
", "rawMarkdown": "Size of written 158 mb - so 10X might be 1580 mb - think we are allowed 5GB during runs so this not it.", "votes": null }, { "id": "711274", "postDate": "01/05/2020 21:46:33", "content": "There is also a limit on number of output files, don't remember exactly, something like 500. Try cleaning with shutil.rmtree()
Thank you @zaharch @pcjimmmy \nI add os.remove() at end of the loop to clean generated .wav. Now it works ! There exactly have a limitation on number of output files. I forget to clean them during my kernel running.
Glad you got it - nice to learn about the limit on output files !
\n\nJust hit submit myself - interested in how long this loop of yours takes on the full test.
", "rawMarkdown": "Glad you got it - nice to learn about the limit on output files ! \n\nJust hit submit myself - interested in how long this loop of yours takes on the full test.", "votes": null } ], "comments": [ { "id": 710748, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/05/2020 06:56:56", "content": "did you load ffmpg and just not show us that in your code?
\n\nEasier for folks to help if you have a public kernel with all the code so I don't have to ask what might be this dumb question
\n\n! tar xvf ../input/ffmpeg-static-build/ffmpeg-git-amd64-static.tar.xz
", "votes": null, "replies": [ { "id": 710757, "author_name": "xiaofengmao", "author_url": "", "post_date": "01/05/2020 07:12:13", "content": "public kernel is at: https://www.kaggle.com/xiaofengmao/kernel553c006564/notebook\nAs my code shows, I just define the model, and not use any model to give a prediction. Instead I use 0.5 as my submission score.\nSo this report error Submission Scoring Error puzzled me a lot. I cannot find anywhere to report this error.
Hmmm - can't run a fork to try as the first three cells are inputs that don't exist on my fork. Don't see any obvious.
", "votes": null, "replies": [] }, { "id": 710763, "author_name": "xiaofengmao", "author_url": "", "post_date": "01/05/2020 07:30:38", "content": "Sorry that they are from my private dataset. \nTry this clean kernel:\nhttps://www.kaggle.com/xiaofengmao/kernel553c006564\nI delete these dataset here.
", "votes": null, "replies": [] }, { "id": 711268, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/05/2020 21:32:55", "content": "OK - it ran ok on the commit. Just submitted now - guessing it might be an hour or so before I know if success. One observation that I want to check out - a lot of files get written during the process - I wonder if the 10X increase in the files being evaluated during the submission leads to using more HDD than we are allowed.
", "votes": null, "replies": [] }, { "id": 711269, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/05/2020 21:34:49", "content": "Got your same error - looks like error occurred about 6 minutes in.
", "votes": null, "replies": [] }, { "id": 711271, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/05/2020 21:41:49", "content": "Size of written 158 mb - so 10X might be 1580 mb - think we are allowed 5GB during runs so this not it.
", "votes": null, "replies": [] }, { "id": 711274, "author_name": "zaharch", "author_url": "", "post_date": "01/05/2020 21:46:33", "content": "There is also a limit on number of output files, don't remember exactly, something like 500. Try cleaning with shutil.rmtree()
Thank you @zaharch @pcjimmmy \nI add os.remove() at end of the loop to clean generated .wav. Now it works ! There exactly have a limitation on number of output files. I forget to clean them during my kernel running.
Glad you got it - nice to learn about the limit on output files !
\n\nJust hit submit myself - interested in how long this loop of yours takes on the full test.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "710721": "When I submit an audio classifier, the `Submission Scoring Error` returned for many times...\n\nSo I use simply use `ffmpeg` to extract audios on test set. But `Submission Scoring Error` still exist...\n\nThere is my code, I cant find anywhere cause the 'Submission Scoring Error'\n\n```python\nimport soundfile as sf\nimport torch\nimport math\nimport numpy as np\nimport os\nfrom pathlib import Path\nimport subprocess\nimport pandas as pd\n\noutput_dir = Path(f\"wavs\")\nPath(output_dir).mkdir(exist_ok=True, parents=True)\nsubmit_ = []\nfor vi in os.listdir('/kaggle/input/deepfake-detection-challenge/test_videos'):\n sub_list = [vi,0.5]\n video_path = os.path.join(\"/kaggle/input/deepfake-detection-challenge/test_videos\", vi)\n command = f\"../working/ffmpeg-git-20191209-amd64-static/ffmpeg -i {video_path} -ab 192000 -ac 2 -ar 44100 -vn {output_dir/vi[:-4]}.wav\"\n subprocess.call(command, shell=True)\n if not os.path.exists(os.path.join(output_dir, vi[:-4] + '.wav')):\n submit_.append(sub_list)\n continue\n submit_.append(sub_list)\n\nsubmission = pd.DataFrame(submit_, columns=['filename', 'label'])\nsubmission.sort_values('filename').to_csv('submission.csv', index=False)\n```\n\nWish you can give me some advices.", "710748": "did you load ffmpg and just not show us that in your code?\n\nEasier for folks to help if you have a public kernel with all the code so I don't have to ask what might be this dumb question\n\n! tar xvf ../input/ffmpeg-static-build/ffmpeg-git-amd64-static.tar.xz", "710757": "public kernel is at: https://www.kaggle.com/xiaofengmao/kernel553c006564/notebook\nAs my code shows, I just define the model, and not use any model to give a prediction. Instead I use 0.5 as my submission score.\nSo this report error `Submission Scoring Error` puzzled me a lot. I cannot find anywhere to report this error.", "710762": "Hmmm - can't run a fork to try as the first three cells are inputs that don't exist on my fork. Don't see any obvious.", "710763": "Sorry that they are from my private dataset. \nTry this clean kernel:\nhttps://www.kaggle.com/xiaofengmao/kernel553c006564\nI delete these dataset here.", "711268": "OK - it ran ok on the commit. Just submitted now - guessing it might be an hour or so before I know if success. One observation that I want to check out - a lot of files get written during the process - I wonder if the 10X increase in the files being evaluated during the submission leads to using more HDD than we are allowed.", "711269": "Got your same error - looks like error occurred about 6 minutes in.", "711271": "Size of written 158 mb - so 10X might be 1580 mb - think we are allowed 5GB during runs so this not it.", "711274": "There is also a limit on number of output files, don't remember exactly, something like 500. Try cleaning with `shutil.rmtree()`", "711386": "Thank you @zaharch @pcjimmmy \nI add `os.remove()` at end of the loop to clean generated `.wav`. Now it works ! There exactly have a limitation on number of output files. I forget to clean them during my kernel running.", "711848": "Glad you got it - nice to learn about the limit on output files ! \n\nJust hit submit myself - interested in how long this loop of yours takes on the full test." }, "source": "meta" }