Has anybody faced the same issue? My kernel runs successfully on public validation set, but fails on public test set.
", "messages": [ { "id": "717921", "postDate": "01/13/2020 19:33:52", "content": "Has anybody faced the same issue? My kernel runs successfully on public validation set, but fails on public test set.
", "rawMarkdown": "Has anybody faced the same issue? My kernel runs successfully on public validation set, but fails on public test set.", "votes": null }, { "id": "717936", "postDate": "01/13/2020 20:05:43", "content": "Yes - I created simple kernel that performs the steps I plan to take - but just does a random probability to generate my submission. I do extract the audio as the only real work.
\n\nKernel always works when run step by step. When \"committed\" it never \"completed\". Closing and returning to edit always found that the kernel had completed in 70 to 80 seconds - did all it was suppose to do.
\n\nI killed off all of the folders and files created during the script - and kind of got the kernel so it does \"complete\" - but the complete message takes an hour to arrive rather than the 76 seconds that is the indicated run time when I reopen later. Submission fails.
\n\nI think my issue is related to using ffmpeg to extract the audio. There are two public kernels using ffmpeg to extract - when I fork those and \"commit\" they also never show \"completed\". But like my kernel, if closed and than opened - the commit seems to have run fine and has a run time that makes sense.
\n\nCommon causes of submission errors I have seen in discussion -\n1. You create more files than the kaggle limit (never have found where the limit is stated - but..).
\n2. You don't error trap reading the test files and there may be a bad file in the private test set.\n3. You don't remove all the stuff you have created (except for the submission) - this restriction appears to be aimed at stopping leader board probing by not letting any files remain in the output except the submission.\n4. You predict a probability of 1.0 and the log loss goes crazy. (or 0.0)\n5. You predict a probability greater than 1.0 and log loss goes crazy. (or less than 0.0)\n6. You kernel takes several hours on the public test and than runs over time on the private - since mine takes only 76 seconds this not my current issue.
Going back and re-doing my efforts for first three - but need to wait for a new day to have two submissions. Will also make sure I have #4 and 5 covered. Guess I will also try a submission without creating audio file in ffmpeg - already tried and found out that not creating an audio file does result in a \"commit\" that completes in the couple of minutes that it should.
\n\nThis competition is a real challenge - I did not expect that I would need to spend 5 days (and counting) just getting a random answer submission to work.
", "rawMarkdown": "Yes - I created simple kernel that performs the steps I plan to take - but just does a random probability to generate my submission. I do extract the audio as the only real work.\n\nKernel always works when run step by step. When \"committed\" it never \"completed\". Closing and returning to edit always found that the kernel had completed in 70 to 80 seconds - did all it was suppose to do.\n\nI killed off all of the folders and files created during the script - and kind of got the kernel so it does \"complete\" - but the complete message takes an hour to arrive rather than the 76 seconds that is the indicated run time when I reopen later. Submission fails.\n\nI think my issue is related to using ffmpeg to extract the audio. There are two public kernels using ffmpeg to extract - when I fork those and \"commit\" they also never show \"completed\". But like my kernel, if closed and than opened - the commit seems to have run fine and has a run time that makes sense. \n\nCommon causes of submission errors I have seen in discussion -\n1. You create more files than the kaggle limit (never have found where the limit is stated - but..). \n2. You don't error trap reading the test files and there may be a bad file in the private test set.\n3. You don't remove all the stuff you have created (except for the submission) - this restriction appears to be aimed at stopping leader board probing by not letting any files remain in the output except the submission.\n4. You predict a probability of 1.0 and the log loss goes crazy. (or 0.0)\n5. You predict a probability greater than 1.0 and log loss goes crazy. (or less than 0.0)\n6. You kernel takes several hours on the public test and than runs over time on the private - since mine takes only 76 seconds this not my current issue.\n\n\nGoing back and re-doing my efforts for first three - but need to wait for a new day to have two submissions. Will also make sure I have #4 and 5 covered. Guess I will also try a submission without creating audio file in ffmpeg - already tried and found out that not creating an audio file does result in a \"commit\" that completes in the couple of minutes that it should.\n\nThis competition is a real challenge - I did not expect that I would need to spend 5 days (and counting) just getting a random answer submission to work.", "votes": null }, { "id": "718046", "postDate": "01/13/2020 23:27:37", "content": "Great post, thank you @pcjimmmy . It is a shame we need to spend so much time on making the submissions work.
\n\nA couple of remarks on your points
\n\nI believe it is max 500 output files, it is mentioned if you try to create more than 500 files already for the 400 validation videos.
Following the forum and my own submissions, I don't think there is an indication yet that there is a corrupted file in the public test set. It is something else, like using ffmpeg.
4,5. The numbers that we predict are clipped to [1e-15; 1-1e-15], so this can not be a problem
", "rawMarkdown": "Great post, thank you @pcjimmmy . It is a shame we need to spend so much time on making the submissions work.\n\nA couple of remarks on your points\n\n1. I believe it is max 500 output files, it is mentioned if you try to create more than 500 files already for the 400 validation videos.\n\n2. Following the forum and my own submissions, I don't think there is an indication yet that there is a corrupted file in the public test set. It is something else, like using ffmpeg.\n\n4,5. The numbers that we predict are clipped to [1e-15; 1-1e-15], so this can not be a problem", "votes": null }, { "id": "718065", "postDate": "01/14/2020 00:24:47", "content": "I haven't had anything worth submitting yet but I dread the process. Thanks to all who are pioneering this.
", "rawMarkdown": "I haven't had anything worth submitting yet but I dread the process. Thanks to all who are pioneering this.", "votes": null }, { "id": "719005", "postDate": "01/15/2020 02:35:56", "content": "Thanks! Im facing the same exact issue. Will try to follow each steps and resubmit my random test submission.
", "rawMarkdown": "Thanks! Im facing the same exact issue. Will try to follow each steps and resubmit my random test submission.", "votes": null }, { "id": "719038", "postDate": "01/15/2020 03:52:25", "content": "If not for the $500K prize (which I have no chance of winning) this competition would be history for me. Another day - another two submission errors. If I was actually making a valid prediction than I might have some happiness once I get a submission to work - but I am only doing a random number prediction - if I ever get a submission to generate a score I only find out what random guesses will achieve.
\n\nTried to use a more current ffmpeg build - no help.\nAdded another layer of error trapping for creation of audio files.\nChanged the logic flow - if there was a bad file in the private test it would have resulted in error in the number of files for the submission.
\nOne more day of ideas and than I guess I will bail out on the audio - most libraries seem to use ffmpeg as the base.
With only one year of Python under my belt my skill level is such that my path to correcting errors most often starts with google search.
", "rawMarkdown": "If not for the $500K prize (which I have no chance of winning) this competition would be history for me. Another day - another two submission errors. If I was actually making a valid prediction than I might have some happiness once I get a submission to work - but I am only doing a random number prediction - if I ever get a submission to generate a score I only find out what random guesses will achieve.\n\nTried to use a more current ffmpeg build - no help.\nAdded another layer of error trapping for creation of audio files.\nChanged the logic flow - if there was a bad file in the private test it would have resulted in error in the number of files for the submission. \nOne more day of ideas and than I guess I will bail out on the audio - most libraries seem to use ffmpeg as the base.\n\nWith only one year of Python under my belt my skill level is such that my path to correcting errors most often starts with google search.", "votes": null }, { "id": "719634", "postDate": "01/15/2020 17:08:48", "content": "I gave up on getting ffmpeg to work - I can get a successful submission if I install but never use ffmpeg. As soon as I use it to extract an audio file(s) the follow happens - (it appears to work with NO issues inside running the kernel step by step or Run All).
\n\nHowever\n1. When I commit - it never stops running. (ok maybe never is stretch - the kernel should run for less than 80 seconds) Going to try commit and than go for a long walk with my dog - maybe after couple of hours it does report completed.\n2. If I close the kernel after commit (waiting a respectful time for it to really complete on its own) - come back in - the commit appears to have been successful - took about the 80 seconds expected. Ran all the way thru the script and generates a submission file.\n3. If I than submit the submission file - I waste one. Error message suggesting that the submission file is fricked.
\n\nSupport is black hole - requests go in - a single automated response comes out. Apparently you get what you pay for - paid nothing - got nothing.
", "rawMarkdown": "I gave up on getting ffmpeg to work - I can get a successful submission if I install but never use ffmpeg. As soon as I use it to extract an audio file(s) the follow happens - (it appears to work with NO issues inside running the kernel step by step or Run All). \n\nHowever\n1. When I commit - it never stops running. (ok maybe never is stretch - the kernel should run for less than 80 seconds) Going to try commit and than go for a long walk with my dog - maybe after couple of hours it does report completed.\n2. If I close the kernel after commit (waiting a respectful time for it to really complete on its own) - come back in - the commit appears to have been successful - took about the 80 seconds expected. Ran all the way thru the script and generates a submission file.\n3. If I than submit the submission file - I waste one. Error message suggesting that the submission file is fricked.\n\nSupport is black hole - requests go in - a single automated response comes out. Apparently you get what you pay for - paid nothing - got nothing.", "votes": null }, { "id": "725111", "postDate": "01/21/2020 20:20:06", "content": "Hey,
\n\nCould you please share your random kernel. I just want to know how input should be taken and where output would be generated.
\n\nShould our model look for: \"test_videos.zip\" or something else? Will it be in the same directory?
", "rawMarkdown": "Hey,\n\nCould you please share your random kernel. I just want to know how input should be taken and where output would be generated.\n\nShould our model look for: \"test_videos.zip\" or something else? Will it be in the same directory?", "votes": null } ], "comments": [ { "id": 717936, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/13/2020 20:05:43", "content": "Yes - I created simple kernel that performs the steps I plan to take - but just does a random probability to generate my submission. I do extract the audio as the only real work.
\n\nKernel always works when run step by step. When \"committed\" it never \"completed\". Closing and returning to edit always found that the kernel had completed in 70 to 80 seconds - did all it was suppose to do.
\n\nI killed off all of the folders and files created during the script - and kind of got the kernel so it does \"complete\" - but the complete message takes an hour to arrive rather than the 76 seconds that is the indicated run time when I reopen later. Submission fails.
\n\nI think my issue is related to using ffmpeg to extract the audio. There are two public kernels using ffmpeg to extract - when I fork those and \"commit\" they also never show \"completed\". But like my kernel, if closed and than opened - the commit seems to have run fine and has a run time that makes sense.
\n\nCommon causes of submission errors I have seen in discussion -\n1. You create more files than the kaggle limit (never have found where the limit is stated - but..).
\n2. You don't error trap reading the test files and there may be a bad file in the private test set.\n3. You don't remove all the stuff you have created (except for the submission) - this restriction appears to be aimed at stopping leader board probing by not letting any files remain in the output except the submission.\n4. You predict a probability of 1.0 and the log loss goes crazy. (or 0.0)\n5. You predict a probability greater than 1.0 and log loss goes crazy. (or less than 0.0)\n6. You kernel takes several hours on the public test and than runs over time on the private - since mine takes only 76 seconds this not my current issue.
Going back and re-doing my efforts for first three - but need to wait for a new day to have two submissions. Will also make sure I have #4 and 5 covered. Guess I will also try a submission without creating audio file in ffmpeg - already tried and found out that not creating an audio file does result in a \"commit\" that completes in the couple of minutes that it should.
\n\nThis competition is a real challenge - I did not expect that I would need to spend 5 days (and counting) just getting a random answer submission to work.
", "votes": null, "replies": [ { "id": 718046, "author_name": "zaharch", "author_url": "", "post_date": "01/13/2020 23:27:37", "content": "Great post, thank you @pcjimmmy . It is a shame we need to spend so much time on making the submissions work.
\n\nA couple of remarks on your points
\n\nI believe it is max 500 output files, it is mentioned if you try to create more than 500 files already for the 400 validation videos.
Following the forum and my own submissions, I don't think there is an indication yet that there is a corrupted file in the public test set. It is something else, like using ffmpeg.
4,5. The numbers that we predict are clipped to [1e-15; 1-1e-15], so this can not be a problem
", "votes": null, "replies": [] }, { "id": 719005, "author_name": "juanumusic", "author_url": "", "post_date": "01/15/2020 02:35:56", "content": "Thanks! Im facing the same exact issue. Will try to follow each steps and resubmit my random test submission.
", "votes": null, "replies": [] }, { "id": 719038, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/15/2020 03:52:25", "content": "If not for the $500K prize (which I have no chance of winning) this competition would be history for me. Another day - another two submission errors. If I was actually making a valid prediction than I might have some happiness once I get a submission to work - but I am only doing a random number prediction - if I ever get a submission to generate a score I only find out what random guesses will achieve.
\n\nTried to use a more current ffmpeg build - no help.\nAdded another layer of error trapping for creation of audio files.\nChanged the logic flow - if there was a bad file in the private test it would have resulted in error in the number of files for the submission.
\nOne more day of ideas and than I guess I will bail out on the audio - most libraries seem to use ffmpeg as the base.
With only one year of Python under my belt my skill level is such that my path to correcting errors most often starts with google search.
", "votes": null, "replies": [] } ] }, { "id": 718065, "author_name": "petewills", "author_url": "", "post_date": "01/14/2020 00:24:47", "content": "I haven't had anything worth submitting yet but I dread the process. Thanks to all who are pioneering this.
", "votes": null, "replies": [] }, { "id": 719634, "author_name": "pcjimmmy", "author_url": "", "post_date": "01/15/2020 17:08:48", "content": "I gave up on getting ffmpeg to work - I can get a successful submission if I install but never use ffmpeg. As soon as I use it to extract an audio file(s) the follow happens - (it appears to work with NO issues inside running the kernel step by step or Run All).
\n\nHowever\n1. When I commit - it never stops running. (ok maybe never is stretch - the kernel should run for less than 80 seconds) Going to try commit and than go for a long walk with my dog - maybe after couple of hours it does report completed.\n2. If I close the kernel after commit (waiting a respectful time for it to really complete on its own) - come back in - the commit appears to have been successful - took about the 80 seconds expected. Ran all the way thru the script and generates a submission file.\n3. If I than submit the submission file - I waste one. Error message suggesting that the submission file is fricked.
\n\nSupport is black hole - requests go in - a single automated response comes out. Apparently you get what you pay for - paid nothing - got nothing.
", "votes": null, "replies": [] }, { "id": 725111, "author_name": "aknirala", "author_url": "", "post_date": "01/21/2020 20:20:06", "content": "Hey,
\n\nCould you please share your random kernel. I just want to know how input should be taken and where output would be generated.
\n\nShould our model look for: \"test_videos.zip\" or something else? Will it be in the same directory?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "717921": "Has anybody faced the same issue? My kernel runs successfully on public validation set, but fails on public test set.", "717936": "Yes - I created simple kernel that performs the steps I plan to take - but just does a random probability to generate my submission. I do extract the audio as the only real work.\n\nKernel always works when run step by step. When \"committed\" it never \"completed\". Closing and returning to edit always found that the kernel had completed in 70 to 80 seconds - did all it was suppose to do.\n\nI killed off all of the folders and files created during the script - and kind of got the kernel so it does \"complete\" - but the complete message takes an hour to arrive rather than the 76 seconds that is the indicated run time when I reopen later. Submission fails.\n\nI think my issue is related to using ffmpeg to extract the audio. There are two public kernels using ffmpeg to extract - when I fork those and \"commit\" they also never show \"completed\". But like my kernel, if closed and than opened - the commit seems to have run fine and has a run time that makes sense. \n\nCommon causes of submission errors I have seen in discussion -\n1. You create more files than the kaggle limit (never have found where the limit is stated - but..). \n2. You don't error trap reading the test files and there may be a bad file in the private test set.\n3. You don't remove all the stuff you have created (except for the submission) - this restriction appears to be aimed at stopping leader board probing by not letting any files remain in the output except the submission.\n4. You predict a probability of 1.0 and the log loss goes crazy. (or 0.0)\n5. You predict a probability greater than 1.0 and log loss goes crazy. (or less than 0.0)\n6. You kernel takes several hours on the public test and than runs over time on the private - since mine takes only 76 seconds this not my current issue.\n\n\nGoing back and re-doing my efforts for first three - but need to wait for a new day to have two submissions. Will also make sure I have #4 and 5 covered. Guess I will also try a submission without creating audio file in ffmpeg - already tried and found out that not creating an audio file does result in a \"commit\" that completes in the couple of minutes that it should.\n\nThis competition is a real challenge - I did not expect that I would need to spend 5 days (and counting) just getting a random answer submission to work.", "718046": "Great post, thank you @pcjimmmy . It is a shame we need to spend so much time on making the submissions work.\n\nA couple of remarks on your points\n\n1. I believe it is max 500 output files, it is mentioned if you try to create more than 500 files already for the 400 validation videos.\n\n2. Following the forum and my own submissions, I don't think there is an indication yet that there is a corrupted file in the public test set. It is something else, like using ffmpeg.\n\n4,5. The numbers that we predict are clipped to [1e-15; 1-1e-15], so this can not be a problem", "718065": "I haven't had anything worth submitting yet but I dread the process. Thanks to all who are pioneering this.", "719005": "Thanks! Im facing the same exact issue. Will try to follow each steps and resubmit my random test submission.", "719038": "If not for the $500K prize (which I have no chance of winning) this competition would be history for me. Another day - another two submission errors. If I was actually making a valid prediction than I might have some happiness once I get a submission to work - but I am only doing a random number prediction - if I ever get a submission to generate a score I only find out what random guesses will achieve.\n\nTried to use a more current ffmpeg build - no help.\nAdded another layer of error trapping for creation of audio files.\nChanged the logic flow - if there was a bad file in the private test it would have resulted in error in the number of files for the submission. \nOne more day of ideas and than I guess I will bail out on the audio - most libraries seem to use ffmpeg as the base.\n\nWith only one year of Python under my belt my skill level is such that my path to correcting errors most often starts with google search.", "719634": "I gave up on getting ffmpeg to work - I can get a successful submission if I install but never use ffmpeg. As soon as I use it to extract an audio file(s) the follow happens - (it appears to work with NO issues inside running the kernel step by step or Run All). \n\nHowever\n1. When I commit - it never stops running. (ok maybe never is stretch - the kernel should run for less than 80 seconds) Going to try commit and than go for a long walk with my dog - maybe after couple of hours it does report completed.\n2. If I close the kernel after commit (waiting a respectful time for it to really complete on its own) - come back in - the commit appears to have been successful - took about the 80 seconds expected. Ran all the way thru the script and generates a submission file.\n3. If I than submit the submission file - I waste one. Error message suggesting that the submission file is fricked.\n\nSupport is black hole - requests go in - a single automated response comes out. Apparently you get what you pay for - paid nothing - got nothing.", "725111": "Hey,\n\nCould you please share your random kernel. I just want to know how input should be taken and where output would be generated.\n\nShould our model look for: \"test_videos.zip\" or something else? Will it be in the same directory?" }, "source": "meta" }