![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1985738%2F7efdaf1430566393553d907320cd0f05%2FScreenshot%20from%202019-12-18%2004-58-54.png?generation=1576663192278282&alt=media\")
![]I have 2 notebooks that Commits without errors but when I Submit I got an error: \"Submission CSV Not Found\". (I suppose there is an error in the notebook before I write the submission.csv file)\n Anyone having the same issue?
\n\n@admin: I trained a model using train_sample_videos, is it available at submission time? (it could be the reason to fail only at submission time)
I have 2 notebooks that Commits without errors but when I Submit I got an error: \"Submission CSV Not Found\". (I suppose there is an error in the notebook before I write the submission.csv file)\n Anyone having the same issue?
\n\n@admin: I trained a model using train_sample_videos, is it available at submission time? (it could be the reason to fail only at submission time)
@titericz Sorry you’re having trouble. Yes, train_sample_videos is available on the synchronous private rerun on the public test set. All of the files will remain the same as are available on the interactive notebook session, with the exception of the test_videos being swapped out for the hidden public test set.
Have you ensured the code will generate predictions on an unseen set of test_videos, that it meets the code requirements defined & that it will run within the time constraints specified?
Just to clarify, when you say \"test_videos being swapped out\", you mean that the kernel should still read from /kaggle/input/deepfake-detection-challenge/test_videos/ and create the submission file based on the files that are in this /kaggle/input/deepfake-detection-challenge/test_videos/ directory?
@humananalog Yes, precisely!
", "rawMarkdown": "@humananalog Yes, precisely!", "votes": 2 }, { "id": 698047, "postDate": "2019-12-18T18:25:57.210Z", "content": "Does the sample submission csv have a reference to the new test_videos- I'm assuming yes? Also - are we told anywhere approximately how much larger the public test set is than the validation set? Thanks!
I'm assuming that the sample submission csv file might not be correct in the new environment so I'm using something like:
\n\ntest_path = Path(\"/kaggle/input/deepfake-detection-challenge/test_videos/\")\ndefault_value = 0.5 # assume video has 50% prob. to be fake, 1.0 would be 100%\ntest_list = [ ( str(vid_filename).rsplit(\"/\")[-1], default_value ) for vid_filename in test_path.glob('*.mp4')]\nsubmission_df = pd.DataFrame( test_list, columns = ['filename', 'label'])\nsubmission_df.set_index('filename', inplace=True)\nDid you guys manage to submit using a GPU kernel? My first two submissions worked without problem, but after adding face detection and a more complex network I keep getting the \"Submission CSV Not Found\" error. Its quite frustrating, it works on the test sample but when I submit I keep getting the error.
\n\nI supposed it was because some exception being raised but I'm pretty sure I added exception handler on all possible places and am still getting the error. The fact that submissions with errors count on the two submission per day limit is also not really helpful.
", "rawMarkdown": "Did you guys manage to submit using a GPU kernel? My first two submissions worked without problem, but after adding face detection and a more complex network I keep getting the \"Submission CSV Not Found\" error. Its quite frustrating, it works on the test sample but when I submit I keep getting the error. \n\nI supposed it was because some exception being raised but I'm pretty sure I added exception handler on all possible places and am still getting the error. The fact that submissions with errors count on the two submission per day limit is also not really helpful.", "votes": 4, "replies": [ { "id": 699068, "postDate": "2019-12-20T02:44:47.660Z", "content": "I ran into this problem too. But I tried adding df_sub.head() and it worked! It was so bizarre.
", "rawMarkdown": "I ran into this problem too. But I tried adding df_sub.head() and it worked! It was so bizarre.", "votes": 1 }, { "id": 699312, "postDate": "2019-12-20T09:52:33.390Z", "content": "Well, it turns out I had a line missing on my exception handling, now it worked. I still find it strange, without the exception handling I added I was able to predict on more than 5000 videos from the train set without problem, and on the private test set some exception is being raised, really hard to figure it out why without access to the error message :(
", "rawMarkdown": "Well, it turns out I had a line missing on my exception handling, now it worked. I still find it strange, without the exception handling I added I was able to predict on more than 5000 videos from the train set without problem, and on the private test set some exception is being raised, really hard to figure it out why without access to the error message :(" }, { "id": 721043, "postDate": "2020-01-17T00:48:57.833Z", "content": "I will try to if it works again.
", "rawMarkdown": "I will try to if it works again. ", "isDeleted": true } ] }, { "id": 696651, "postDate": "2019-12-16T22:18:07.143Z", "content": "Like I said in the other topic, I did the same, although I trained offline and just loaded the weights of the network on the notebook. The submission worked without problems. Make sure you read the input from the file test_videos and that you are treating possible exceptions if you are doing any processing on the data.
\n\nI didn't do any pre-processing so I didn't have problems on this end, however now that I'm experimenting with face detection I've seem some face detection models that throws exceptions when it's not able to detect a face, which could happen in some cases, so be sure to be treating every possible exception on your data processing pipeline.
", "rawMarkdown": "Like I said in the other topic, I did the same, although I trained offline and just loaded the weights of the network on the notebook. The submission worked without problems. Make sure you read the input from the file test_videos and that you are treating possible exceptions if you are doing any processing on the data. \n\nI didn't do any pre-processing so I didn't have problems on this end, however now that I'm experimenting with face detection I've seem some face detection models that throws exceptions when it's not able to detect a face, which could happen in some cases, so be sure to be treating every possible exception on your data processing pipeline.", "votes": 1 }, { "id": 695994, "postDate": "2019-12-16T01:26:05.357Z", "content": "@juliaelliott I ran into this problem too. It took about 4 minutes to process the public validation set. Will it be able to process the entire public test set in time? I can't find any other possible problem that will cause this issue.
", "rawMarkdown": "@juliaelliott I ran into this problem too. It took about 4 minutes to process the public validation set. Will it be able to process the entire public test set in time? I can't find any other possible problem that will cause this issue.", "votes": 1 }, { "id": 695984, "postDate": "2019-12-16T00:51:26.510Z", "content": "I ran into this problem yesterday. Based on what I changed, two guesses:\n1. That there are files that are larger in the test set than in the small training data subset.\n2. I made the last step of the notebook output the csv file: sub_df.to_csv('submission.csv', index=False)
", "rawMarkdown": "I ran into this problem yesterday. Based on what I changed, two guesses:\n1. That there are files that are larger in the test set than in the small training data subset.\n2. I made the last step of the notebook output the csv file: sub_df.to_csv('submission.csv', index=False)", "votes": 1 }, { "id": 694490, "postDate": "2019-12-13T17:34:55.350Z", "content": "You can check if it exists with a test submission. Try just reading something from train_sample_videos and write a sample submission in the same cell. If they are committed in the same cell then you know the error came because the video wasn't there.
", "rawMarkdown": "You can check if it exists with a test submission. Try just reading something from train_sample_videos and write a sample submission in the same cell. If they are committed in the same cell then you know the error came because the video wasn't there. ", "votes": 1 }, { "id": 696300, "postDate": "2019-12-16T12:32:23.267Z", "content": "Hello, I also encountered such a problem. After submitting it many times, the result is ‘Submission CSV Not Found’. The code I wrote is very simple:\n`\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os
\n\nmyans = np.random.rand(400,1)\nprint(myans.shape)\nss = pd.read_csv(\"/kaggle/input/deepfake-detection-challenge/sample_submission.csv\")\nss['label'] = myans\nprint(ss.shape)\nprint(ss)\nprint(ss.describe())\nss.to_csv('submission.csv', index=False)\n`\nI don't know where I went wrong, this problem has troubled me for a long time.
", "rawMarkdown": "Hello, I also encountered such a problem. After submitting it many times, the result is ‘Submission CSV Not Found’. The code I wrote is very simple:\n`\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\n\nmyans = np.random.rand(400,1)\nprint(myans.shape)\nss = pd.read_csv(\"/kaggle/input/deepfake-detection-challenge/sample_submission.csv\")\nss['label'] = myans\nprint(ss.shape)\nprint(ss)\nprint(ss.describe())\nss.to_csv('submission.csv', index=False)\n`\nI don't know where I went wrong, this problem has troubled me for a long time.", "votes": 2, "replies": [ { "id": 697465, "postDate": "2019-12-18T00:55:22.900Z", "content": "Public testset size is different. Change this line ss['label'] = np.random.rand( ss.shape[0] ,1)
Thank you for your help! I think I know why the error is reported. Once I submit my code, Kaggle seems to test my program with another hidden data set, and the size of this hidden data set may be different. Maybe the best way is to re-read the data from test_videos.
", "rawMarkdown": "Thank you for your help! I think I know why the error is reported. Once I submit my code, Kaggle seems to test my program with another hidden data set, and the size of this hidden data set may be different. Maybe the best way is to re-read the data from test_videos." }, { "id": 698637, "postDate": "2019-12-19T13:54:28.343Z", "content": "Yes, that is the correct way to do it. The hidden test set is supposed to be approximately 4000 videos.
", "rawMarkdown": "Yes, that is the correct way to do it. The hidden test set is supposed to be approximately 4000 videos." }, { "id": 721047, "postDate": "2020-01-17T00:55:49.017Z", "content": "The size of videos will change on public test set and public validation set. It also apply in fit_generator, steps = NUM_SAMPLES // BATCH_SIZE instead of steps=400//BATCH_SIZE.
", "rawMarkdown": "The size of videos will change on public test set and public validation set. It also apply in fit_generator, steps = NUM_SAMPLES // BATCH_SIZE instead of steps=400//BATCH_SIZE. ", "isDeleted": true } ] }, { "id": 731287, "postDate": "2020-01-28T13:28:02.333Z", "content": "If I have a lot of function and class definitions of my own that I need in the submission kernel, is the best way to put them all in a .py file and save it at ../working/? Is this allowed for the current competition?
Upload as a Dataset in a zip file and attach the dataset
", "rawMarkdown": "Upload as a Dataset in a zip file and attach the dataset", "votes": 1 } ] }, { "id": 697717, "postDate": "2019-12-18T10:02:15.670Z", "content": "![]
I can't submit many time, with error is : No additional details provided for this error.\nThis is my kernel https://www.kaggle.com/minhtam/fake-detect-basic\nThanks.
", "rawMarkdown": "\n![]\n\nI can't submit many time, with error is : No additional details provided for this error.\nThis is my kernel https://www.kaggle.com/minhtam/fake-detect-basic\nThanks.", "replies": [ { "id": 698023, "postDate": "2019-12-18T17:30:14.927Z", "content": "I tried your kernel and get the same error. There doesn't appear to be anything obviously wrong with your kernel but it does take 2 hours to run. Perhaps it uses too much memory or CPU and gets killed? It's hard to say without more information.
", "rawMarkdown": "I tried your kernel and get the same error. There doesn't appear to be anything obviously wrong with your kernel but it does take 2 hours to run. Perhaps it uses too much memory or CPU and gets killed? It's hard to say without more information." }, { "id": 698118, "postDate": "2019-12-18T20:29:52.140Z", "content": "Just for giggles, I changed the notebook to do for i in range(0, 1) so that it looks only at a single frame from each video. Now it finished successfully after 3 hours. The score is \"only\" 0.91994 so not at the top of the leaderboard yet. ;-)
So why didn't the full notebook work? My guess is that it consumes too many resources and gets killed.
", "rawMarkdown": "Just for giggles, I changed the notebook to do `for i in range(0, 1)` so that it looks only at a single frame from each video. Now it finished successfully after 3 hours. The score is \"only\" 0.91994 so not at the top of the leaderboard yet. ;-)\n\nSo why didn't the full notebook work? My guess is that it consumes too many resources and gets killed." }, { "id": 698986, "postDate": "2019-12-20T00:46:03.280Z", "content": "If it takes two hours to run on the public commit then it will surely fail on the 10x test set because that would likely take 20 hours and timeout.
", "rawMarkdown": "If it takes two hours to run on the public commit then it will surely fail on the 10x test set because that would likely take 20 hours and timeout. " } ] }, { "id": 721041, "postDate": "2020-01-17T00:48:35.927Z", "content": "I will try to see if it works again.
", "rawMarkdown": "I will try to see if it works again. ", "isDeleted": true }, { "id": 699010, "postDate": "2019-12-20T01:12:38.087Z", "rawMarkdown": "", "isDeleted": true }, { "id": 697925, "postDate": "2019-12-18T14:57:42.243Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 694513, "author_name": "Julia Elliott", "author_url": "", "post_date": "2019-12-13T18:22:42.917000", "content": "@titericz Sorry you’re having trouble. Yes, train_sample_videos is available on the synchronous private rerun on the public test set. All of the files will remain the same as are available on the interactive notebook session, with the exception of the test_videos being swapped out for the hidden public test set.
Have you ensured the code will generate predictions on an unseen set of test_videos, that it meets the code requirements defined & that it will run within the time constraints specified?
Just to clarify, when you say \"test_videos being swapped out\", you mean that the kernel should still read from /kaggle/input/deepfake-detection-challenge/test_videos/ and create the submission file based on the files that are in this /kaggle/input/deepfake-detection-challenge/test_videos/ directory?
@humananalog Yes, precisely!
", "votes": 2, "replies": [] }, { "id": 698047, "author_name": "Rob Mulla", "author_url": "", "post_date": "2019-12-18T18:25:57.210000", "content": "Does the sample submission csv have a reference to the new test_videos- I'm assuming yes? Also - are we told anywhere approximately how much larger the public test set is than the validation set? Thanks!
I'm assuming that the sample submission csv file might not be correct in the new environment so I'm using something like:
\n\ntest_path = Path(\"/kaggle/input/deepfake-detection-challenge/test_videos/\")\ndefault_value = 0.5 # assume video has 50% prob. to be fake, 1.0 would be 100%\ntest_list = [ ( str(vid_filename).rsplit(\"/\")[-1], default_value ) for vid_filename in test_path.glob('*.mp4')]\nsubmission_df = pd.DataFrame( test_list, columns = ['filename', 'label'])\nsubmission_df.set_index('filename', inplace=True)\nDid you guys manage to submit using a GPU kernel? My first two submissions worked without problem, but after adding face detection and a more complex network I keep getting the \"Submission CSV Not Found\" error. Its quite frustrating, it works on the test sample but when I submit I keep getting the error.
\n\nI supposed it was because some exception being raised but I'm pretty sure I added exception handler on all possible places and am still getting the error. The fact that submissions with errors count on the two submission per day limit is also not really helpful.
", "votes": 4, "replies": [ { "id": 699068, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2019-12-20T02:44:47.660000", "content": "I ran into this problem too. But I tried adding df_sub.head() and it worked! It was so bizarre.
", "votes": 1, "replies": [] }, { "id": 699312, "author_name": "Pedro Bernardo", "author_url": "", "post_date": "2019-12-20T09:52:33.390000", "content": "Well, it turns out I had a line missing on my exception handling, now it worked. I still find it strange, without the exception handling I added I was able to predict on more than 5000 videos from the train set without problem, and on the private test set some exception is being raised, really hard to figure it out why without access to the error message :(
", "votes": 0, "replies": [] }, { "id": 721043, "author_name": "", "author_url": "", "post_date": "2020-01-17T00:48:57.833000", "content": "I will try to if it works again.
", "votes": 0, "replies": [] } ] }, { "id": 696651, "author_name": "Pedro Bernardo", "author_url": "", "post_date": "2019-12-16T22:18:07.143000", "content": "Like I said in the other topic, I did the same, although I trained offline and just loaded the weights of the network on the notebook. The submission worked without problems. Make sure you read the input from the file test_videos and that you are treating possible exceptions if you are doing any processing on the data.
\n\nI didn't do any pre-processing so I didn't have problems on this end, however now that I'm experimenting with face detection I've seem some face detection models that throws exceptions when it's not able to detect a face, which could happen in some cases, so be sure to be treating every possible exception on your data processing pipeline.
", "votes": 1, "replies": [] }, { "id": 695994, "author_name": "Shangqiu Li", "author_url": "", "post_date": "2019-12-16T01:26:05.357000", "content": "@juliaelliott I ran into this problem too. It took about 4 minutes to process the public validation set. Will it be able to process the entire public test set in time? I can't find any other possible problem that will cause this issue.
", "votes": 1, "replies": [] }, { "id": 695984, "author_name": "Brian", "author_url": "", "post_date": "2019-12-16T00:51:26.510000", "content": "I ran into this problem yesterday. Based on what I changed, two guesses:\n1. That there are files that are larger in the test set than in the small training data subset.\n2. I made the last step of the notebook output the csv file: sub_df.to_csv('submission.csv', index=False)
", "votes": 1, "replies": [] }, { "id": 694490, "author_name": "ryches", "author_url": "", "post_date": "2019-12-13T17:34:55.350000", "content": "You can check if it exists with a test submission. Try just reading something from train_sample_videos and write a sample submission in the same cell. If they are committed in the same cell then you know the error came because the video wasn't there.
", "votes": 1, "replies": [] }, { "id": 696300, "author_name": "pick", "author_url": "", "post_date": "2019-12-16T12:32:23.267000", "content": "Hello, I also encountered such a problem. After submitting it many times, the result is ‘Submission CSV Not Found’. The code I wrote is very simple:\n`\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os
\n\nmyans = np.random.rand(400,1)\nprint(myans.shape)\nss = pd.read_csv(\"/kaggle/input/deepfake-detection-challenge/sample_submission.csv\")\nss['label'] = myans\nprint(ss.shape)\nprint(ss)\nprint(ss.describe())\nss.to_csv('submission.csv', index=False)\n`\nI don't know where I went wrong, this problem has troubled me for a long time.
", "votes": 2, "replies": [ { "id": 697465, "author_name": "Giba", "author_url": "", "post_date": "2019-12-18T00:55:22.900000", "content": "Public testset size is different. Change this line ss['label'] = np.random.rand( ss.shape[0] ,1)
Thank you for your help! I think I know why the error is reported. Once I submit my code, Kaggle seems to test my program with another hidden data set, and the size of this hidden data set may be different. Maybe the best way is to re-read the data from test_videos.
", "votes": 0, "replies": [] }, { "id": 698637, "author_name": "Human Analog", "author_url": "", "post_date": "2019-12-19T13:54:28.343000", "content": "Yes, that is the correct way to do it. The hidden test set is supposed to be approximately 4000 videos.
", "votes": 0, "replies": [] }, { "id": 721047, "author_name": "", "author_url": "", "post_date": "2020-01-17T00:55:49.017000", "content": "The size of videos will change on public test set and public validation set. It also apply in fit_generator, steps = NUM_SAMPLES // BATCH_SIZE instead of steps=400//BATCH_SIZE.
", "votes": 0, "replies": [] } ] }, { "id": 731287, "author_name": "wafflebufflo", "author_url": "", "post_date": "2020-01-28T13:28:02.333000", "content": "If I have a lot of function and class definitions of my own that I need in the submission kernel, is the best way to put them all in a .py file and save it at ../working/? Is this allowed for the current competition?
Upload as a Dataset in a zip file and attach the dataset
", "votes": 1, "replies": [] } ] }, { "id": 697717, "author_name": "Minh Tâm", "author_url": "", "post_date": "2019-12-18T10:02:15.670000", "content": "![]
I can't submit many time, with error is : No additional details provided for this error.\nThis is my kernel https://www.kaggle.com/minhtam/fake-detect-basic\nThanks.
", "votes": 0, "replies": [ { "id": 698023, "author_name": "Human Analog", "author_url": "", "post_date": "2019-12-18T17:30:14.927000", "content": "I tried your kernel and get the same error. There doesn't appear to be anything obviously wrong with your kernel but it does take 2 hours to run. Perhaps it uses too much memory or CPU and gets killed? It's hard to say without more information.
", "votes": 0, "replies": [] }, { "id": 698118, "author_name": "Human Analog", "author_url": "", "post_date": "2019-12-18T20:29:52.140000", "content": "Just for giggles, I changed the notebook to do for i in range(0, 1) so that it looks only at a single frame from each video. Now it finished successfully after 3 hours. The score is \"only\" 0.91994 so not at the top of the leaderboard yet. ;-)
So why didn't the full notebook work? My guess is that it consumes too many resources and gets killed.
", "votes": 0, "replies": [] }, { "id": 698986, "author_name": "ryches", "author_url": "", "post_date": "2019-12-20T00:46:03.280000", "content": "If it takes two hours to run on the public commit then it will surely fail on the 10x test set because that would likely take 20 hours and timeout.
", "votes": 0, "replies": [] } ] }, { "id": 721041, "author_name": "", "author_url": "", "post_date": "2020-01-17T00:48:35.927000", "content": "I will try to see if it works again.
", "votes": 0, "replies": [] }, { "id": 699010, "author_name": "", "author_url": "", "post_date": "2019-12-20T01:12:38.087000", "content": "", "votes": 0, "replies": [] }, { "id": 697925, "author_name": "", "author_url": "", "post_date": "2019-12-18T14:57:42.243000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "694445": "I have 2 notebooks that Commits without errors but when I Submit I got an error: \"Submission CSV Not Found\". (I suppose there is an error in the notebook before I write the submission.csv file)\n Anyone having the same issue?\n\n@admin: I trained a model using `train_sample_videos`, is it available at submission time? (it could be the reason to fail only at submission time)", "694513": "@titericz Sorry you’re having trouble. Yes, `train_sample_videos` is available on the synchronous private rerun on the public test set. All of the files will remain the same as are available on the interactive notebook session, with the exception of the `test_videos` being swapped out for the hidden public test set.\n\nHave you ensured the code will generate predictions on an unseen set of `test_videos`, that it meets the code requirements defined & that it will run within the time constraints specified?", "698981": "Did you guys manage to submit using a GPU kernel? My first two submissions worked without problem, but after adding face detection and a more complex network I keep getting the \"Submission CSV Not Found\" error. Its quite frustrating, it works on the test sample but when I submit I keep getting the error. \n\nI supposed it was because some exception being raised but I'm pretty sure I added exception handler on all possible places and am still getting the error. The fact that submissions with errors count on the two submission per day limit is also not really helpful.", "696651": "Like I said in the other topic, I did the same, although I trained offline and just loaded the weights of the network on the notebook. The submission worked without problems. Make sure you read the input from the file test_videos and that you are treating possible exceptions if you are doing any processing on the data. \n\nI didn't do any pre-processing so I didn't have problems on this end, however now that I'm experimenting with face detection I've seem some face detection models that throws exceptions when it's not able to detect a face, which could happen in some cases, so be sure to be treating every possible exception on your data processing pipeline.", "695994": "@juliaelliott I ran into this problem too. It took about 4 minutes to process the public validation set. Will it be able to process the entire public test set in time? I can't find any other possible problem that will cause this issue.", "695984": "I ran into this problem yesterday. Based on what I changed, two guesses:\n1. That there are files that are larger in the test set than in the small training data subset.\n2. I made the last step of the notebook output the csv file: sub_df.to_csv('submission.csv', index=False)", "694490": "You can check if it exists with a test submission. Try just reading something from train_sample_videos and write a sample submission in the same cell. If they are committed in the same cell then you know the error came because the video wasn't there. ", "696300": "Hello, I also encountered such a problem. After submitting it many times, the result is ‘Submission CSV Not Found’. The code I wrote is very simple:\n`\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\n\nmyans = np.random.rand(400,1)\nprint(myans.shape)\nss = pd.read_csv(\"/kaggle/input/deepfake-detection-challenge/sample_submission.csv\")\nss['label'] = myans\nprint(ss.shape)\nprint(ss)\nprint(ss.describe())\nss.to_csv('submission.csv', index=False)\n`\nI don't know where I went wrong, this problem has troubled me for a long time.", "731287": "If I have a lot of function and class definitions of my own that I need in the submission kernel, is the best way to put them all in a .py file and save it at `../working/`? Is this allowed for the current competition?", "697717": "\n![]\n\nI can't submit many time, with error is : No additional details provided for this error.\nThis is my kernel https://www.kaggle.com/minhtam/fake-detect-basic\nThanks.", "721041": "I will try to see if it works again. ", "699010": "", "697925": "" } }