Dear all,
\n\nDespite of the number of notebooks showing how to create a submission code, it is still not clear for me how the overall evaluation process works.
\n\nIf I understand there are three steps in this process:
\n\nThe first step seems straightforward since we know the videos in the Public Validation Set. So we can just create a notebook that generate a random submission file with the correct format and with the correct filenames.
\n\nBut, I don't understand how Kaggle can runs the notebook automatically for the step 2 and 3 and gives to it the correct filenames if it is not specified elsewhere.
\n\nShould we specify the input of the notebook or is the a convention on where to get the data on the filesystem ?
", "messages": [ { "id": "737561", "postDate": "02/05/2020 14:31:43", "content": "Dear all,
\n\nDespite of the number of notebooks showing how to create a submission code, it is still not clear for me how the overall evaluation process works.
\n\nIf I understand there are three steps in this process:
\n\nThe first step seems straightforward since we know the videos in the Public Validation Set. So we can just create a notebook that generate a random submission file with the correct format and with the correct filenames.
\n\nBut, I don't understand how Kaggle can runs the notebook automatically for the step 2 and 3 and gives to it the correct filenames if it is not specified elsewhere.
\n\nShould we specify the input of the notebook or is the a convention on where to get the data on the filesystem ?
", "rawMarkdown": "Dear all,\n\nDespite of the number of notebooks showing how to create a submission code, it is still not clear for me how the overall evaluation process works.\n\nIf I understand there are three steps in this process:\n\n1. Run the notebook to compute the score on the Public Validation Set \n2. Rerun the notebook to compute the scores on the Public Test Set\n3. Rerun the notebook to compute the final scores on the Private Test Set \n\nThe first step seems straightforward since we know the videos in the Public Validation Set. So we can just create a notebook that generate a random submission file with the correct format and with the correct filenames.\n\nBut, I don't understand how Kaggle can runs the notebook automatically for the step 2 and 3 and gives to it the correct filenames if it is not specified elsewhere.\n\nShould we specify the input of the notebook or is the a convention on where to get the data on the filesystem ?", "votes": null }, { "id": "737564", "postDate": "02/05/2020 14:36:24", "content": "@rajao236146 Based on my understanding, Kaggle will change the contents of the test_videos folder when your notebook reruns in steps 2 and 3; so, basically, your kernel must accept and be able to handle any video occurs in test_videos folder.
That sounds reasonable. So I think it is sufficient to list /kaggle/input/deepfake-detection-challenge/test_videos/ ?
Yes, I use this method as well and it works fine.
", "rawMarkdown": "Yes, I use this method as well and it works fine.", "votes": null }, { "id": "737604", "postDate": "02/05/2020 15:06:59", "content": "@phunghieu Thanks 👍
", "rawMarkdown": "phunghieu Thanks 👍", "votes": null }, { "id": "742226", "postDate": "02/11/2020 07:00:10", "content": "Hey! Do you mean listing the files and changing the predictions thereafter?
", "rawMarkdown": "Hey! Do you mean listing the files and changing the predictions thereafter?", "votes": null } ], "comments": [ { "id": 737564, "author_name": "phunghieu", "author_url": "", "post_date": "02/05/2020 14:36:24", "content": "@rajao236146 Based on my understanding, Kaggle will change the contents of the test_videos folder when your notebook reruns in steps 2 and 3; so, basically, your kernel must accept and be able to handle any video occurs in test_videos folder.
That sounds reasonable. So I think it is sufficient to list /kaggle/input/deepfake-detection-challenge/test_videos/ ?
Yes, I use this method as well and it works fine.
", "votes": null, "replies": [] }, { "id": 737604, "author_name": "rajao236146", "author_url": "", "post_date": "02/05/2020 15:06:59", "content": "@phunghieu Thanks 👍
", "votes": null, "replies": [] }, { "id": 742226, "author_name": "akashchinta", "author_url": "", "post_date": "02/11/2020 07:00:10", "content": "Hey! Do you mean listing the files and changing the predictions thereafter?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "737561": "Dear all,\n\nDespite of the number of notebooks showing how to create a submission code, it is still not clear for me how the overall evaluation process works.\n\nIf I understand there are three steps in this process:\n\n1. Run the notebook to compute the score on the Public Validation Set \n2. Rerun the notebook to compute the scores on the Public Test Set\n3. Rerun the notebook to compute the final scores on the Private Test Set \n\nThe first step seems straightforward since we know the videos in the Public Validation Set. So we can just create a notebook that generate a random submission file with the correct format and with the correct filenames.\n\nBut, I don't understand how Kaggle can runs the notebook automatically for the step 2 and 3 and gives to it the correct filenames if it is not specified elsewhere.\n\nShould we specify the input of the notebook or is the a convention on where to get the data on the filesystem ?", "737564": "rajao236146 Based on my understanding, Kaggle will change the contents of the `test_videos` folder when your notebook reruns in steps 2 and 3; so, basically, your kernel must accept and be able to handle any video occurs in `test_videos` folder.", "737573": "That sounds reasonable. So I think it is sufficient to list `/kaggle/input/deepfake-detection-challenge/test_videos/` ?", "737575": "Yes, I use this method as well and it works fine.", "737604": "phunghieu Thanks 👍", "742226": "Hey! Do you mean listing the files and changing the predictions thereafter?" }, "source": "meta" }