{
  "id": 121309,
  "title": "How does it calculate the score on test set?",
  "url": "/competitions/deepfake-detection-challenge/discussion/121309",
  "author_name": "",
  "post_date": "2019-12-12T10:35:24.528878300Z",
  "votes": 3,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I just submit the results of 400 videos on Public Validation Set. How does it calculate the score of the whole Public Test Set?</p>",
  "messages": [
    {
      "id": "693404",
      "postDate": "12/12/2019 10:35:24",
      "content": "<p>I just submit the results of 400 videos on Public Validation Set. How does it calculate the score of the whole Public Test Set?</p>",
      "rawMarkdown": "I just submit the results of 400 videos on Public Validation Set. How does it calculate the score of the whole Public Test Set?",
      "votes": null
    },
    {
      "id": "693830",
      "postDate": "12/12/2019 20:34:56",
      "content": "<p>Read the evaluation guidelines: <a href=\"https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation\">https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation</a></p>",
      "rawMarkdown": "Read the evaluation guidelines: [https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation](https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation)",
      "votes": null
    },
    {
      "id": "693882",
      "postDate": "12/12/2019 22:08:49",
      "content": "<p>First, it is important you read the <a href=\"https://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started\">Getting Started</a> page carefully. This describes in a very detailed way the different datasets involved and how those evaluations take place. </p>\n\n<p>You <strong>do not</strong> receive a score on the public validation set. You <strong>do</strong> receive a score on the public test set. This public test set is hidden and scored when you \"Submit to Competition\" to the public leaderboard.</p>",
      "rawMarkdown": "First, it is important you read the [Getting Started](https://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started) page carefully. This describes in a very detailed way the different datasets involved and how those evaluations take place. \n\nYou **do not** receive a score on the public validation set. You **do** receive a score on the public test set. This public test set is hidden and scored when you \"Submit to Competition\" to the public leaderboard.",
      "votes": null
    },
    {
      "id": "694836",
      "postDate": "12/14/2019 06:46:57",
      "content": "<p>But I didn't submit anything (code or model) except a CSV file of the results on public validation set. How was the public test set scored?😳 </p>",
      "rawMarkdown": "But I didn't submit anything (code or model) except a CSV file of the results on public validation set. How was the public test set scored?😳",
      "votes": null
    },
    {
      "id": "694851",
      "postDate": "12/14/2019 07:24:38",
      "content": "<p><a href=\"/haozshno1\">@haozshno1</a> - You trigger the submission by clicking on “Submit to Competition” on the output of the notebook that writes predictions to the output. However, that submission then triggers a re-run of that same notebook with the <code>test_videos</code> file (and corresponding <code>sample_submission</code> file) swapped out for the hidden public test set. The resulting prediction file is scored and writes that score to the public leaderboard. So for your code to be capable of generating a public score, it must be transferable to that hidden test set. Your code must be writing to a <code>submission.csv</code> with valid format generalized to the id’s associated with the test videos.</p>",
      "rawMarkdown": "haozshno1 - You trigger the submission by clicking on “Submit to Competition” on the output of the notebook that writes predictions to the output. However, that submission then triggers a re-run of that same notebook with the `test_videos` file (and corresponding `sample_submission` file) swapped out for the hidden public test set. The resulting prediction file is scored and writes that score to the public leaderboard. So for your code to be capable of generating a public score, it must be transferable to that hidden test set. Your code must be writing to a `submission.csv` with valid format generalized to the id’s associated with the test videos.",
      "votes": null
    },
    {
      "id": "695035",
      "postDate": "12/14/2019 13:21:30",
      "content": "<p>Thank you very much! Now I understand.</p>",
      "rawMarkdown": "Thank you very much! Now I understand.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 693830,
      "author_name": "techytushar",
      "author_url": "",
      "post_date": "12/12/2019 20:34:56",
      "content": "<p>Read the evaluation guidelines: <a href=\"https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation\">https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 693882,
      "author_name": "juliaelliott",
      "author_url": "",
      "post_date": "12/12/2019 22:08:49",
      "content": "<p>First, it is important you read the <a href=\"https://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started\">Getting Started</a> page carefully. This describes in a very detailed way the different datasets involved and how those evaluations take place. </p>\n\n<p>You <strong>do not</strong> receive a score on the public validation set. You <strong>do</strong> receive a score on the public test set. This public test set is hidden and scored when you \"Submit to Competition\" to the public leaderboard.</p>",
      "votes": null,
      "replies": [
        {
          "id": 694836,
          "author_name": "haozshno1",
          "author_url": "",
          "post_date": "12/14/2019 06:46:57",
          "content": "<p>But I didn't submit anything (code or model) except a CSV file of the results on public validation set. How was the public test set scored?😳 </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 694851,
          "author_name": "juliaelliott",
          "author_url": "",
          "post_date": "12/14/2019 07:24:38",
          "content": "<p><a href=\"/haozshno1\">@haozshno1</a> - You trigger the submission by clicking on “Submit to Competition” on the output of the notebook that writes predictions to the output. However, that submission then triggers a re-run of that same notebook with the <code>test_videos</code> file (and corresponding <code>sample_submission</code> file) swapped out for the hidden public test set. The resulting prediction file is scored and writes that score to the public leaderboard. So for your code to be capable of generating a public score, it must be transferable to that hidden test set. Your code must be writing to a <code>submission.csv</code> with valid format generalized to the id’s associated with the test videos.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 695035,
          "author_name": "haozshno1",
          "author_url": "",
          "post_date": "12/14/2019 13:21:30",
          "content": "<p>Thank you very much! Now I understand.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "693404": "I just submit the results of 400 videos on Public Validation Set. How does it calculate the score of the whole Public Test Set?",
    "693830": "Read the evaluation guidelines: [https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation](https://www.kaggle.com/c/deepfake-detection-challenge/overview/evaluation)",
    "693882": "First, it is important you read the [Getting Started](https://www.kaggle.com/c/deepfake-detection-challenge/overview/getting-started) page carefully. This describes in a very detailed way the different datasets involved and how those evaluations take place. \n\nYou **do not** receive a score on the public validation set. You **do** receive a score on the public test set. This public test set is hidden and scored when you \"Submit to Competition\" to the public leaderboard.",
    "694836": "But I didn't submit anything (code or model) except a CSV file of the results on public validation set. How was the public test set scored?😳",
    "694851": "haozshno1 - You trigger the submission by clicking on “Submit to Competition” on the output of the notebook that writes predictions to the output. However, that submission then triggers a re-run of that same notebook with the `test_videos` file (and corresponding `sample_submission` file) swapped out for the hidden public test set. The resulting prediction file is scored and writes that score to the public leaderboard. So for your code to be capable of generating a public score, it must be transferable to that hidden test set. Your code must be writing to a `submission.csv` with valid format generalized to the id’s associated with the test videos.",
    "695035": "Thank you very much! Now I understand."
  },
  "source": "meta"
}