{
  "id": 205210,
  "title": "An answer to \"Where is the test set? Test set 1 image?\" etc.",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/205210",
  "author_name": "",
  "post_date": "2020-12-19T02:44:40.703599400Z",
  "votes": 4,
  "comment_count": 1,
  "views": 0,
  "content": "<p>People who are wondering why the test set only has ONE image, here is how competitions work:</p>\n<ul>\n<li>The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and <strong>ONLY THE PRIVATE SCORE</strong> will be used for final standing. </li>\n<li>Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).</li>\n<li>This is a CODE COMPETITION, so: you must turn off Internet in your submission notebook. Internet is not allowed in making a submission to prevent competitors mining the ground truths. TPU reads data from GCS so Internet must be turned on to be used, which you cannot when submitting.</li>\n</ul>\n<p>So what you should do:</p>\n<ul>\n<li>Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.</li>\n<li>Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. </li>\n</ul>\n<p>Also make sure that the notebook runtime does not exceed the requirements.</p>\n<p>Good luck in climbing the leaderboard!</p>",
  "messages": [
    {
      "id": "1118375",
      "postDate": "12/19/2020 02:44:40",
      "content": "<p>People who are wondering why the test set only has ONE image, here is how competitions work:</p>\n<ul>\n<li>The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and <strong>ONLY THE PRIVATE SCORE</strong> will be used for final standing. </li>\n<li>Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).</li>\n<li>This is a CODE COMPETITION, so: you must turn off Internet in your submission notebook. Internet is not allowed in making a submission to prevent competitors mining the ground truths. TPU reads data from GCS so Internet must be turned on to be used, which you cannot when submitting.</li>\n</ul>\n<p>So what you should do:</p>\n<ul>\n<li>Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.</li>\n<li>Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. </li>\n</ul>\n<p>Also make sure that the notebook runtime does not exceed the requirements.</p>\n<p>Good luck in climbing the leaderboard!</p>",
      "rawMarkdown": "People who are wondering why the test set only has ONE image, here is how competitions work:\n\n- The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and **ONLY THE PRIVATE SCORE** will be used for final standing. \n- Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).\n- This is a CODE COMPETITION, so: you must turn off Internet in your submission notebook. Internet is not allowed in making a submission to prevent competitors mining the ground truths. TPU reads data from GCS so Internet must be turned on to be used, which you cannot when submitting.\n\nSo what you should do:\n- Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.\n- Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. \n\nAlso make sure that the notebook runtime does not exceed the requirements.\n\nGood luck in climbing the leaderboard!",
      "votes": null
    },
    {
      "id": "1172833",
      "postDate": "01/27/2021 15:07:33",
      "content": "<p>Thanks for writing this!!   I found this thread simply by searching for \"load test\" in the Discussion section of this competition, thank you for writing this useful information !!</p>",
      "rawMarkdown": "Thanks for writing this!!   I found this thread simply by searching for \"load test\" in the Discussion section of this competition, thank you for writing this useful information !!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1172833,
      "author_name": "jonas0",
      "author_url": "",
      "post_date": "01/27/2021 15:07:33",
      "content": "<p>Thanks for writing this!!   I found this thread simply by searching for \"load test\" in the Discussion section of this competition, thank you for writing this useful information !!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1118375": "People who are wondering why the test set only has ONE image, here is how competitions work:\n\n- The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and **ONLY THE PRIVATE SCORE** will be used for final standing. \n- Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).\n- This is a CODE COMPETITION, so: you must turn off Internet in your submission notebook. Internet is not allowed in making a submission to prevent competitors mining the ground truths. TPU reads data from GCS so Internet must be turned on to be used, which you cannot when submitting.\n\nSo what you should do:\n- Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.\n- Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. \n\nAlso make sure that the notebook runtime does not exceed the requirements.\n\nGood luck in climbing the leaderboard!",
    "1172833": "Thanks for writing this!!   I found this thread simply by searching for \"load test\" in the Discussion section of this competition, thank you for writing this useful information !!"
  },
  "source": "meta"
}