{
  "id": 199056,
  "title": "where are test dataset?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/199056",
  "author_name": "",
  "post_date": "2020-11-24T07:08:01.074811100Z",
  "votes": 8,
  "comment_count": 3,
  "views": 0,
  "content": "<p>competition newbie here</p>\n<p>I have trained a model within a Kaggle notebook, but I can find only one test data.</p>\n<p>Here are my questions</p>\n<ol>\n<li><p>I have read some threads the submission should be made with a \"offline\" mode notebook. Then, how can I train a model in a notebook? </p>\n<ul>\n<li>can I train a model and save it, and can I use it for a \"inference\" only notebook?</li>\n<li>if so, how can I save the model and share in another notebook?</li></ul></li>\n<li><p>I have tested a notebook with \"offline\" mode by just printing out test dataset files.</p>\n<ul>\n<li>but I could find only one test data still. what would be the problem?</li></ul></li>\n</ol>\n<p>If anyone knows how to solve the issues, please let me know.<br>\nThanks for helping me out. I really do appreciate to all Kagglers!</p>",
  "messages": [
    {
      "id": "1089059",
      "postDate": "11/24/2020 07:08:01",
      "content": "<p>competition newbie here</p>\n<p>I have trained a model within a Kaggle notebook, but I can find only one test data.</p>\n<p>Here are my questions</p>\n<ol>\n<li><p>I have read some threads the submission should be made with a \"offline\" mode notebook. Then, how can I train a model in a notebook? </p>\n<ul>\n<li>can I train a model and save it, and can I use it for a \"inference\" only notebook?</li>\n<li>if so, how can I save the model and share in another notebook?</li></ul></li>\n<li><p>I have tested a notebook with \"offline\" mode by just printing out test dataset files.</p>\n<ul>\n<li>but I could find only one test data still. what would be the problem?</li></ul></li>\n</ol>\n<p>If anyone knows how to solve the issues, please let me know.<br>\nThanks for helping me out. I really do appreciate to all Kagglers!</p>",
      "rawMarkdown": "competition newbie here\n\nI have trained a model within a Kaggle notebook, but I can find only one test data.\n\nHere are my questions\n\n1. I have read some threads the submission should be made with a \"offline\" mode notebook. Then, how can I train a model in a notebook? \n  - can I train a model and save it, and can I use it for a \"inference\" only notebook?\n  - if so, how can I save the model and share in another notebook?\n\n2. I have tested a notebook with \"offline\" mode by just printing out test dataset files.\n  - but I could find only one test data still. what would be the problem?\n\nIf anyone knows how to solve the issues, please let me know.\nThanks for helping me out. I really do appreciate to all Kagglers!",
      "votes": null
    },
    {
      "id": "1089067",
      "postDate": "11/24/2020 07:22:04",
      "content": "<p>Since you are new I will tell you about how competitions works:</p>\n<ul>\n<li>The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and ONLY THE PRIVATE SCORE will be used for final standing. </li>\n<li>Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).</li>\n<li>Offline mode notebook has two meaning: 1. You must turn off Internet in your submission notebook, this is because this competition is a Code competition. And 2. They train the model offline (on their local machines), then they upload the model to a dataset, load it and make predictions. Internet is not allowed in making a submission to prevent competitors mine the ground truths, and also TPU reads data from GCS so Internet must be turned on to be used.</li>\n</ul>\n<p>So what you should do:</p>\n<ul>\n<li>Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.</li>\n<li>Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. </li>\n</ul>\n<p>Also make sure that the notebook runtime does not exceed the requirements.</p>\n<p>Good luck in climbing the leaderboard!</p>",
      "rawMarkdown": "Since you are new I will tell you about how competitions works:\n\n- The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and ONLY THE PRIVATE SCORE will be used for final standing. \n- Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).\n- Offline mode notebook has two meaning: 1. You must turn off Internet in your submission notebook, this is because this competition is a Code competition. And 2. They train the model offline (on their local machines), then they upload the model to a dataset, load it and make predictions. Internet is not allowed in making a submission to prevent competitors mine the ground truths, and also TPU reads data from GCS so Internet must be turned on to be used.\n\nSo what you should do:\n- Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.\n- Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. \n\nAlso make sure that the notebook runtime does not exceed the requirements.\n\nGood luck in climbing the leaderboard!",
      "votes": null
    },
    {
      "id": "1089078",
      "postDate": "11/24/2020 07:32:20",
      "content": "<ol>\n<li><p>Yes, you can. If you save your model's state_dict in the training notebook, it will be available as the output of this notebook. You can link the output of a notebook as a data source in the inference notebook or create a dataset with saved models and add it as a data source.</p></li>\n<li><p>This one image is left for debugging reasons I think, to know the directory structure etc. When you make a submission, the notebook is re-run with the hidden test set available, so you only have to make sure that the submission is created correctly (on this one image). You can use glob() to get all images from the test set directory (you still won't see them, but it will get you all test images during re-run).   </p></li>\n</ol>",
      "rawMarkdown": "1. Yes, you can. If you save your model's state_dict in the training notebook, it will be available as the output of this notebook. You can link the output of a notebook as a data source in the inference notebook or create a dataset with saved models and add it as a data source.\n\n2. This one image is left for debugging reasons I think, to know the directory structure etc. When you make a submission, the notebook is re-run with the hidden test set available, so you only have to make sure that the submission is created correctly (on this one image). You can use glob() to get all images from the test set directory (you still won't see them, but it will get you all test images during re-run).",
      "votes": null
    },
    {
      "id": "1089417",
      "postDate": "11/24/2020 13:32:35",
      "content": "<p><a href=\"https://www.kaggle.com/aeryss\" target=\"_blank\">@aeryss</a> <a href=\"https://www.kaggle.com/szacho\" target=\"_blank\">@szacho</a> Thank you so much<br>\nThe comments are very informative! :) </p>\n<p>I will keep trying to make my first submit!</p>",
      "rawMarkdown": "aeryss @szacho Thank you so much\nThe comments are very informative! :) \n\nI will keep trying to make my first submit!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1089067,
      "author_name": "aeryss",
      "author_url": "",
      "post_date": "11/24/2020 07:22:04",
      "content": "<p>Since you are new I will tell you about how competitions works:</p>\n<ul>\n<li>The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and ONLY THE PRIVATE SCORE will be used for final standing. </li>\n<li>Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).</li>\n<li>Offline mode notebook has two meaning: 1. You must turn off Internet in your submission notebook, this is because this competition is a Code competition. And 2. They train the model offline (on their local machines), then they upload the model to a dataset, load it and make predictions. Internet is not allowed in making a submission to prevent competitors mine the ground truths, and also TPU reads data from GCS so Internet must be turned on to be used.</li>\n</ul>\n<p>So what you should do:</p>\n<ul>\n<li>Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.</li>\n<li>Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. </li>\n</ul>\n<p>Also make sure that the notebook runtime does not exceed the requirements.</p>\n<p>Good luck in climbing the leaderboard!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1089078,
      "author_name": "szacho",
      "author_url": "",
      "post_date": "11/24/2020 07:32:20",
      "content": "<ol>\n<li><p>Yes, you can. If you save your model's state_dict in the training notebook, it will be available as the output of this notebook. You can link the output of a notebook as a data source in the inference notebook or create a dataset with saved models and add it as a data source.</p></li>\n<li><p>This one image is left for debugging reasons I think, to know the directory structure etc. When you make a submission, the notebook is re-run with the hidden test set available, so you only have to make sure that the submission is created correctly (on this one image). You can use glob() to get all images from the test set directory (you still won't see them, but it will get you all test images during re-run).   </p></li>\n</ol>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1089417,
      "author_name": "tomcspark",
      "author_url": "",
      "post_date": "11/24/2020 13:32:35",
      "content": "<p><a href=\"https://www.kaggle.com/aeryss\" target=\"_blank\">@aeryss</a> <a href=\"https://www.kaggle.com/szacho\" target=\"_blank\">@szacho</a> Thank you so much<br>\nThe comments are very informative! :) </p>\n<p>I will keep trying to make my first submit!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1089059": "competition newbie here\n\nI have trained a model within a Kaggle notebook, but I can find only one test data.\n\nHere are my questions\n\n1. I have read some threads the submission should be made with a \"offline\" mode notebook. Then, how can I train a model in a notebook? \n  - can I train a model and save it, and can I use it for a \"inference\" only notebook?\n  - if so, how can I save the model and share in another notebook?\n\n2. I have tested a notebook with \"offline\" mode by just printing out test dataset files.\n  - but I could find only one test data still. what would be the problem?\n\nIf anyone knows how to solve the issues, please let me know.\nThanks for helping me out. I really do appreciate to all Kagglers!",
    "1089067": "Since you are new I will tell you about how competitions works:\n\n- The test set is divided into two parts: Public and Private part. When you submit, you only see the Public score. The private score will be kept secret until the competition ends, and ONLY THE PRIVATE SCORE will be used for final standing. \n- Where is the Test set? In this one, all the public and private sets are hidden from us. What you see in \"test\" is only a sample. When you submit, your notebook will be rerun: the test set will be replaced by a bigger one, then it calculates the score, and only shows you the public part (in this one, 40% of the test data).\n- Offline mode notebook has two meaning: 1. You must turn off Internet in your submission notebook, this is because this competition is a Code competition. And 2. They train the model offline (on their local machines), then they upload the model to a dataset, load it and make predictions. Internet is not allowed in making a submission to prevent competitors mine the ground truths, and also TPU reads data from GCS so Internet must be turned on to be used.\n\nSo what you should do:\n- Case 1: 1 notebook for training and 1 notebook for submitting. Just train the first one as usual, save the output, download (or make new dataset), load it in your submission notebook, make predictions and you are good to go. TURN OFF INTERNET IN THE SUBMISSION NOTEBOOK.\n- Case 2: training and submit in a notebook: it is just case 1 in only one notebook, but its internet must be turn off. In this case, to load a pretrained model, you can use public datasets and load it. Check out my datasets, I made datasets of all models from TF/Keras 2.2 and 2.3 already. \n\nAlso make sure that the notebook runtime does not exceed the requirements.\n\nGood luck in climbing the leaderboard!",
    "1089078": "1. Yes, you can. If you save your model's state_dict in the training notebook, it will be available as the output of this notebook. You can link the output of a notebook as a data source in the inference notebook or create a dataset with saved models and add it as a data source.\n\n2. This one image is left for debugging reasons I think, to know the directory structure etc. When you make a submission, the notebook is re-run with the hidden test set available, so you only have to make sure that the submission is created correctly (on this one image). You can use glob() to get all images from the test set directory (you still won't see them, but it will get you all test images during re-run).",
    "1089417": "aeryss @szacho Thank you so much\nThe comments are very informative! :) \n\nI will keep trying to make my first submit!"
  },
  "source": "meta"
}