Can I train models on the local machine, and only submit the trained-weights and the inference notebook in this competetion ?
", "messages": [ { "id": 1561054, "postDate": "2021-10-27T09:49:33.657Z", "content": "Can I train models on the local machine, and only submit the trained-weights and the inference notebook in this competetion ?
", "rawMarkdown": "Can I train models on the local machine, and only submit the trained-weights and the inference notebook in this competetion ?", "votes": 8 }, { "id": 1561092, "postDate": "2021-10-27T10:26:58.033Z", "content": "Hi, yes as far as i understand and did so in other competition, you can download the data on your laptop or computer - if this is meant by your local machine, you can train there and then for the submission you can use the weights and interfer from a new kaggle notebook. You can of course also train on google colab or any cloud service.
", "rawMarkdown": "Hi, yes as far as i understand and did so in other competition, you can download the data on your laptop or computer - if this is meant by your local machine, you can train there and then for the submission you can use the weights and interfer from a new kaggle notebook. You can of course also train on google colab or any cloud service.\n ", "votes": 5, "replies": [ { "id": 1561152, "postDate": "2021-10-27T11:45:14.177Z", "content": "tysm for clearing that up :D
", "rawMarkdown": "tysm for clearing that up :D" }, { "id": 1561199, "postDate": "2021-10-27T12:19:11.207Z", "content": "What is also possible: you can use several kaggle notebooks, one for preprocessing where you Write proprocessed data to our output folder by the commit and run order. And then you can open up a new kaggle notebook and use the other one before as input to use its proprocessed data. The same applies for training and submission, that's the way I do it.
", "rawMarkdown": "What is also possible: you can use several kaggle notebooks, one for preprocessing where you Write proprocessed data to our output folder by the commit and run order. And then you can open up a new kaggle notebook and use the other one before as input to use its proprocessed data. The same applies for training and submission, that's the way I do it. ", "votes": 1 }, { "id": 1563188, "postDate": "2021-10-28T05:45:51.257Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1563239, "postDate": "2021-10-28T06:43:49.890Z", "content": "@weka511 the same question was raised in another competition that just ended and is answered bellow. I’m assuming the same reasoning applies here.
\n\n", "rawMarkdown": "@weka511 the same question was raised in another competition that just ended and is answered bellow. I’m assuming the same reasoning applies here.\n> External data that must be publicly available is anything you did not create yourself. That could mean supplemental financial data or a pretrained model from tfhub.dev. Anything that you have created, like model weights trained offline on the competition training set, does not need to be made publicly available.", "votes": 2 }, { "id": 1563365, "postDate": "2021-10-28T08:09:53.170Z", "content": "External data that must be publicly available is anything you did not create yourself. That could mean supplemental financial data or a pretrained model from tfhub.dev. Anything that you have created, like model weights trained offline on the competition training set, does not need to be made publicly available.
\n
Internal data means data that can be derived from the original competition data. For example you take the pngs from the original data and resize it and store it as Jpeg, doing all that on your local machine and then you can upload it to your kaggle notebook. As a side information you can upload several data sets to a kaggle notebook. Another example would be you train your cnn on the original png or jpeg data and upload the model or the weights to your kaggle notebook. It's all derived by the original data.
\nExternal means for example you have a friend at a university having trained 100000 of images on a similar data set and your friend only gives you the trained weights. Then it's obviously not fair and that's forbidden.
\nIn science that would be perfectly fine to use external data you just have to mention it but here it's a competition and everybody should have the same chances.
\nSo if you use external data everybody has to have access to it. For example pretrained models from keras which are accessible for everyone.
", "rawMarkdown": "Internal data means data that can be derived from the original competition data. For example you take the pngs from the original data and resize it and store it as Jpeg, doing all that on your local machine and then you can upload it to your kaggle notebook. As a side information you can upload several data sets to a kaggle notebook. Another example would be you train your cnn on the original png or jpeg data and upload the model or the weights to your kaggle notebook. It's all derived by the original data.\n\nExternal means for example you have a friend at a university having trained 100000 of images on a similar data set and your friend only gives you the trained weights. Then it's obviously not fair and that's forbidden.\n\nIn science that would be perfectly fine to use external data you just have to mention it but here it's a competition and everybody should have the same chances. \n\nSo if you use external data everybody has to have access to it. For example pretrained models from keras which are accessible for everyone. ", "votes": 1 } ] }, { "id": 1590543, "postDate": "2021-11-21T11:49:11.647Z", "content": "The code competition requirements also include that the notebook must not run longer than 9 hours which is the upper limit for Kaggle notebooks. Does anyone know whether this also applies for the training notebooks? As an example, am I allowed to train over 2 notebook runs, each of them taking 8h (just an example) or must the complete training be doable within 9h?
", "rawMarkdown": "The code competition requirements also include that the notebook must not run longer than 9 hours which is the upper limit for Kaggle notebooks. Does anyone know whether this also applies for the training notebooks? As an example, am I allowed to train over 2 notebook runs, each of them taking 8h (just an example) or must the complete training be doable within 9h?" }, { "id": 1561072, "postDate": "2021-10-27T09:59:01.033Z", "content": "i have the same question due to being new here at Kaggle , hopefully someone can provide answer.
", "rawMarkdown": "i have the same question due to being new here at Kaggle , hopefully someone can provide answer.", "replies": [ { "id": 1590711, "postDate": "2021-11-21T15:15:34.530Z", "content": "You can take a long as you want to train. No limitations on training.
", "rawMarkdown": "You can take a long as you want to train. No limitations on training. ", "votes": 1 } ] }, { "id": 1581263, "postDate": "2021-11-13T15:12:19.373Z", "rawMarkdown": "", "votes": -1, "isDeleted": true } ], "comments": [ { "id": 1561092, "author_name": "Andreas Horlbeck", "author_url": "", "post_date": "2021-10-27T10:26:58.033000", "content": "Hi, yes as far as i understand and did so in other competition, you can download the data on your laptop or computer - if this is meant by your local machine, you can train there and then for the submission you can use the weights and interfer from a new kaggle notebook. You can of course also train on google colab or any cloud service.
", "votes": 5, "replies": [ { "id": 1561152, "author_name": "Kesh_Here", "author_url": "", "post_date": "2021-10-27T11:45:14.177000", "content": "tysm for clearing that up :D
", "votes": 0, "replies": [] }, { "id": 1561199, "author_name": "Andreas Horlbeck", "author_url": "", "post_date": "2021-10-27T12:19:11.207000", "content": "What is also possible: you can use several kaggle notebooks, one for preprocessing where you Write proprocessed data to our output folder by the commit and run order. And then you can open up a new kaggle notebook and use the other one before as input to use its proprocessed data. The same applies for training and submission, that's the way I do it.
", "votes": 1, "replies": [] }, { "id": 1563188, "author_name": "", "author_url": "", "post_date": "2021-10-28T05:45:51.257000", "content": "", "votes": 1, "replies": [] }, { "id": 1563239, "author_name": "Slawek Biel", "author_url": "", "post_date": "2021-10-28T06:43:49.890000", "content": "@weka511 the same question was raised in another competition that just ended and is answered bellow. I’m assuming the same reasoning applies here.
\n\n", "votes": 2, "replies": [] }, { "id": 1563365, "author_name": "Andreas Horlbeck", "author_url": "", "post_date": "2021-10-28T08:09:53.170000", "content": "External data that must be publicly available is anything you did not create yourself. That could mean supplemental financial data or a pretrained model from tfhub.dev. Anything that you have created, like model weights trained offline on the competition training set, does not need to be made publicly available.
\n
Internal data means data that can be derived from the original competition data. For example you take the pngs from the original data and resize it and store it as Jpeg, doing all that on your local machine and then you can upload it to your kaggle notebook. As a side information you can upload several data sets to a kaggle notebook. Another example would be you train your cnn on the original png or jpeg data and upload the model or the weights to your kaggle notebook. It's all derived by the original data.
\nExternal means for example you have a friend at a university having trained 100000 of images on a similar data set and your friend only gives you the trained weights. Then it's obviously not fair and that's forbidden.
\nIn science that would be perfectly fine to use external data you just have to mention it but here it's a competition and everybody should have the same chances.
\nSo if you use external data everybody has to have access to it. For example pretrained models from keras which are accessible for everyone.
", "votes": 1, "replies": [] } ] }, { "id": 1590543, "author_name": "omallo", "author_url": "", "post_date": "2021-11-21T11:49:11.647000", "content": "The code competition requirements also include that the notebook must not run longer than 9 hours which is the upper limit for Kaggle notebooks. Does anyone know whether this also applies for the training notebooks? As an example, am I allowed to train over 2 notebook runs, each of them taking 8h (just an example) or must the complete training be doable within 9h?
", "votes": 0, "replies": [] }, { "id": 1561072, "author_name": "Kesh_Here", "author_url": "", "post_date": "2021-10-27T09:59:01.033000", "content": "i have the same question due to being new here at Kaggle , hopefully someone can provide answer.
", "votes": 0, "replies": [ { "id": 1590711, "author_name": "quadcore/Richard Epstein", "author_url": "", "post_date": "2021-11-21T15:15:34.530000", "content": "You can take a long as you want to train. No limitations on training.
", "votes": 1, "replies": [] } ] }, { "id": 1581263, "author_name": "", "author_url": "", "post_date": "2021-11-13T15:12:19.373000", "content": "", "votes": -1, "replies": [] } ], "raw_markdown_by_id": { "1561054": "Can I train models on the local machine, and only submit the trained-weights and the inference notebook in this competetion ?", "1561092": "Hi, yes as far as i understand and did so in other competition, you can download the data on your laptop or computer - if this is meant by your local machine, you can train there and then for the submission you can use the weights and interfer from a new kaggle notebook. You can of course also train on google colab or any cloud service.\n ", "1590543": "The code competition requirements also include that the notebook must not run longer than 9 hours which is the upper limit for Kaggle notebooks. Does anyone know whether this also applies for the training notebooks? As an example, am I allowed to train over 2 notebook runs, each of them taking 8h (just an example) or must the complete training be doable within 9h?", "1561072": "i have the same question due to being new here at Kaggle , hopefully someone can provide answer.", "1581263": "" } }