Hello everyone,
\n\nFirst of all I would like to say that I am quite new to this website, I lurked around for a long time, but only recently started being active. I participated in the Kannada-MNIST competition, but that one had a way smaller dataset, so I have a few questions about how the notebooks work:
\n\nFrom reading the competition description I understood it's ok to train the model offline and just do the predictions in the notebook. Right now I have a local script that I've been building for this competition which contains both training, saving the model, loading a pretrained model and doing the predictions and I have some constants with which I can control how it behaves, however it's getting a bit difficult to manage. My question is how much of the training code should be present in the submission notebook? Do I need to have the code I used to train the model there, even if it isn't used? Or can i just have the code for load_model -> load_test_data -> do predictions -> write to file.
In the Kannada-MNIST competition I used a GPU notebook. However the fact that it's limited to only 30 hours/week made it a hell to manage as I was always trying to keep the gpu only when it was needed. Like I ran the notebook, saw some results and while I was thinking or changing the code I would switch off the GPU and back on when I was done. For this competition however since there is only 12 images in the initial test dataset, I don't need the GPU and I have it running on CPU. However I would like the notebook to use the GPU when submitting (when the test data is a lot bigger) as it currently takes quite a while for a submission (this may also be because my code isn't ideal as I realized I forgot some debug print lines there). I would like to have a checkbox to run on GPU only when submitting to the competition, without having to set the notebook to gpu when commiting it.
Well this was meant to be a small post with a few questions, but with all the explanations and details it got quite long :). Thanks in advance.
", "messages": [ { "id": "705804", "postDate": "12/29/2019 13:50:46", "content": "Hello everyone,
\n\nFirst of all I would like to say that I am quite new to this website, I lurked around for a long time, but only recently started being active. I participated in the Kannada-MNIST competition, but that one had a way smaller dataset, so I have a few questions about how the notebooks work:
\n\nFrom reading the competition description I understood it's ok to train the model offline and just do the predictions in the notebook. Right now I have a local script that I've been building for this competition which contains both training, saving the model, loading a pretrained model and doing the predictions and I have some constants with which I can control how it behaves, however it's getting a bit difficult to manage. My question is how much of the training code should be present in the submission notebook? Do I need to have the code I used to train the model there, even if it isn't used? Or can i just have the code for load_model -> load_test_data -> do predictions -> write to file.
In the Kannada-MNIST competition I used a GPU notebook. However the fact that it's limited to only 30 hours/week made it a hell to manage as I was always trying to keep the gpu only when it was needed. Like I ran the notebook, saw some results and while I was thinking or changing the code I would switch off the GPU and back on when I was done. For this competition however since there is only 12 images in the initial test dataset, I don't need the GPU and I have it running on CPU. However I would like the notebook to use the GPU when submitting (when the test data is a lot bigger) as it currently takes quite a while for a submission (this may also be because my code isn't ideal as I realized I forgot some debug print lines there). I would like to have a checkbox to run on GPU only when submitting to the competition, without having to set the notebook to gpu when commiting it.
Well this was meant to be a small post with a few questions, but with all the explanations and details it got quite long :). Thanks in advance.
", "rawMarkdown": "Hello everyone,\n\nFirst of all I would like to say that I am quite new to this website, I lurked around for a long time, but only recently started being active. I participated in the Kannada-MNIST competition, but that one had a way smaller dataset, so I have a few questions about how the notebooks work:\n\n1. From reading the competition description I understood it's ok to train the model offline and just do the predictions in the notebook. Right now I have a local script that I've been building for this competition which contains both training, saving the model, loading a pretrained model and doing the predictions and I have some constants with which I can control how it behaves, however it's getting a bit difficult to manage. My question is how much of the training code should be present in the submission notebook? Do I need to have the code I used to train the model there, even if it isn't used? Or can i just have the code for load_model -> load_test_data -> do predictions -> write to file.\n\n2. In the Kannada-MNIST competition I used a GPU notebook. However the fact that it's limited to only 30 hours/week made it a hell to manage as I was always trying to keep the gpu only when it was needed. Like I ran the notebook, saw some results and while I was thinking or changing the code I would switch off the GPU and back on when I was done. For this competition however since there is only 12 images in the initial test dataset, I don't need the GPU and I have it running on CPU. However I would like the notebook to use the GPU when submitting (when the test data is a lot bigger) as it currently takes quite a while for a submission (this may also be because my code isn't ideal as I realized I forgot some debug print lines there). I would like to have a checkbox to run on GPU only when submitting to the competition, without having to set the notebook to gpu when commiting it.\n\nWell this was meant to be a small post with a few questions, but with all the explanations and details it got quite long :). Thanks in advance.", "votes": null }, { "id": "705807", "postDate": "12/29/2019 14:04:10", "content": "You don't have to train your model in the submission notebook. load the trained model and do predictions.\nfor the second part as you know the available test set contains only 12 images so saved time by doing that on CPU is not much plus we don't have to modify the inference notebook most of the time anyways.
", "rawMarkdown": "You don't have to train your model in the submission notebook. load the trained model and do predictions.\nfor the second part as you know the available test set contains only 12 images so saved time by doing that on CPU is not much plus we don't have to modify the inference notebook most of the time anyways.", "votes": null } ], "comments": [ { "id": 705807, "author_name": "dhananjay3", "author_url": "", "post_date": "12/29/2019 14:04:10", "content": "You don't have to train your model in the submission notebook. load the trained model and do predictions.\nfor the second part as you know the available test set contains only 12 images so saved time by doing that on CPU is not much plus we don't have to modify the inference notebook most of the time anyways.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "705804": "Hello everyone,\n\nFirst of all I would like to say that I am quite new to this website, I lurked around for a long time, but only recently started being active. I participated in the Kannada-MNIST competition, but that one had a way smaller dataset, so I have a few questions about how the notebooks work:\n\n1. From reading the competition description I understood it's ok to train the model offline and just do the predictions in the notebook. Right now I have a local script that I've been building for this competition which contains both training, saving the model, loading a pretrained model and doing the predictions and I have some constants with which I can control how it behaves, however it's getting a bit difficult to manage. My question is how much of the training code should be present in the submission notebook? Do I need to have the code I used to train the model there, even if it isn't used? Or can i just have the code for load_model -> load_test_data -> do predictions -> write to file.\n\n2. In the Kannada-MNIST competition I used a GPU notebook. However the fact that it's limited to only 30 hours/week made it a hell to manage as I was always trying to keep the gpu only when it was needed. Like I ran the notebook, saw some results and while I was thinking or changing the code I would switch off the GPU and back on when I was done. For this competition however since there is only 12 images in the initial test dataset, I don't need the GPU and I have it running on CPU. However I would like the notebook to use the GPU when submitting (when the test data is a lot bigger) as it currently takes quite a while for a submission (this may also be because my code isn't ideal as I realized I forgot some debug print lines there). I would like to have a checkbox to run on GPU only when submitting to the competition, without having to set the notebook to gpu when commiting it.\n\nWell this was meant to be a small post with a few questions, but with all the explanations and details it got quite long :). Thanks in advance.", "705807": "You don't have to train your model in the submission notebook. load the trained model and do predictions.\nfor the second part as you know the available test set contains only 12 images so saved time by doing that on CPU is not much plus we don't have to modify the inference notebook most of the time anyways." }, "source": "meta" }