Hi Administrator,
\n\nWe use Caffe and several servers to train our models.\nConsidering that our models is really heavy and it may take quite a long time to retrain the model, I wonder if we need to exactly reproduce our result from training phase in the testing platform.\nA whole procedure of retraining our model with a NVIDIA TITAN X may take several months. And the batch size can't be as big as we used on our server.
\n\nRegards,
\n\nJinwei
", "messages": [ { "id": "110502", "postDate": "03/06/2016 04:28:34", "content": "Hi Administrator,
\n\nWe use Caffe and several servers to train our models.\nConsidering that our models is really heavy and it may take quite a long time to retrain the model, I wonder if we need to exactly reproduce our result from training phase in the testing platform.\nA whole procedure of retraining our model with a NVIDIA TITAN X may take several months. And the batch size can't be as big as we used on our server.
\n\nRegards,
\n\nJinwei
", "rawMarkdown": "Hi Administrator,\r\n\r\nWe use Caffe and several servers to train our models.\r\nConsidering that our models is really heavy and it may take quite a long time to retrain the model, I wonder if we need to exactly reproduce our result from training phase in the testing platform.\r\nA whole procedure of retraining our model with a NVIDIA TITAN X may take several months. And the batch size can't be as big as we used on our server.\r\n\r\nRegards,\r\n\r\nJinwei", "votes": null }, { "id": "110567", "postDate": "03/06/2016 18:12:03", "content": "Can you clarify what you mean by "reproduce our result from training phase in the testing platform"?
\n\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.
", "rawMarkdown": "Can you clarify what you mean by \"reproduce our result from training phase in the testing platform\"?\r\n\r\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.", "votes": null }, { "id": "110615", "postDate": "03/07/2016 00:38:01", "content": "[quote=William Cukierski;110567]
\n\nCan you clarify what you mean by "reproduce our result from training phase in the testing platform"?
\n\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.
\n\n[/quote]
\n\nMany thanks for your reply.\n"Reproduce our result from training phase in the testing platform" means retraining our model on the test platform of Kaggle. Since it may takes months to retrain our model on a single NVIDIA TITAN X, the training phase may not be easliy reproducible.\nOn our platform, retrain our model may take several days. So are we allowed to retrain our model with the validation set just using the uploaded training code?\nBy the way, because our labeling process is a little bit slow, are we allowed to upload new hand labeled data on train and validation set after March 7th?
", "rawMarkdown": "[quote=William Cukierski;110567]\r\n\r\nCan you clarify what you mean by \"reproduce our result from training phase in the testing platform\"?\r\n\r\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.\r\n\r\n[/quote]\r\n\r\nMany thanks for your reply.\r\n\"Reproduce our result from training phase in the testing platform\" means retraining our model on the test platform of Kaggle. Since it may takes months to retrain our model on a single NVIDIA TITAN X, the training phase may not be easliy reproducible.\r\nOn our platform, retrain our model may take several days. So are we allowed to retrain our model with the validation set just using the uploaded training code?\r\nBy the way, because our labeling process is a little bit slow, are we allowed to upload new hand labeled data on train and validation set after March 7th?", "votes": null }, { "id": "110619", "postDate": "03/07/2016 01:53:03", "content": "If you win, we'd work out a way to verify the code.
\n\n\n\n\n"are we allowed to retrain our model with the validation set just using the uploaded training code"
\n
I think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.
\n\n\n\n\n"are we allowed to upload new hand labeled data on train and validation set after March 7th?"
\n
There are no more uploads after tomorrow's deadline. You may hand annotate the validation set.
", "rawMarkdown": "If you win, we'd work out a way to verify the code.\r\n\r\n> \"are we allowed to retrain our model with the validation set just using the uploaded training code\"\r\n\r\nI think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.\r\n\r\n> \"are we allowed to upload new hand labeled data on train and validation set after March 7th?\"\r\n\r\nThere are no more uploads after tomorrow's deadline. You may hand annotate the validation set.", "votes": null }, { "id": "110629", "postDate": "03/07/2016 06:11:39", "content": "[quote=William Cukierski;110619]
\n\nIf you win, we'd work out a way to verify the code.
\n\n\n\n\n"are we allowed to retrain our model with the validation set just using the uploaded training code"
\n
I think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.
\n\n\n\n\n"are we allowed to upload new hand labeled data on train and validation set after March 7th?"
\n
There are no more uploads after tomorrow's deadline. You should not be hand labeling and using those results after the deadline, since it would mean your submitted model is not sufficient to reproduce your final submission.
\n\n[/quote]
\n\nThanks for your reply.
\n\n"There are no more uploads after tomorrow's deadline. "\nSo how can we upload our retrained CNN models?
", "rawMarkdown": "[quote=William Cukierski;110619]\r\n\r\nIf you win, we'd work out a way to verify the code.\r\n\r\n> \"are we allowed to retrain our model with the validation set just using the uploaded training code\"\r\n\r\nI think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.\r\n\r\n> \"are we allowed to upload new hand labeled data on train and validation set after March 7th?\"\r\n\r\nThere are no more uploads after tomorrow's deadline. You should not be hand labeling and using those results after the deadline, since it would mean your submitted model is not sufficient to reproduce your final submission.\r\n\r\n[/quote]\r\n\r\nThanks for your reply. \r\n\r\n\"There are no more uploads after tomorrow's deadline. \"\r\nSo how can we upload our retrained CNN models?", "votes": null }, { "id": "110631", "postDate": "03/07/2016 06:24:24", "content": "Here it is said that labeling patients 500-700 IS allowed in the 2nd part and not in the 1st part.
\n\n\n\nMy plan was to hand-annotate patient 500-700 after 7 march. (ie. centers of heart to regress on)\nI can also do it today and include it in my model upload.\nShould I do that then ?
", "rawMarkdown": "Here it is said that labeling patients 500-700 IS allowed in the 2nd part and not in the 1st part.\r\n\r\nhttps://www.kaggle.com/c/second-annual-data-science-bowl/forums/t/18599/questions-about-hand-labeling/109809\r\n\r\nMy plan was to hand-annotate patient 500-700 after 7 march. (ie. centers of heart to regress on)\r\nI can also do it today and include it in my model upload.\r\nShould I do that then ?", "votes": null }, { "id": "110667", "postDate": "03/07/2016 13:00:41", "content": "@Jinwei I spoke without thinking in my previous comment. We are allowing hand annotation on the validation set (because it is tantamount to retraining on the validation set, with similar pros and cons). I'll update my comment.
\n\nYou don't upload your final retrained models unless you are a prize winner.
", "rawMarkdown": "Jinwei I spoke without thinking in my previous comment. We are allowing hand annotation on the validation set (because it is tantamount to retraining on the validation set, with similar pros and cons). I'll update my comment.\r\n\r\nYou don't upload your final retrained models unless you are a prize winner.", "votes": null } ], "comments": [ { "id": 110567, "author_name": "wcukierski", "author_url": "", "post_date": "03/06/2016 18:12:03", "content": "Can you clarify what you mean by "reproduce our result from training phase in the testing platform"?
\n\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.
", "votes": null, "replies": [] }, { "id": 110615, "author_name": "wjw8090711", "author_url": "", "post_date": "03/07/2016 00:38:01", "content": "[quote=William Cukierski;110567]
\n\nCan you clarify what you mean by "reproduce our result from training phase in the testing platform"?
\n\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.
\n\n[/quote]
\n\nMany thanks for your reply.\n"Reproduce our result from training phase in the testing platform" means retraining our model on the test platform of Kaggle. Since it may takes months to retrain our model on a single NVIDIA TITAN X, the training phase may not be easliy reproducible.\nOn our platform, retrain our model may take several days. So are we allowed to retrain our model with the validation set just using the uploaded training code?\nBy the way, because our labeling process is a little bit slow, are we allowed to upload new hand labeled data on train and validation set after March 7th?
", "votes": null, "replies": [] }, { "id": 110619, "author_name": "wcukierski", "author_url": "", "post_date": "03/07/2016 01:53:03", "content": "If you win, we'd work out a way to verify the code.
\n\n\n\n\n"are we allowed to retrain our model with the validation set just using the uploaded training code"
\n
I think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.
\n\n\n\n\n"are we allowed to upload new hand labeled data on train and validation set after March 7th?"
\n
There are no more uploads after tomorrow's deadline. You may hand annotate the validation set.
", "votes": null, "replies": [] }, { "id": 110629, "author_name": "wjw8090711", "author_url": "", "post_date": "03/07/2016 06:11:39", "content": "[quote=William Cukierski;110619]
\n\nIf you win, we'd work out a way to verify the code.
\n\n\n\n\n"are we allowed to retrain our model with the validation set just using the uploaded training code"
\n
I think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.
\n\n\n\n\n"are we allowed to upload new hand labeled data on train and validation set after March 7th?"
\n
There are no more uploads after tomorrow's deadline. You should not be hand labeling and using those results after the deadline, since it would mean your submitted model is not sufficient to reproduce your final submission.
\n\n[/quote]
\n\nThanks for your reply.
\n\n"There are no more uploads after tomorrow's deadline. "\nSo how can we upload our retrained CNN models?
", "votes": null, "replies": [] }, { "id": 110631, "author_name": "juliandewit", "author_url": "", "post_date": "03/07/2016 06:24:24", "content": "Here it is said that labeling patients 500-700 IS allowed in the 2nd part and not in the 1st part.
\n\n\n\nMy plan was to hand-annotate patient 500-700 after 7 march. (ie. centers of heart to regress on)\nI can also do it today and include it in my model upload.\nShould I do that then ?
", "votes": null, "replies": [] }, { "id": 110667, "author_name": "wcukierski", "author_url": "", "post_date": "03/07/2016 13:00:41", "content": "@Jinwei I spoke without thinking in my previous comment. We are allowing hand annotation on the validation set (because it is tantamount to retraining on the validation set, with similar pros and cons). I'll update my comment.
\n\nYou don't upload your final retrained models unless you are a prize winner.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "110502": "Hi Administrator,\r\n\r\nWe use Caffe and several servers to train our models.\r\nConsidering that our models is really heavy and it may take quite a long time to retrain the model, I wonder if we need to exactly reproduce our result from training phase in the testing platform.\r\nA whole procedure of retraining our model with a NVIDIA TITAN X may take several months. And the batch size can't be as big as we used on our server.\r\n\r\nRegards,\r\n\r\nJinwei", "110567": "Can you clarify what you mean by \"reproduce our result from training phase in the testing platform\"?\r\n\r\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.", "110615": "[quote=William Cukierski;110567]\r\n\r\nCan you clarify what you mean by \"reproduce our result from training phase in the testing platform\"?\r\n\r\nYour model must reproduce your selected submission(s). If it takes months to retrain on new training data, you would have to just apply the model as-is to the new test data.\r\n\r\n[/quote]\r\n\r\nMany thanks for your reply.\r\n\"Reproduce our result from training phase in the testing platform\" means retraining our model on the test platform of Kaggle. Since it may takes months to retrain our model on a single NVIDIA TITAN X, the training phase may not be easliy reproducible.\r\nOn our platform, retrain our model may take several days. So are we allowed to retrain our model with the validation set just using the uploaded training code?\r\nBy the way, because our labeling process is a little bit slow, are we allowed to upload new hand labeled data on train and validation set after March 7th?", "110619": "If you win, we'd work out a way to verify the code.\r\n\r\n> \"are we allowed to retrain our model with the validation set just using the uploaded training code\"\r\n\r\nI think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.\r\n\r\n> \"are we allowed to upload new hand labeled data on train and validation set after March 7th?\"\r\n\r\nThere are no more uploads after tomorrow's deadline. You may hand annotate the validation set.", "110629": "[quote=William Cukierski;110619]\r\n\r\nIf you win, we'd work out a way to verify the code.\r\n\r\n> \"are we allowed to retrain our model with the validation set just using the uploaded training code\"\r\n\r\nI think I'm still not understanding, as this sounds like it's exactly what you should be doing. You are allowed to retrain your model on the train + validation set together. The code you use to do this is the code you upload for tomorrow's deadline.\r\n\r\n> \"are we allowed to upload new hand labeled data on train and validation set after March 7th?\"\r\n\r\nThere are no more uploads after tomorrow's deadline. You should not be hand labeling and using those results after the deadline, since it would mean your submitted model is not sufficient to reproduce your final submission.\r\n\r\n[/quote]\r\n\r\nThanks for your reply. \r\n\r\n\"There are no more uploads after tomorrow's deadline. \"\r\nSo how can we upload our retrained CNN models?", "110631": "Here it is said that labeling patients 500-700 IS allowed in the 2nd part and not in the 1st part.\r\n\r\nhttps://www.kaggle.com/c/second-annual-data-science-bowl/forums/t/18599/questions-about-hand-labeling/109809\r\n\r\nMy plan was to hand-annotate patient 500-700 after 7 march. (ie. centers of heart to regress on)\r\nI can also do it today and include it in my model upload.\r\nShould I do that then ?", "110667": "Jinwei I spoke without thinking in my previous comment. We are allowing hand annotation on the validation set (because it is tantamount to retraining on the validation set, with similar pros and cons). I'll update my comment.\r\n\r\nYou don't upload your final retrained models unless you are a prize winner." }, "source": "meta" }