{ "id": 175770, "title": "Notebook Timeout Error", "url": "/competitions/osic-pulmonary-fibrosis-progression/discussion/175770", "author_name": "Ajay Singh", "post_date": "2020-08-19T11:10:51.793000", "votes": 3, "comment_count": 7, "views": 0, "content": "

My notebook produced output with in 1 hour but getting time out error on submission. Any advises ?

", "messages": [ { "id": 977232, "postDate": "2020-08-19T11:10:51.793Z", "content": "

My notebook produced output with in 1 hour but getting time out error on submission. Any advises ?

", "rawMarkdown": "My notebook produced output with in 1 hour but getting time out error on submission. Any advises ?", "votes": 3 }, { "id": 977256, "postDate": "2020-08-19T11:31:05.590Z", "content": "

@ajay19 Can you provide more details like whether you are using CPU or GPU in your notebook? Also keep in mind that if you are using the DICOM images in your notebook you will have to optimize your code to be used with GPU so that it can be run on the full private test set of about 200 patients in less than 4 hours.

\n

If your notebook is taking 1 hour just to predict on the 5 sample test patients, then your data pipeline is too slow.

", "rawMarkdown": "@ajay19 Can you provide more details like whether you are using CPU or GPU in your notebook? Also keep in mind that if you are using the DICOM images in your notebook you will have to optimize your code to be used with GPU so that it can be run on the full private test set of about 200 patients in less than 4 hours.\n\nIf your notebook is taking 1 hour just to predict on the 5 sample test patients, then your data pipeline is too slow.", "votes": 1, "replies": [ { "id": 978228, "postDate": "2020-08-20T03:29:10.360Z", "content": "

Actually, I was first training on the data and then predicting so prediction was maybe 1 or 2 minutes but I think I should train in a separate notebook. And yes, GPU was on.

", "rawMarkdown": "Actually, I was first training on the data and then predicting so prediction was maybe 1 or 2 minutes but I think I should train in a separate notebook. And yes, GPU was on. " }, { "id": 978229, "postDate": "2020-08-20T03:29:10.367Z", "rawMarkdown": "", "isDeleted": true }, { "id": 978392, "postDate": "2020-08-20T06:38:19.387Z", "content": "

@ajay19 Yes, it's best to split the training and inference part of your notebook. For me just by looking at my submission run-times of my inference kernel using dicom images, it takes around 2.5 hours just to pre-process the the private dataset dicom images. The whole notebook takes a minimum of 3 hours to finish when i submit to the competition.

\n

So if you're training and predicting in the same notebook, it would definitely exceed the 4 hour limit using GPU. If your model is not heavily dependent on GPU maybe try using CPU only which gives you a total runtime limit of 9 hours.

", "rawMarkdown": "@ajay19 Yes, it's best to split the training and inference part of your notebook. For me just by looking at my submission run-times of my inference kernel using dicom images, it takes around 2.5 hours just to pre-process the the private dataset dicom images. The whole notebook takes a minimum of 3 hours to finish when i submit to the competition.\n\nSo if you're training and predicting in the same notebook, it would definitely exceed the 4 hour limit using GPU. If your model is not heavily dependent on GPU maybe try using CPU only which gives you a total runtime limit of 9 hours.", "votes": 1 }, { "id": 995803, "postDate": "2020-09-02T19:39:20.437Z", "content": "

@yovinyahathugoda Thanks for the answer. Could you tell me how you know the approximate number of patients?

", "rawMarkdown": "@yovinyahathugoda Thanks for the answer. Could you tell me how you know the approximate number of patients?" }, { "id": 995948, "postDate": "2020-09-03T01:22:26.707Z", "content": "

@yohannwattiez The competition organizers mentioned it in the following discussion.
\nhttps://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165723#948386

", "rawMarkdown": "@yohannwattiez The competition organizers mentioned it in the following discussion.\nhttps://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165723#948386", "votes": 1 }, { "id": 996590, "postDate": "2020-09-03T12:15:21.483Z", "content": "

@yovinyahathugoda Thank you very much !

", "rawMarkdown": "@yovinyahathugoda Thank you very much !" } ] } ], "comments": [ { "id": 977256, "author_name": "Yovin Yahathugoda", "author_url": "", "post_date": "2020-08-19T11:31:05.590000", "content": "

@ajay19 Can you provide more details like whether you are using CPU or GPU in your notebook? Also keep in mind that if you are using the DICOM images in your notebook you will have to optimize your code to be used with GPU so that it can be run on the full private test set of about 200 patients in less than 4 hours.

\n

If your notebook is taking 1 hour just to predict on the 5 sample test patients, then your data pipeline is too slow.

", "votes": 1, "replies": [ { "id": 978228, "author_name": "Ajay Singh", "author_url": "", "post_date": "2020-08-20T03:29:10.360000", "content": "

Actually, I was first training on the data and then predicting so prediction was maybe 1 or 2 minutes but I think I should train in a separate notebook. And yes, GPU was on.

", "votes": 0, "replies": [] }, { "id": 978229, "author_name": "", "author_url": "", "post_date": "2020-08-20T03:29:10.367000", "content": "", "votes": 0, "replies": [] }, { "id": 978392, "author_name": "Yovin Yahathugoda", "author_url": "", "post_date": "2020-08-20T06:38:19.387000", "content": "

@ajay19 Yes, it's best to split the training and inference part of your notebook. For me just by looking at my submission run-times of my inference kernel using dicom images, it takes around 2.5 hours just to pre-process the the private dataset dicom images. The whole notebook takes a minimum of 3 hours to finish when i submit to the competition.

\n

So if you're training and predicting in the same notebook, it would definitely exceed the 4 hour limit using GPU. If your model is not heavily dependent on GPU maybe try using CPU only which gives you a total runtime limit of 9 hours.

", "votes": 1, "replies": [] }, { "id": 995803, "author_name": "Yohann Wattiez", "author_url": "", "post_date": "2020-09-02T19:39:20.437000", "content": "

@yovinyahathugoda Thanks for the answer. Could you tell me how you know the approximate number of patients?

", "votes": 0, "replies": [] }, { "id": 995948, "author_name": "Yovin Yahathugoda", "author_url": "", "post_date": "2020-09-03T01:22:26.707000", "content": "

@yohannwattiez The competition organizers mentioned it in the following discussion.
\nhttps://www.kaggle.com/c/osic-pulmonary-fibrosis-progression/discussion/165723#948386

", "votes": 1, "replies": [] }, { "id": 996590, "author_name": "Yohann Wattiez", "author_url": "", "post_date": "2020-09-03T12:15:21.483000", "content": "

@yovinyahathugoda Thank you very much !

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "977232": "My notebook produced output with in 1 hour but getting time out error on submission. Any advises ?", "977256": "@ajay19 Can you provide more details like whether you are using CPU or GPU in your notebook? Also keep in mind that if you are using the DICOM images in your notebook you will have to optimize your code to be used with GPU so that it can be run on the full private test set of about 200 patients in less than 4 hours.\n\nIf your notebook is taking 1 hour just to predict on the 5 sample test patients, then your data pipeline is too slow." } }