If the organizers want solutions to incorporate ct-scan images they should give more time, even you save model weights and perform only inference on submission, processing 200 pacients from private test and make all predictions need more than 4 hours in my opinion.
", "messages": [ { "id": "998654", "postDate": "09/04/2020 23:00:17", "content": "If the organizers want solutions to incorporate ct-scan images they should give more time, even you save model weights and perform only inference on submission, processing 200 pacients from private test and make all predictions need more than 4 hours in my opinion.
", "rawMarkdown": "If the organizers want solutions to incorporate ct-scan images they should give more time, even you save model weights and perform only inference on submission, processing 200 pacients from private test and make all predictions need more than 4 hours in my opinion.", "votes": null }, { "id": "999077", "postDate": "09/05/2020 10:34:01", "content": "It would indeed be good to have more time, although some kagglers were able to train models and predict within the time frame. Have you benchmarked your notebook with some of the public notebooks?
\n\nWishing you success in the competition.,\nDoug
", "rawMarkdown": "It would indeed be good to have more time, although some kagglers were able to train models and predict within the time frame. Have you benchmarked your notebook with some of the public notebooks?\n\nWishing you success in the competition.,\nDoug", "votes": null }, { "id": "999114", "postDate": "09/05/2020 10:59:39", "content": "Of course if i make a Linear model i will be able to perform the predictions with in the time frame, but if i want to use image/scans features it´s impossible we need more time
", "rawMarkdown": "Of course if i make a Linear model i will be able to perform the predictions with in the time frame, but if i want to use image/scans features it´s impossible we need more time", "votes": null }, { "id": "999254", "postDate": "09/05/2020 14:03:36", "content": "I'm using a Neural Network model with both tabular and CT scans. My submission notebook finishes within 3.5 hours on the hidden test set using only CPU, so time limits set are reasonable.
\nAn important step is to first preprocess the ct scan images and cache them so that the data loader pipeline is efficient.
", "rawMarkdown": "I'm using a Neural Network model with both tabular and CT scans. My submission notebook finishes within 3.5 hours on the hidden test set using only CPU, so time limits set are reasonable.\n\nAn important step is to first preprocess the ct scan images and cache them so that the data loader pipeline is efficient.", "votes": null }, { "id": "1000316", "postDate": "09/06/2020 13:10:53", "content": "First of all, train separately, use trained models, and write a inference script only.
\nNo, the problem is my ct-scan preprocessing, i am working on improve performance and speed it up but maybe i will have to sacrifice a bit of performance ….
", "rawMarkdown": "No, the problem is my ct-scan preprocessing, i am working on improve performance and speed it up but maybe i will have to sacrifice a bit of performance ....", "votes": null } ], "comments": [ { "id": 1000316, "author_name": "furcifer", "author_url": "", "post_date": "09/06/2020 13:10:53", "content": "First of all, train separately, use trained models, and write a inference script only.
\nNo, the problem is my ct-scan preprocessing, i am working on improve performance and speed it up but maybe i will have to sacrifice a bit of performance ….
", "votes": null, "replies": [] } ] }, { "id": 999077, "author_name": "douglaskgaraujo", "author_url": "", "post_date": "09/05/2020 10:34:01", "content": "It would indeed be good to have more time, although some kagglers were able to train models and predict within the time frame. Have you benchmarked your notebook with some of the public notebooks?
\n\nWishing you success in the competition.,\nDoug
", "votes": null, "replies": [ { "id": 999114, "author_name": "enric1296", "author_url": "", "post_date": "09/05/2020 10:59:39", "content": "Of course if i make a Linear model i will be able to perform the predictions with in the time frame, but if i want to use image/scans features it´s impossible we need more time
", "votes": null, "replies": [] }, { "id": 999254, "author_name": "yovinyahathugoda", "author_url": "", "post_date": "09/05/2020 14:03:36", "content": "I'm using a Neural Network model with both tabular and CT scans. My submission notebook finishes within 3.5 hours on the hidden test set using only CPU, so time limits set are reasonable.
\nAn important step is to first preprocess the ct scan images and cache them so that the data loader pipeline is efficient.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "998654": "If the organizers want solutions to incorporate ct-scan images they should give more time, even you save model weights and perform only inference on submission, processing 200 pacients from private test and make all predictions need more than 4 hours in my opinion.", "999077": "It would indeed be good to have more time, although some kagglers were able to train models and predict within the time frame. Have you benchmarked your notebook with some of the public notebooks?\n\nWishing you success in the competition.,\nDoug", "999114": "Of course if i make a Linear model i will be able to perform the predictions with in the time frame, but if i want to use image/scans features it´s impossible we need more time", "999254": "I'm using a Neural Network model with both tabular and CT scans. My submission notebook finishes within 3.5 hours on the hidden test set using only CPU, so time limits set are reasonable.\n\nAn important step is to first preprocess the ct scan images and cache them so that the data loader pipeline is efficient.", "1000316": "First of all, train separately, use trained models, and write a inference script only.\n1. I don't think processing 200 patients is a big deal here, I used both CT scan features and tabular features (my script took 1.5 hours approx. with GPU), I didn't even use any multiprocessing.\n2. Use multi-processing if possible to speed up the data pipeline.\n3. I think the problem is your model which is slow, try to simplify the model if possible, if you're using a deep model which takes 3d scans + tabular features it would be slow, so try to downsample the scans or reduce layers.", "1002023": "No, the problem is my ct-scan preprocessing, i am working on improve performance and speed it up but maybe i will have to sacrifice a bit of performance ...." }, "source": "meta" }