Hey guys, I'm quite new to this as this is my first independent out-of-class machine learning kaggle competition. The training data for this is huge and I was wondering how I could feasibly train a dataset this large? I tried uploading the training data from Open Images to a google cloud project. But that endeavor has proved completely time consuming and useless. \nHow should I tackle this problem??
", "messages": [ { "id": 397492, "postDate": "2018-10-02T14:56:24.253Z", "content": "You can use a good gpu to work on
", "rawMarkdown": "You can use a good gpu to work on" }, { "id": 395624, "postDate": "2018-09-29T01:59:53.483Z", "content": "Start with 1 subset of the training dataset (32 GB). It should fit on a VM so you can write code to train a simple model.
", "rawMarkdown": "Start with 1 subset of the training dataset (32 GB). It should fit on a VM so you can write code to train a simple model." }, { "id": 395478, "postDate": "2018-09-28T16:44:16.920Z", "content": "well I got gsutils working but now cant give my self permission to my own bucket
", "rawMarkdown": "well I got gsutils working but now cant give my self permission to my own bucket" }, { "id": 395463, "postDate": "2018-09-28T16:17:21.967Z", "content": "I have also followed the directions online to try and install gsutil (didn't work) so using that is not an option
", "rawMarkdown": "I have also followed the directions online to try and install gsutil (didn't work) so using that is not an option" }, { "id": 395461, "postDate": "2018-09-28T16:15:49.797Z", "content": "Hey guys, I'm quite new to this as this is my first independent out-of-class machine learning kaggle competition. The training data for this is huge and I was wondering how I could feasibly train a dataset this large? I tried uploading the training data from Open Images to a google cloud project. But that endeavor has proved completely time consuming and useless. \nHow should I tackle this problem??
", "rawMarkdown": "Hey guys, I'm quite new to this as this is my first independent out-of-class machine learning kaggle competition. The training data for this is huge and I was wondering how I could feasibly train a dataset this large? I tried uploading the training data from Open Images to a google cloud project. But that endeavor has proved completely time consuming and useless. \nHow should I tackle this problem??" }, { "id": 399284, "postDate": "2018-10-05T14:42:57.583Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 400094, "postDate": "2018-10-07T15:27:08.117Z", "content": "Hard to answer your question without model architecture, batch size, image size, the amount of augmentation in mind.
\n\nOn my single gpu local machine, resnet50 with batch size 50, image of size (224, 224, 3) with no augmentation, took about a hour for 10000 batches.
", "rawMarkdown": "Hard to answer your question without model architecture, batch size, image size, the amount of augmentation in mind. \n\nOn my single gpu local machine, resnet50 with batch size 50, image of size (224, 224, 3) with no augmentation, took about a hour for 10000 batches.\n", "votes": 1 } ] } ], "comments": [ { "id": 397492, "author_name": "Abhishek Pande", "author_url": "", "post_date": "2018-10-02T14:56:24.253000", "content": "You can use a good gpu to work on
", "votes": 0, "replies": [] }, { "id": 395624, "author_name": "Alan Khoa Nguyen", "author_url": "", "post_date": "2018-09-29T01:59:53.483000", "content": "Start with 1 subset of the training dataset (32 GB). It should fit on a VM so you can write code to train a simple model.
", "votes": 0, "replies": [] }, { "id": 395478, "author_name": "TuomasTalvitie", "author_url": "", "post_date": "2018-09-28T16:44:16.920000", "content": "well I got gsutils working but now cant give my self permission to my own bucket
", "votes": 0, "replies": [] }, { "id": 395463, "author_name": "TuomasTalvitie", "author_url": "", "post_date": "2018-09-28T16:17:21.967000", "content": "I have also followed the directions online to try and install gsutil (didn't work) so using that is not an option
", "votes": 0, "replies": [] }, { "id": 399284, "author_name": "", "author_url": "", "post_date": "2018-10-05T14:42:57.583000", "content": "", "votes": 0, "replies": [ { "id": 400094, "author_name": "Ren", "author_url": "", "post_date": "2018-10-07T15:27:08.117000", "content": "Hard to answer your question without model architecture, batch size, image size, the amount of augmentation in mind.
\n\nOn my single gpu local machine, resnet50 with batch size 50, image of size (224, 224, 3) with no augmentation, took about a hour for 10000 batches.
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "397492": "You can use a good gpu to work on", "395624": "Start with 1 subset of the training dataset (32 GB). It should fit on a VM so you can write code to train a simple model.", "395478": "well I got gsutils working but now cant give my self permission to my own bucket", "395463": "I have also followed the directions online to try and install gsutil (didn't work) so using that is not an option", "395461": "Hey guys, I'm quite new to this as this is my first independent out-of-class machine learning kaggle competition. The training data for this is huge and I was wondering how I could feasibly train a dataset this large? I tried uploading the training data from Open Images to a google cloud project. But that endeavor has proved completely time consuming and useless. \nHow should I tackle this problem??", "399284": "" } }