![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2F7caa478cef50e25faf2d1c23f63f3ddd%2Fp1.png?generation=1573595772004320&alt=media\")
Kaggle just added a new Dataset tier to everyone's Kaggle profile. So, I decided to create a dataset here to help everyone save TIME and MONEY! And maybe some of you will upvote it :P
\n\nTo find the optimal setup (architecture, loss, optimizer, training schedule, etc) requires lots of experiments. Ideas can be tested on smaller cloud images to save time.
\n\nI created a Kaggle dataset with folders containing resized training and test images for sizes 384x576, 320x480, 256x384, 192x288, 128x192, and 64x96. Also each folder has an associated train_WxH.csv with resized masks. (I used Hieu's great notebook here to create the images with INTERPOLATION = cv2.INTER_AREA).
Use your existing code and attach this Kaggle dataset to your Kaggle notebook. Then change the name of train_images folder to train_images_320x480 or whatever. And change the name of train.csv to train_320x480.csv. (And update calls to rle2mask to use new dim). That's it, then your experiments will run quicker and you will save TIME.
When you're ready to make a submission, you can use switch back to a larger image folder for higher CV and LB score.
\n\nMost of us need to stop training, predict validation masks, and apply post process, to compute an accurate validation Dice score. Instead of doing that, you can include post process directly in your Keras model metric. You will immediately know how your model is performing without having to waste valuable time stopping to check.
\n\ndef get_kaggle_dice(pix=0.5, dim=(320,480)):\n def kaggle_dice(y_true, y_pred0, pix=pix, dim=dim):\n\n # PIXEL THRESHOLD\n y_pred = K.cast( K.greater(y_pred0,pix), K.floatx() )\n\n # MIN AREA THRESHOLD\n area = 20000.*dim[0]/350.*dim[1]/525.\n s = K.sum(y_pred, axis=(1,2))\n s = K.cast( K.greater(s, area), K.floatx() )\n\n # REMOVE MIN AREA\n s = K.reshape(s,(-1,1))\n s = K.repeat(s,dim[0]*dim[1])\n s = K.reshape(s,(-1,1))\n y_pred = K.permute_dimensions(y_pred,(0,3,1,2))\n y_pred = K.reshape(y_pred,shape=(-1,1))\n y_pred = s*y_pred\n y_pred = K.reshape(y_pred,(-1,y_pred0.shape[3],dim[0],dim[1]))\n y_pred = K.permute_dimensions(y_pred,(0,2,3,1))\n\n # COMPUTE KAGGLE DICE\n intersection = K.sum(y_true * y_pred, axis=(1,2))\n total_y_true = K.sum(y_true, axis=(1,2))\n total_y_pred = K.sum(y_pred, axis=(1,2))\n return K.mean( (2*intersection+1e-9) / (total_y_true+total_y_pred+1e-9) )\n\n return kaggle_dice\nThen you include this in your Keras model shown below. Change (320,480) to the image size that you are using.
metric = get_kaggle_dice(pix=0.5, dim=(320,480)) \nmodel.compile(optimizer=opt, loss=loss, metrics=[metric])\nGoogle Colab is offering free online GPU notebooks. Simply click here and begin! Then follow the steps below to start training your cloud models
\n\nAfter clicking the link above, add a GPU to your notebook, by selecting \"Runtime\", \"Change runtime type\", \"Hardware accelerator\", \"GPU\"\n
Then choose GPU
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2Fdbe5cf159bfc3eee4ff8535ef3eebb36%2Fp2.png?generation=1573595833983800&alt=media\")
Transfer my \"Cloud Images Resized\" Kaggle dataset to Google Colab as follows. First do the first 6 steps from this Stack Overflow post here. Afterward type
\n\n!kaggle datasets download -d Cloud-Images-Resized\nand next execute
\n\nimport zipfile\nwith zipfile.ZipFile('Cloud-Images-Resized.zip', 'r') as zip_ref:\n zip_ref.extractall('.')\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2Fd3d206973476204ea0fa52e3fd7e2e16%2Fp4.png?generation=1573596944665825&alt=media\")
Next upload your model code and train. My \"Cloud Images Resized\" dataset has all the files you need including train_images_WxH, test_images_WxH, train_WxH.csv, and sample_submission.csv.
You can make a submission to our competition directly from Google Colab by typing
\n\n!kaggle competitions understanding_cloud_organization -f submission.csv -m message\nOr you can save your model and download to your local machine. The image below shows how. After download, infer the test images locally or upload your model to Kaggle notebook and infer there.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2Ffa5f4ee24c560d3072c1dbc0c8530931%2Fp6.png?generation=1573597242931290&alt=media\")
Just last week, Kaggle added a new way to achieve medals at Kaggle. We are now encouraged to create helpful datasets for other Kagglers. More info about this new tier is here. Enjoy the dataset, and enjoy saving TIME and MONEY!
", "messages": [ { "id": 671587, "postDate": "2019-11-12T23:03:55.410Z", "content": "Kaggle just added a new Dataset tier to everyone's Kaggle profile. So, I decided to create a dataset here to help everyone save TIME and MONEY! And maybe some of you will upvote it :P
\n\nTo find the optimal setup (architecture, loss, optimizer, training schedule, etc) requires lots of experiments. Ideas can be tested on smaller cloud images to save time.
\n\nI created a Kaggle dataset with folders containing resized training and test images for sizes 384x576, 320x480, 256x384, 192x288, 128x192, and 64x96. Also each folder has an associated train_WxH.csv with resized masks. (I used Hieu's great notebook here to create the images with INTERPOLATION = cv2.INTER_AREA).
Use your existing code and attach this Kaggle dataset to your Kaggle notebook. Then change the name of train_images folder to train_images_320x480 or whatever. And change the name of train.csv to train_320x480.csv. (And update calls to rle2mask to use new dim). That's it, then your experiments will run quicker and you will save TIME.
When you're ready to make a submission, you can use switch back to a larger image folder for higher CV and LB score.
\n\nMost of us need to stop training, predict validation masks, and apply post process, to compute an accurate validation Dice score. Instead of doing that, you can include post process directly in your Keras model metric. You will immediately know how your model is performing without having to waste valuable time stopping to check.
\n\ndef get_kaggle_dice(pix=0.5, dim=(320,480)):\n def kaggle_dice(y_true, y_pred0, pix=pix, dim=dim):\n\n # PIXEL THRESHOLD\n y_pred = K.cast( K.greater(y_pred0,pix), K.floatx() )\n\n # MIN AREA THRESHOLD\n area = 20000.*dim[0]/350.*dim[1]/525.\n s = K.sum(y_pred, axis=(1,2))\n s = K.cast( K.greater(s, area), K.floatx() )\n\n # REMOVE MIN AREA\n s = K.reshape(s,(-1,1))\n s = K.repeat(s,dim[0]*dim[1])\n s = K.reshape(s,(-1,1))\n y_pred = K.permute_dimensions(y_pred,(0,3,1,2))\n y_pred = K.reshape(y_pred,shape=(-1,1))\n y_pred = s*y_pred\n y_pred = K.reshape(y_pred,(-1,y_pred0.shape[3],dim[0],dim[1]))\n y_pred = K.permute_dimensions(y_pred,(0,2,3,1))\n\n # COMPUTE KAGGLE DICE\n intersection = K.sum(y_true * y_pred, axis=(1,2))\n total_y_true = K.sum(y_true, axis=(1,2))\n total_y_pred = K.sum(y_pred, axis=(1,2))\n return K.mean( (2*intersection+1e-9) / (total_y_true+total_y_pred+1e-9) )\n\n return kaggle_dice\nThen you include this in your Keras model shown below. Change (320,480) to the image size that you are using.
metric = get_kaggle_dice(pix=0.5, dim=(320,480)) \nmodel.compile(optimizer=opt, loss=loss, metrics=[metric])\nGoogle Colab is offering free online GPU notebooks. Simply click here and begin! Then follow the steps below to start training your cloud models
\n\nAfter clicking the link above, add a GPU to your notebook, by selecting \"Runtime\", \"Change runtime type\", \"Hardware accelerator\", \"GPU\"\n
Then choose GPU
\n\n
Transfer my \"Cloud Images Resized\" Kaggle dataset to Google Colab as follows. First do the first 6 steps from this Stack Overflow post here. Afterward type
\n\n!kaggle datasets download -d Cloud-Images-Resized\nand next execute
\n\nimport zipfile\nwith zipfile.ZipFile('Cloud-Images-Resized.zip', 'r') as zip_ref:\n zip_ref.extractall('.')\n
Next upload your model code and train. My \"Cloud Images Resized\" dataset has all the files you need including train_images_WxH, test_images_WxH, train_WxH.csv, and sample_submission.csv.
You can make a submission to our competition directly from Google Colab by typing
\n\n!kaggle competitions understanding_cloud_organization -f submission.csv -m message\nOr you can save your model and download to your local machine. The image below shows how. After download, infer the test images locally or upload your model to Kaggle notebook and infer there.
\n\n
Just last week, Kaggle added a new way to achieve medals at Kaggle. We are now encouraged to create helpful datasets for other Kagglers. More info about this new tier is here. Enjoy the dataset, and enjoy saving TIME and MONEY!
", "rawMarkdown": "Kaggle just added a new Dataset tier to everyone's Kaggle profile. So, I decided to create a dataset [here][2] to help everyone save TIME and MONEY! And maybe some of you will upvote it :P\n \n# How to Save Time\n## Step 1\nTo find the optimal setup (architecture, loss, optimizer, training schedule, etc) requires lots of experiments. Ideas can be tested on smaller cloud images to save time.\n\nI created a Kaggle [dataset][2] with folders containing resized training and test images for sizes `384x576`, `320x480`, `256x384`, `192x288`, `128x192`, and `64x96`. Also each folder has an associated `train_WxH.csv` with resized masks. (I used Hieu's great notebook [here][5] to create the images with `INTERPOLATION = cv2.INTER_AREA`).\n\nUse your existing code and attach this Kaggle dataset to your Kaggle notebook. Then change the name of `train_images` folder to `train_images_320x480` or whatever. And change the name of `train.csv` to `train_320x480.csv`. (And update calls to `rle2mask` to use new dim). That's it, then your experiments will run quicker and you will save TIME.\n\nWhen you're ready to make a submission, you can use switch back to a larger image folder for higher CV and LB score.\n \n## Step 2\nMost of us need to stop training, predict validation masks, and apply post process, to compute an accurate validation Dice score. Instead of doing that, you can include post process directly in your Keras model metric. You will immediately know how your model is performing without having to waste valuable time stopping to check.\n\n def get_kaggle_dice(pix=0.5, dim=(320,480)):\n def kaggle_dice(y_true, y_pred0, pix=pix, dim=dim):\n \n # PIXEL THRESHOLD\n y_pred = K.cast( K.greater(y_pred0,pix), K.floatx() )\n \n # MIN AREA THRESHOLD\n area = 20000.*dim[0]/350.*dim[1]/525.\n s = K.sum(y_pred, axis=(1,2))\n s = K.cast( K.greater(s, area), K.floatx() )\n\n # REMOVE MIN AREA\n s = K.reshape(s,(-1,1))\n s = K.repeat(s,dim[0]*dim[1])\n s = K.reshape(s,(-1,1))\n y_pred = K.permute_dimensions(y_pred,(0,3,1,2))\n y_pred = K.reshape(y_pred,shape=(-1,1))\n y_pred = s*y_pred\n y_pred = K.reshape(y_pred,(-1,y_pred0.shape[3],dim[0],dim[1]))\n y_pred = K.permute_dimensions(y_pred,(0,2,3,1))\n\n # COMPUTE KAGGLE DICE\n intersection = K.sum(y_true * y_pred, axis=(1,2))\n total_y_true = K.sum(y_true, axis=(1,2))\n total_y_pred = K.sum(y_pred, axis=(1,2))\n return K.mean( (2*intersection+1e-9) / (total_y_true+total_y_pred+1e-9) )\n\n return kaggle_dice\n\nThen you include this in your Keras model shown below. Change `(320,480)` to the image size that you are using.\n \n metric = get_kaggle_dice(pix=0.5, dim=(320,480)) \n model.compile(optimizer=opt, loss=loss, metrics=[metric])\n\n\n# How to Save Money \nGoogle Colab is offering free online GPU notebooks. Simply click [here][1] and begin! Then follow the steps below to start training your cloud models\n \n## Step 1\nAfter clicking the link above, add a GPU to your notebook, by selecting \"Runtime\", \"Change runtime type\", \"Hardware accelerator\", \"GPU\"\n\n \nThen choose GPU\n\n\n## Step 2\nTransfer my [\"Cloud Images Resized\" Kaggle dataset][2] to Google Colab as follows. First do the first 6 steps from this Stack Overflow post [here][3]. Afterward type\n\n !kaggle datasets download -d Cloud-Images-Resized\n\nand next execute\n\n import zipfile\n with zipfile.ZipFile('Cloud-Images-Resized.zip', 'r') as zip_ref:\n zip_ref.extractall('.')\n\n \n## Step 3 Train your model\nNext upload your model code and train. My \"Cloud Images Resized\" dataset has all the files you need including `train_images_WxH`, `test_images_WxH`, `train_WxH.csv`, and `sample_submission.csv`. \n\n## Step 4 Submit from Colab OR Download your model\nYou can make a submission to our competition directly from Google Colab by typing\n \n !kaggle competitions understanding_cloud_organization -f submission.csv -m message\n \nOr you can save your model and download to your local machine. The image below shows how. After download, infer the test images locally or upload your model to Kaggle notebook and infer there.\n \n\n \n# New Kaggle Dataset Medal Progression System\nJust last week, Kaggle added a new way to achieve medals at Kaggle. We are now encouraged to create helpful datasets for other Kagglers. More info about this new tier is [here][4]. Enjoy the dataset, and enjoy saving TIME and MONEY!\n\n\n[1]: https://colab.research.google.com/notebook#create=true&language=python3\n[2]: https://www.kaggle.com/cdeotte/cloud-images-resized\n[3]: https://stackoverflow.com/questions/49310470/using-kaggle-datasets-in-google-colab\n[4]: https://www.kaggle.com/product-feedback/116079\n[5]: https://www.kaggle.com/phunghieu/dataset-preparation-resize-images", "votes": 68 }, { "id": 673049, "postDate": "2019-11-14T12:56:21.387Z", "content": "Also, if anyone wants more ram on colab, just crash the session by exploding the memory usage. It will give you an option to increase the ram from 13gigs to 26gigs.\nUsually, I use\n\nd = []\nwhile 1: d.append('1')\n
Thanks for sharing this.
", "rawMarkdown": "Thanks for sharing this." }, { "id": 674733, "postDate": "2019-11-17T01:56:41.387Z", "content": "wow!
", "rawMarkdown": "wow!" }, { "id": 680247, "postDate": "2019-11-24T10:19:28.727Z", "content": "A small trick to make it crash faster :p
\n\n\nd = []
\nwhile 1:
\n d.append(d + ['1'])
\n
That's really great! Thank you for your helpfulness!
", "rawMarkdown": "That's really great! Thank you for your helpfulness! ", "votes": 3 }, { "id": 693114, "postDate": "2019-12-12T03:35:49.100Z", "content": "Good job, I just begin to use colabπ @cdeotte
", "rawMarkdown": "Good job, I just begin to use colabπ @cdeotte ", "votes": 1 }, { "id": 673723, "postDate": "2019-11-15T11:55:05.933Z", "content": "π Hakunamatata! π
", "rawMarkdown": "π Hakunamatata! π ", "votes": 1 }, { "id": 671724, "postDate": "2019-11-13T04:53:04.010Z", "content": "thanks for sharing this @cdeotte .............!!!
", "rawMarkdown": "thanks for sharing this @cdeotte .............!!!", "votes": 1 }, { "id": 671668, "postDate": "2019-11-13T02:50:02.250Z", "content": "That's really great! Thank you for Sharing @cdeotte
", "rawMarkdown": "That's really great! Thank you for Sharing @cdeotte ", "votes": 1 }, { "id": 671643, "postDate": "2019-11-13T01:29:00.330Z", "content": "Chris! You are a non stop beast! Please continue : ) without burnout. However, in your experiments, are you saying that training resolution didn't matter at all and it's all about prediction resolution? Or are you saying experiment on small res and then once found best structure, train on higher res and predict on higher res?
", "rawMarkdown": "Chris! You are a non stop beast! Please continue : ) without burnout. However, in your experiments, are you saying that training resolution didn't matter at all and it's all about prediction resolution? Or are you saying experiment on small res and then once found best structure, train on higher res and predict on higher res?", "votes": 1 }, { "id": 671597, "postDate": "2019-11-12T23:36:45.140Z", "content": "Wonderful @cdeotte ! Very detail!
", "rawMarkdown": "Wonderful @cdeotte ! Very detail!", "votes": 1 }, { "id": 671788, "postDate": "2019-11-13T06:37:20.050Z", "content": "didn't know about the way of submitting from colab,thanks a lot for wisdom
", "rawMarkdown": "didn't know about the way of submitting from colab,thanks a lot for wisdom", "votes": 2 }, { "id": 820252, "postDate": "2020-04-25T09:25:38.173Z", "rawMarkdown": "", "isDeleted": true }, { "id": 672665, "postDate": "2019-11-14T03:53:52.487Z", "content": "Thanks for sharing information !
", "rawMarkdown": "Thanks for sharing information !", "votes": 1 }, { "id": 671776, "postDate": "2019-11-13T06:29:17.870Z", "content": "thanks chris
", "rawMarkdown": "thanks chris", "votes": 1 } ], "comments": [ { "id": 673049, "author_name": "Rohit Gupta", "author_url": "", "post_date": "2019-11-14T12:56:21.387000", "content": "Also, if anyone wants more ram on colab, just crash the session by exploding the memory usage. It will give you an option to increase the ram from 13gigs to 26gigs.\nUsually, I use\n\nd = []\nwhile 1: d.append('1')\n
Thanks for sharing this.
", "votes": 0, "replies": [] }, { "id": 674733, "author_name": "c-weed", "author_url": "", "post_date": "2019-11-17T01:56:41.387000", "content": "wow!
", "votes": 0, "replies": [] }, { "id": 680247, "author_name": "Rohan Rajpal", "author_url": "", "post_date": "2019-11-24T10:19:28.727000", "content": "A small trick to make it crash faster :p
\n\n\nd = []
\nwhile 1:
\n d.append(d + ['1'])
\n
That's really great! Thank you for your helpfulness!
", "votes": 3, "replies": [] }, { "id": 693114, "author_name": "DiegoJohnson", "author_url": "", "post_date": "2019-12-12T03:35:49.100000", "content": "Good job, I just begin to use colabπ @cdeotte
", "votes": 1, "replies": [] }, { "id": 673723, "author_name": "CBR", "author_url": "", "post_date": "2019-11-15T11:55:05.933000", "content": "π Hakunamatata! π
", "votes": 1, "replies": [] }, { "id": 671724, "author_name": "Saurav Anand", "author_url": "", "post_date": "2019-11-13T04:53:04.010000", "content": "thanks for sharing this @cdeotte .............!!!
", "votes": 1, "replies": [] }, { "id": 671668, "author_name": "Ailurophile", "author_url": "", "post_date": "2019-11-13T02:50:02.250000", "content": "That's really great! Thank you for Sharing @cdeotte
", "votes": 1, "replies": [] }, { "id": 671643, "author_name": "yevg", "author_url": "", "post_date": "2019-11-13T01:29:00.330000", "content": "Chris! You are a non stop beast! Please continue : ) without burnout. However, in your experiments, are you saying that training resolution didn't matter at all and it's all about prediction resolution? Or are you saying experiment on small res and then once found best structure, train on higher res and predict on higher res?
", "votes": 1, "replies": [] }, { "id": 671597, "author_name": "Hieu Phung", "author_url": "", "post_date": "2019-11-12T23:36:45.140000", "content": "Wonderful @cdeotte ! Very detail!
", "votes": 1, "replies": [] }, { "id": 671788, "author_name": "Mobassir", "author_url": "", "post_date": "2019-11-13T06:37:20.050000", "content": "didn't know about the way of submitting from colab,thanks a lot for wisdom
", "votes": 2, "replies": [] }, { "id": 820252, "author_name": "", "author_url": "", "post_date": "2020-04-25T09:25:38.173000", "content": "", "votes": 0, "replies": [] }, { "id": 672665, "author_name": "uttam kumar", "author_url": "", "post_date": "2019-11-14T03:53:52.487000", "content": "Thanks for sharing information !
", "votes": 1, "replies": [] }, { "id": 671776, "author_name": "liuze", "author_url": "", "post_date": "2019-11-13T06:29:17.870000", "content": "thanks chris
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "671587": "Kaggle just added a new Dataset tier to everyone's Kaggle profile. So, I decided to create a dataset [here][2] to help everyone save TIME and MONEY! And maybe some of you will upvote it :P\n \n# How to Save Time\n## Step 1\nTo find the optimal setup (architecture, loss, optimizer, training schedule, etc) requires lots of experiments. Ideas can be tested on smaller cloud images to save time.\n\nI created a Kaggle [dataset][2] with folders containing resized training and test images for sizes `384x576`, `320x480`, `256x384`, `192x288`, `128x192`, and `64x96`. Also each folder has an associated `train_WxH.csv` with resized masks. (I used Hieu's great notebook [here][5] to create the images with `INTERPOLATION = cv2.INTER_AREA`).\n\nUse your existing code and attach this Kaggle dataset to your Kaggle notebook. Then change the name of `train_images` folder to `train_images_320x480` or whatever. And change the name of `train.csv` to `train_320x480.csv`. (And update calls to `rle2mask` to use new dim). That's it, then your experiments will run quicker and you will save TIME.\n\nWhen you're ready to make a submission, you can use switch back to a larger image folder for higher CV and LB score.\n \n## Step 2\nMost of us need to stop training, predict validation masks, and apply post process, to compute an accurate validation Dice score. Instead of doing that, you can include post process directly in your Keras model metric. You will immediately know how your model is performing without having to waste valuable time stopping to check.\n\n def get_kaggle_dice(pix=0.5, dim=(320,480)):\n def kaggle_dice(y_true, y_pred0, pix=pix, dim=dim):\n \n # PIXEL THRESHOLD\n y_pred = K.cast( K.greater(y_pred0,pix), K.floatx() )\n \n # MIN AREA THRESHOLD\n area = 20000.*dim[0]/350.*dim[1]/525.\n s = K.sum(y_pred, axis=(1,2))\n s = K.cast( K.greater(s, area), K.floatx() )\n\n # REMOVE MIN AREA\n s = K.reshape(s,(-1,1))\n s = K.repeat(s,dim[0]*dim[1])\n s = K.reshape(s,(-1,1))\n y_pred = K.permute_dimensions(y_pred,(0,3,1,2))\n y_pred = K.reshape(y_pred,shape=(-1,1))\n y_pred = s*y_pred\n y_pred = K.reshape(y_pred,(-1,y_pred0.shape[3],dim[0],dim[1]))\n y_pred = K.permute_dimensions(y_pred,(0,2,3,1))\n\n # COMPUTE KAGGLE DICE\n intersection = K.sum(y_true * y_pred, axis=(1,2))\n total_y_true = K.sum(y_true, axis=(1,2))\n total_y_pred = K.sum(y_pred, axis=(1,2))\n return K.mean( (2*intersection+1e-9) / (total_y_true+total_y_pred+1e-9) )\n\n return kaggle_dice\n\nThen you include this in your Keras model shown below. Change `(320,480)` to the image size that you are using.\n \n metric = get_kaggle_dice(pix=0.5, dim=(320,480)) \n model.compile(optimizer=opt, loss=loss, metrics=[metric])\n\n\n# How to Save Money \nGoogle Colab is offering free online GPU notebooks. Simply click [here][1] and begin! Then follow the steps below to start training your cloud models\n \n## Step 1\nAfter clicking the link above, add a GPU to your notebook, by selecting \"Runtime\", \"Change runtime type\", \"Hardware accelerator\", \"GPU\"\n\n \nThen choose GPU\n\n\n## Step 2\nTransfer my [\"Cloud Images Resized\" Kaggle dataset][2] to Google Colab as follows. First do the first 6 steps from this Stack Overflow post [here][3]. Afterward type\n\n !kaggle datasets download -d Cloud-Images-Resized\n\nand next execute\n\n import zipfile\n with zipfile.ZipFile('Cloud-Images-Resized.zip', 'r') as zip_ref:\n zip_ref.extractall('.')\n\n \n## Step 3 Train your model\nNext upload your model code and train. My \"Cloud Images Resized\" dataset has all the files you need including `train_images_WxH`, `test_images_WxH`, `train_WxH.csv`, and `sample_submission.csv`. \n\n## Step 4 Submit from Colab OR Download your model\nYou can make a submission to our competition directly from Google Colab by typing\n \n !kaggle competitions understanding_cloud_organization -f submission.csv -m message\n \nOr you can save your model and download to your local machine. The image below shows how. After download, infer the test images locally or upload your model to Kaggle notebook and infer there.\n \n\n \n# New Kaggle Dataset Medal Progression System\nJust last week, Kaggle added a new way to achieve medals at Kaggle. We are now encouraged to create helpful datasets for other Kagglers. More info about this new tier is [here][4]. Enjoy the dataset, and enjoy saving TIME and MONEY!\n\n\n[1]: https://colab.research.google.com/notebook#create=true&language=python3\n[2]: https://www.kaggle.com/cdeotte/cloud-images-resized\n[3]: https://stackoverflow.com/questions/49310470/using-kaggle-datasets-in-google-colab\n[4]: https://www.kaggle.com/product-feedback/116079\n[5]: https://www.kaggle.com/phunghieu/dataset-preparation-resize-images", "673049": "Also, if anyone wants more ram on colab, just crash the session by exploding the memory usage. It will give you an option to increase the ram from 13gigs to 26gigs.\nUsually, I use\n```\nd = []\nwhile 1: d.append('1')\n```", "671634": "That's really great! Thank you for your helpfulness! ", "693114": "Good job, I just begin to use colabπ @cdeotte ", "673723": "π Hakunamatata! π ", "671724": "thanks for sharing this @cdeotte .............!!!", "671668": "That's really great! Thank you for Sharing @cdeotte ", "671643": "Chris! You are a non stop beast! Please continue : ) without burnout. However, in your experiments, are you saying that training resolution didn't matter at all and it's all about prediction resolution? Or are you saying experiment on small res and then once found best structure, train on higher res and predict on higher res?", "671597": "Wonderful @cdeotte ! Very detail!", "671788": "didn't know about the way of submitting from colab,thanks a lot for wisdom", "820252": "", "672665": "Thanks for sharing information !", "671776": "thanks chris" } }