{ "id": 100444, "title": "I am having trouble chaining kernels, one for preprocessing and next modeling", "url": "/competitions/aptos2019-blindness-detection/discussion/100444", "author_name": "", "post_date": "2019-07-18T15:44:57.508061800Z", "votes": null, "comment_count": 4, "views": 0, "content": "

I tried to do the preprocessing as in the winning solution and shared here in kernel , My idea is to create preprocessed and resized images for training and validation zip files . But i can't load that data files.\"\"\n\"\"\n\"\"\nI hope this can be done as we can load the output of kernels into other kernels. \nThank you.

", "messages": [ { "id": "579196", "postDate": "07/18/2019 15:44:57", "content": "

I tried to do the preprocessing as in the winning solution and shared here in kernel , My idea is to create preprocessed and resized images for training and validation zip files . But i can't load that data files.\"\"\n\"\"\n\"\"\nI hope this can be done as we can load the output of kernels into other kernels. \nThank you.

", "rawMarkdown": "I tried to do the preprocessing as in the winning solution and shared here in [kernel](https://www.kaggle.com/ratthachat/aptos-updated-preprocessing-ben-s-cropping) , My idea is to create preprocessed and resized images for training and validation zip files . But i can't load that data files.![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F455315%2F30a32457d7e646ba930e5ac76cb3d679%2Fdetect1.PNG?generation=1563464221145914&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F455315%2F52cc424a0dd2c1eb7098d00a0ed02628%2Fdetect2.PNG?generation=1563464251102084&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F455315%2Fa4c83dd6d22056a0e7bdbf272a053ddc%2Fdetect3.PNG?generation=1563464281285123&alt=media)\nI hope this can be done as we can load the output of kernels into other kernels. \nThank you.", "votes": null }, { "id": "579412", "postDate": "07/18/2019 20:07:22", "content": "

There is a limit to how many files a kernel can output(500 i think).\nBut you shouldn't do this, do preprocessing in the dataloader(not before running the script)

", "rawMarkdown": "There is a limit to how many files a kernel can output(500 i think).\nBut you shouldn't do this, do preprocessing in the dataloader(not before running the script)", "votes": null }, { "id": "579513", "postDate": "07/18/2019 22:56:27", "content": "

Hi the disk output limit is 5GB so either you have to use small picture size or do output in several kernels

", "rawMarkdown": "Hi the disk output limit is 5GB so either you have to use small picture size or do output in several kernels", "votes": null }, { "id": "580048", "postDate": "07/19/2019 15:32:45", "content": "

Oh, I didn't think about it. Thank you.

", "rawMarkdown": "Oh, I didn't think about it. Thank you.", "votes": null }, { "id": "580050", "postDate": "07/19/2019 15:33:36", "content": "

That's why I made it a zip file but Is there any restrictions on doing this?

", "rawMarkdown": "That's why I made it a zip file but Is there any restrictions on doing this?", "votes": null } ], "comments": [ { "id": 579412, "author_name": "sidhanthholalkere", "author_url": "", "post_date": "07/18/2019 20:07:22", "content": "

There is a limit to how many files a kernel can output(500 i think).\nBut you shouldn't do this, do preprocessing in the dataloader(not before running the script)

", "votes": null, "replies": [ { "id": 580050, "author_name": "abh001", "author_url": "", "post_date": "07/19/2019 15:33:36", "content": "

That's why I made it a zip file but Is there any restrictions on doing this?

", "votes": null, "replies": [] } ] }, { "id": 579513, "author_name": "ratthachat", "author_url": "", "post_date": "07/18/2019 22:56:27", "content": "

Hi the disk output limit is 5GB so either you have to use small picture size or do output in several kernels

", "votes": null, "replies": [ { "id": 580048, "author_name": "abh001", "author_url": "", "post_date": "07/19/2019 15:32:45", "content": "

Oh, I didn't think about it. Thank you.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "579196": "I tried to do the preprocessing as in the winning solution and shared here in [kernel](https://www.kaggle.com/ratthachat/aptos-updated-preprocessing-ben-s-cropping) , My idea is to create preprocessed and resized images for training and validation zip files . But i can't load that data files.![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F455315%2F30a32457d7e646ba930e5ac76cb3d679%2Fdetect1.PNG?generation=1563464221145914&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F455315%2F52cc424a0dd2c1eb7098d00a0ed02628%2Fdetect2.PNG?generation=1563464251102084&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F455315%2Fa4c83dd6d22056a0e7bdbf272a053ddc%2Fdetect3.PNG?generation=1563464281285123&alt=media)\nI hope this can be done as we can load the output of kernels into other kernels. \nThank you.", "579412": "There is a limit to how many files a kernel can output(500 i think).\nBut you shouldn't do this, do preprocessing in the dataloader(not before running the script)", "579513": "Hi the disk output limit is 5GB so either you have to use small picture size or do output in several kernels", "580048": "Oh, I didn't think about it. Thank you.", "580050": "That's why I made it a zip file but Is there any restrictions on doing this?" }, "source": "meta" }