Hello guys,\n This is my first Deep Learning competition. I am spending significant amount of time on this competition but getting no where. The kernels take lot of time to run and if my LB isn't improving certainly I get discouraged. I want some one to please guide me through this competition. I am willing to learn a lot and give my all to this competition. Thank you.
", "messages": [ { "id": "589112", "postDate": "07/31/2019 13:29:38", "content": "Hello guys,\n This is my first Deep Learning competition. I am spending significant amount of time on this competition but getting no where. The kernels take lot of time to run and if my LB isn't improving certainly I get discouraged. I want some one to please guide me through this competition. I am willing to learn a lot and give my all to this competition. Thank you.
", "rawMarkdown": "Hello guys,\n This is my first Deep Learning competition. I am spending significant amount of time on this competition but getting no where. The kernels take lot of time to run and if my LB isn't improving certainly I get discouraged. I want some one to please guide me through this competition. I am willing to learn a lot and give my all to this competition. Thank you.", "votes": null }, { "id": "589125", "postDate": "07/31/2019 13:47:23", "content": "kernels take a lot of time: \nHave you turned on GPU button? What image size are you using, which model, augmentation, etc etc depends on lot of things. Start with minimal configuration and then build on top of that.
LB isn't improving: \nGo through all the public kernels and discussion threads of this competition, they have more than enough information to get you to top 100.
Thank you for your reply. Ya I have turned on the GPU button. And ya I went through many kernels. I try to replicate what they have done on my own and the kernel runs fine locally. But when I submit, it runs like for 1 hour and then some error. That discourages me.
", "rawMarkdown": "Thank you for your reply. Ya I have turned on the GPU button. And ya I went through many kernels. I try to replicate what they have done on my own and the kernel runs fine locally. But when I submit, it runs like for 1 hour and then some error. That discourages me.", "votes": null }, { "id": "589236", "postDate": "07/31/2019 16:54:15", "content": "Are you loading all images in memory before predicting? \nDon't load all the images in memory while predicting, read and predict batch by batch.
", "rawMarkdown": "Are you loading all images in memory before predicting? \nDon't load all the images in memory while predicting, read and predict batch by batch.", "votes": null }, { "id": "589333", "postDate": "07/31/2019 18:59:31", "content": "I would suggest fork any kernel as a starting point run it,make sure it runs perfectly ,dig into the details of the code and then make changes accordingly .Just out of curiosity what library are u using?
", "rawMarkdown": "I would suggest fork any kernel as a starting point run it,make sure it runs perfectly ,dig into the details of the code and then make changes accordingly .Just out of curiosity what library are u using?", "votes": null }, { "id": "589474", "postDate": "08/01/2019 00:04:27", "content": "When you run the kernel yourself, it runs only on the public data set. When you commit, it runs on both public & private datasets. As Risabh said, if you are loading all images into the memory, you can hit OOM (out of memory) in the commit.
\n\nYou can simulate the same issue in your own kernel run. In your kernel, replicate the input data 10 times. And run the kernel yourself. It will mostly fail at the same point as the commit run and you will be able to see the error and debug it.
\n\nIf loading all images into the memory happens to be the root cause, change to batch by batch processing.
", "rawMarkdown": "When you run the kernel yourself, it runs only on the public data set. When you commit, it runs on both public & private datasets. As Risabh said, if you are loading all images into the memory, you can hit OOM (out of memory) in the commit.\n\nYou can simulate the same issue in your own kernel run. In your kernel, replicate the input data 10 times. And run the kernel yourself. It will mostly fail at the same point as the commit run and you will be able to see the error and debug it.\n\nIf loading all images into the memory happens to be the root cause, change to batch by batch processing.", "votes": null }, { "id": "589551", "postDate": "08/01/2019 04:47:12", "content": "Do u mean framework ? I am using Keras. And yes, I had forked some one else's kernel and tried to change some things. Thank you for your concern .
", "rawMarkdown": "Do u mean framework ? I am using Keras. And yes, I had forked some one else's kernel and tried to change some things. Thank you for your concern .", "votes": null }, { "id": "589552", "postDate": "08/01/2019 04:48:54", "content": "Oh wow, I never thought of this. Thank you. I think loading all the test images in memory is the root cause of submission error. I'll update my code. Thanks once again both of you.
", "rawMarkdown": "Oh wow, I never thought of this. Thank you. I think loading all the test images in memory is the root cause of submission error. I'll update my code. Thanks once again both of you.", "votes": null } ], "comments": [ { "id": 589125, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "07/31/2019 13:47:23", "content": "kernels take a lot of time: \nHave you turned on GPU button? What image size are you using, which model, augmentation, etc etc depends on lot of things. Start with minimal configuration and then build on top of that.
LB isn't improving: \nGo through all the public kernels and discussion threads of this competition, they have more than enough information to get you to top 100.
Thank you for your reply. Ya I have turned on the GPU button. And ya I went through many kernels. I try to replicate what they have done on my own and the kernel runs fine locally. But when I submit, it runs like for 1 hour and then some error. That discourages me.
", "votes": null, "replies": [] }, { "id": 589236, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "07/31/2019 16:54:15", "content": "Are you loading all images in memory before predicting? \nDon't load all the images in memory while predicting, read and predict batch by batch.
", "votes": null, "replies": [] }, { "id": 589474, "author_name": "ravivadapalli", "author_url": "", "post_date": "08/01/2019 00:04:27", "content": "When you run the kernel yourself, it runs only on the public data set. When you commit, it runs on both public & private datasets. As Risabh said, if you are loading all images into the memory, you can hit OOM (out of memory) in the commit.
\n\nYou can simulate the same issue in your own kernel run. In your kernel, replicate the input data 10 times. And run the kernel yourself. It will mostly fail at the same point as the commit run and you will be able to see the error and debug it.
\n\nIf loading all images into the memory happens to be the root cause, change to batch by batch processing.
", "votes": null, "replies": [] }, { "id": 589552, "author_name": "virajbagal", "author_url": "", "post_date": "08/01/2019 04:48:54", "content": "Oh wow, I never thought of this. Thank you. I think loading all the test images in memory is the root cause of submission error. I'll update my code. Thanks once again both of you.
", "votes": null, "replies": [] } ] }, { "id": 589333, "author_name": "amardeepganguly", "author_url": "", "post_date": "07/31/2019 18:59:31", "content": "I would suggest fork any kernel as a starting point run it,make sure it runs perfectly ,dig into the details of the code and then make changes accordingly .Just out of curiosity what library are u using?
", "votes": null, "replies": [ { "id": 589551, "author_name": "virajbagal", "author_url": "", "post_date": "08/01/2019 04:47:12", "content": "Do u mean framework ? I am using Keras. And yes, I had forked some one else's kernel and tried to change some things. Thank you for your concern .
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "589112": "Hello guys,\n This is my first Deep Learning competition. I am spending significant amount of time on this competition but getting no where. The kernels take lot of time to run and if my LB isn't improving certainly I get discouraged. I want some one to please guide me through this competition. I am willing to learn a lot and give my all to this competition. Thank you.", "589125": "1. kernels take a lot of time: \nHave you turned on GPU button? What image size are you using, which model, augmentation, etc etc depends on lot of things. Start with minimal configuration and then build on top of that. \n\n2. LB isn't improving: \nGo through all the public kernels and discussion threads of this competition, they have more than enough information to get you to top 100.", "589139": "Thank you for your reply. Ya I have turned on the GPU button. And ya I went through many kernels. I try to replicate what they have done on my own and the kernel runs fine locally. But when I submit, it runs like for 1 hour and then some error. That discourages me.", "589236": "Are you loading all images in memory before predicting? \nDon't load all the images in memory while predicting, read and predict batch by batch.", "589333": "I would suggest fork any kernel as a starting point run it,make sure it runs perfectly ,dig into the details of the code and then make changes accordingly .Just out of curiosity what library are u using?", "589474": "When you run the kernel yourself, it runs only on the public data set. When you commit, it runs on both public & private datasets. As Risabh said, if you are loading all images into the memory, you can hit OOM (out of memory) in the commit.\n\nYou can simulate the same issue in your own kernel run. In your kernel, replicate the input data 10 times. And run the kernel yourself. It will mostly fail at the same point as the commit run and you will be able to see the error and debug it.\n\nIf loading all images into the memory happens to be the root cause, change to batch by batch processing.", "589551": "Do u mean framework ? I am using Keras. And yes, I had forked some one else's kernel and tried to change some things. Thank you for your concern .", "589552": "Oh wow, I never thought of this. Thank you. I think loading all the test images in memory is the root cause of submission error. I'll update my code. Thanks once again both of you." }, "source": "meta" }