![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1133510%2F21f2e5ff974a3d58b326c18d73d98811%2F2019-07-07%209.14.59.png?generation=1562505454092595&alt=media\")
A lot of people seem to be having problems with submitting in this competition, so I'm creating this guide.
\n\nThis is Synchronous Kernels Only competition where your kernel will run against an unseen dataset in addition to the visible test set!
\n\nFirst of all, you need to train the model. In this kernel here: https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59 I have shown how to train a simple high scoring model using pytorch.
\n\nIn the \"Get the model\" section, you can find the model definition which is a simple ResNet101.
\n\n\nmodel = torchvision.models.resnet101(pretrained=False)\nmodel.load_state_dict(torch.load(\"../input/pytorch-pretrained-models/resnet101-5d3b4d8f.pth\"))\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\n
Since pretrained weights are allowed, we will use pre-trained imagenet weights as the starting point.
\n\nOnce the model is trained and you are happy with the loss/accuracy, save the model weights.
\n\nIf you are using keras, you can do:
\n\n\nfrom keras.models import load_model\nmodel.save('model.h5')\n
If you are using pytorch, look at the kernel mentioned above. It saves a \"model.bin\":
\n\n\ntorch.save(model.state_dict(), \"model.bin\")\n
Now remember 2 things. The model and the preprocessing! For example, if you resized training images to 128x128, you need to do the same for test images (most of the time).
\n\nSo, once you have the model saved, click on datasets on the top of the page, and save you model in kaggle datasets.
\n\nThis step does not need to run on kaggle kernels, ofcourse you can use it but if you prefer your own beefy machine, feel free to do so.
\n\nNow moving to the next and final step.
\n\nDuring inference, you will make your predictions on the unseen test images. Unseen? YES! We have two sets of test images, one that you can download and other one you cannot. The inference step must run in a Kaggle Kernel.
\n\nAfter you start the kernel, add the dataset you uploaded in the previous step.
\n\nWhat's different when your kernel runs on unseen test images:
\n\nKeep in mind that we dont know how many test images are there in the second stage.
\n\nYou can base your inference kernel on: https://www.kaggle.com/abhishek/pytorch-inference-kernel-lazy-tta
\n\nNow you need the same model again!
\n\npytorch:
\n\n\nmodel = torchvision.models.resnet101(pretrained=False)\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\nmodel.load_state_dict(torch.load(\"../input/model/model.bin\"))\n
in keras, you can do:
\n\n\nfrom keras.models import load_model\nmodel = load_model('../model/model.h5')\n
Once you have the model in place, start generating predictions for test images using either sample_submission.csv or test.csv. You need to use the same pre-processing as you did for training.!
\n\nYou need to take care of the following things:\n- kernels dont have a lot of memory, use small images\n- use small batches\n- do not hardcode number of test images! you dont know how many are there!\n- unseen test images are approximately 6 times the test images that you see.
\n\nStart with very simple inference kernel with 16 batch size and 128x128 images. Once that works, improve it!
\n\nOnce the kernel is working fine, commit it to run it from top to bottom. When the commit is ready, select the submission.csv in output tab and submit to the competition and wait.
\n\nNOTE: If you make a mistake and kernel fails, you will lose a submission. So, submit only when you are sure that you are not using anything that will break!!
\n\nIn case of any questions, ask here and ill try my best to answer :)
", "messages": [ { "id": 566062, "postDate": "2019-07-01T18:56:01.793Z", "content": "A lot of people seem to be having problems with submitting in this competition, so I'm creating this guide.
\n\nThis is Synchronous Kernels Only competition where your kernel will run against an unseen dataset in addition to the visible test set!
\n\nFirst of all, you need to train the model. In this kernel here: https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59 I have shown how to train a simple high scoring model using pytorch.
\n\nIn the \"Get the model\" section, you can find the model definition which is a simple ResNet101.
\n\n\nmodel = torchvision.models.resnet101(pretrained=False)\nmodel.load_state_dict(torch.load(\"../input/pytorch-pretrained-models/resnet101-5d3b4d8f.pth\"))\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\n
Since pretrained weights are allowed, we will use pre-trained imagenet weights as the starting point.
\n\nOnce the model is trained and you are happy with the loss/accuracy, save the model weights.
\n\nIf you are using keras, you can do:
\n\n\nfrom keras.models import load_model\nmodel.save('model.h5')\n
If you are using pytorch, look at the kernel mentioned above. It saves a \"model.bin\":
\n\n\ntorch.save(model.state_dict(), \"model.bin\")\n
Now remember 2 things. The model and the preprocessing! For example, if you resized training images to 128x128, you need to do the same for test images (most of the time).
\n\nSo, once you have the model saved, click on datasets on the top of the page, and save you model in kaggle datasets.
\n\nThis step does not need to run on kaggle kernels, ofcourse you can use it but if you prefer your own beefy machine, feel free to do so.
\n\nNow moving to the next and final step.
\n\nDuring inference, you will make your predictions on the unseen test images. Unseen? YES! We have two sets of test images, one that you can download and other one you cannot. The inference step must run in a Kaggle Kernel.
\n\nAfter you start the kernel, add the dataset you uploaded in the previous step.
\n\nWhat's different when your kernel runs on unseen test images:
\n\nKeep in mind that we dont know how many test images are there in the second stage.
\n\nYou can base your inference kernel on: https://www.kaggle.com/abhishek/pytorch-inference-kernel-lazy-tta
\n\nNow you need the same model again!
\n\npytorch:
\n\n\nmodel = torchvision.models.resnet101(pretrained=False)\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\nmodel.load_state_dict(torch.load(\"../input/model/model.bin\"))\n
in keras, you can do:
\n\n\nfrom keras.models import load_model\nmodel = load_model('../model/model.h5')\n
Once you have the model in place, start generating predictions for test images using either sample_submission.csv or test.csv. You need to use the same pre-processing as you did for training.!
\n\nYou need to take care of the following things:\n- kernels dont have a lot of memory, use small images\n- use small batches\n- do not hardcode number of test images! you dont know how many are there!\n- unseen test images are approximately 6 times the test images that you see.
\n\nStart with very simple inference kernel with 16 batch size and 128x128 images. Once that works, improve it!
\n\nOnce the kernel is working fine, commit it to run it from top to bottom. When the commit is ready, select the submission.csv in output tab and submit to the competition and wait.
\n\nNOTE: If you make a mistake and kernel fails, you will lose a submission. So, submit only when you are sure that you are not using anything that will break!!
\n\nIn case of any questions, ask here and ill try my best to answer :)
", "rawMarkdown": "A lot of people seem to be having problems with submitting in this competition, so I'm creating this guide.\n\nThis is Synchronous Kernels Only competition where your kernel will run against an unseen dataset in addition to the visible test set!\n\n\n## The training phase\n\nFirst of all, you need to train the model. In this kernel here: https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59 I have shown how to train a simple high scoring model using pytorch.\n\nIn the \"Get the model\" section, you can find the model definition which is a simple ResNet101.\n\n\n```\nmodel = torchvision.models.resnet101(pretrained=False)\nmodel.load_state_dict(torch.load(\"../input/pytorch-pretrained-models/resnet101-5d3b4d8f.pth\"))\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\n```\n\n\nSince pretrained weights are allowed, we will use pre-trained imagenet weights as the starting point.\n\nOnce the model is trained and you are happy with the loss/accuracy, save the model weights.\n\nIf you are using keras, you can do:\n\n```\nfrom keras.models import load_model\nmodel.save('model.h5')\n```\n\nIf you are using pytorch, look at the kernel mentioned above. It saves a \"model.bin\":\n\n\n```\ntorch.save(model.state_dict(), \"model.bin\")\n```\n\nNow remember 2 things. The model and the preprocessing! For example, if you resized training images to 128x128, you need to do the same for test images (most of the time).\n\nSo, once you have the model saved, click on datasets on the top of the page, and save you model in kaggle datasets.\n\n\nThis step does not need to run on kaggle kernels, ofcourse you can use it but if you prefer your own beefy machine, feel free to do so.\n\n\nNow moving to the next and final step.\n\n\n## The inference phase\n\nDuring inference, you will make your predictions on the unseen test images. Unseen? YES! We have two sets of test images, one that you can download and other one you cannot. The inference step must run in a Kaggle Kernel.\n\nAfter you start the kernel, add the dataset you uploaded in the previous step.\n\nWhat's different when your kernel runs on unseen test images:\n\n- sample_submission.csv\n- test.csv\n- test.zip\n\nKeep in mind that we dont know how many test images are there in the second stage. \n\n\nYou can base your inference kernel on: https://www.kaggle.com/abhishek/pytorch-inference-kernel-lazy-tta\n\nNow you need the same model again!\n\npytorch:\n\n```\nmodel = torchvision.models.resnet101(pretrained=False)\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\nmodel.load_state_dict(torch.load(\"../input/model/model.bin\"))\n```\n\nin keras, you can do:\n\n```\nfrom keras.models import load_model\nmodel = load_model('../model/model.h5')\n```\n\nOnce you have the model in place, start generating predictions for test images using either sample_submission.csv or test.csv. You need to use the same pre-processing as you did for training.!\n\n\nYou need to take care of the following things:\n- kernels dont have a lot of memory, use small images\n- use small batches\n- do not hardcode number of test images! you dont know how many are there!\n- unseen test images are approximately 6 times the test images that you see.\n\nStart with very simple inference kernel with 16 batch size and 128x128 images. Once that works, improve it!\n\nOnce the kernel is working fine, commit it to run it from top to bottom. When the commit is ready, select the submission.csv in output tab and submit to the competition and wait.\n\n\n*NOTE: If you make a mistake and kernel fails, you will lose a submission. So, submit only when you are sure that you are not using anything that will break!!*\n\n---\n\n\nIn case of any questions, ask here and ill try my best to answer :)\n\n\n\n\n\n\n\n", "votes": 82 }, { "id": 569857, "postDate": "2019-07-07T13:07:37.357Z", "content": "I copy paste my post from my discussion just in case it can helps !
\n\nHello everyone, after few days of research, I finaly found the problem. It was not about memory, running time or whatever. The problem was nothing else than my preprocessing function. It was croping black border to reduce the noise and maximise the signal. The preprocessing function was working perfectly on public test set and training set. However, I truely believe that the private test set got fully dark / black images. Resulting in a full croping (image[0:0] or something like that). Therefore, the predict fail and to_csv also.
\n\nHope it will help ! GL
\n\nVincent
", "rawMarkdown": "I copy paste my post from my discussion just in case it can helps ! \n\nHello everyone, after few days of research, I finaly found the problem. It was not about memory, running time or whatever. The problem was nothing else than my preprocessing function. It was croping black border to reduce the noise and maximise the signal. The preprocessing function was working perfectly on public test set and training set. However, I truely believe that the private test set got fully dark / black images. Resulting in a full croping (image[0:0] or something like that). Therefore, the predict fail and to_csv also.\n\nHope it will help ! GL\n\nVincent", "votes": 5 }, { "id": 567133, "postDate": "2019-07-03T05:12:20.960Z", "content": "This is very helpful @abhishek . I face a little different problem with my kernel. During training, the weighted kappa score of the model is above ~0.7 but after I run predictions to submit the results, I get a leaderboard score of 0. Although my model is a little biased towards class 0 and 2, the score on leaderboard is still 0. Do you have any idea with what potentially could be wrong. I am using pytorch with fastai wrapper.
", "rawMarkdown": "This is very helpful @abhishek . I face a little different problem with my kernel. During training, the weighted kappa score of the model is above ~0.7 but after I run predictions to submit the results, I get a leaderboard score of 0. Although my model is a little biased towards class 0 and 2, the score on leaderboard is still 0. Do you have any idea with what potentially could be wrong. I am using _pytorch_ with _fastai_ wrapper.", "votes": 6, "replies": [ { "id": 567162, "postDate": "2019-07-03T06:05:03.903Z", "content": "This happened to me as well. I managed to fix it by creating a new notebook with the exact same code and submitting it. I'm not exactly sure why it didn't work with my original notebook though.
", "rawMarkdown": "This happened to me as well. I managed to fix it by creating a new notebook with the exact same code and submitting it. I'm not exactly sure why it didn't work with my original notebook though. ", "votes": 1 }, { "id": 567643, "postDate": "2019-07-03T19:24:37.933Z", "content": "Thanks @frefger, it did worked.
", "rawMarkdown": "Thanks @frefger, it did worked. ", "votes": 2 }, { "id": 569808, "postDate": "2019-07-07T11:41:42.597Z", "content": "I was also having a 0 score each time, but what has solved that was only to decrease the batchsize (in my case to 1). Was certainly a memory issue, but still hard to figure out without logs or even an error message. Thanks for this posts and comment that helped to solve !
", "rawMarkdown": "I was also having a 0 score each time, but what has solved that was only to decrease the batchsize (in my case to 1). Was certainly a memory issue, but still hard to figure out without logs or even an error message. Thanks for this posts and comment that helped to solve !" } ] }, { "id": 567110, "postDate": "2019-07-03T04:44:19.490Z", "content": "\"your own beefy machine\" --> my beefy machine is Kaggle Kernels 😅
", "rawMarkdown": "\"your own beefy machine\" --> my beefy machine is Kaggle Kernels 😅 ", "votes": 4, "replies": [ { "id": 568507, "postDate": "2019-07-05T03:15:11.683Z", "content": "Kaggle kernels and google colab are very nice resources!\nThankfully its a synchronous KO competition, which allows you to train one model over multiple kernels so theoretically you could train your model for tens of hours in total.
\n\nTo do this, just download your model after running a kernel for the given 9 hours, then upload it as a custom dataset, import that dataset and continue training it for another 9 hours. Rinse and repeat until you are satisfied :)
", "rawMarkdown": "Kaggle kernels and google colab are very nice resources!\nThankfully its a synchronous KO competition, which allows you to train one model over multiple kernels so theoretically you could train your model for tens of hours in total.\n\nTo do this, just download your model after running a kernel for the given 9 hours, then upload it as a custom dataset, import that dataset and continue training it for another 9 hours. Rinse and repeat until you are satisfied :)" } ] }, { "id": 570293, "postDate": "2019-07-08T05:18:01.283Z", "content": "Hi everyone, maybe you can help me, I've trained a model using this amazing kernel https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59
\n\nand when I try to load it:\n\nmodel.load_state_dict(torch.load(\"../input/aptos-pretrained-models/model.bin\"))\n\nI obtain this error:
\nError(s) in loading state_dict for ResNet:\n Missing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_li\n
I'm just trying the kernels from @abhishek , so I think this is not about the code :(
", "rawMarkdown": "Hi everyone, maybe you can help me, I've trained a model using this amazing kernel https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59\n\nand when I try to load it:\n``` \nmodel.load_state_dict(torch.load(\"../input/aptos-pretrained-models/model.bin\"))\n```\nI obtain this error:\n\n```\nError(s) in loading state_dict for ResNet:\n\tMissing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_li\n```\n\nI'm just trying the kernels from @abhishek , so I think this is not about the code :(", "votes": 1, "replies": [ { "id": 570299, "postDate": "2019-07-08T05:27:22.913Z", "content": "load_state_dict(state_dict, strict=False) ?
", "rawMarkdown": "load_state_dict(state_dict, strict=False) ?" }, { "id": 570303, "postDate": "2019-07-08T05:30:09.380Z", "content": "yeah, I tried but then the model doesn't work properly, I'll check it again ...\nI don't understand why abhishek doesn't have that error hahaha and I do using his own kernel\nanyway, thanks @jionie !
", "rawMarkdown": "yeah, I tried but then the model doesn't work properly, I'll check it again ...\nI don't understand why abhishek doesn't have that error hahaha and I do using his own kernel\nanyway, thanks @jionie !" }, { "id": 570305, "postDate": "2019-07-08T05:32:46.767Z", "content": "Emmmm, interesting, I always open the internet and set pretrain=True😂
", "rawMarkdown": "Emmmm, interesting, I always open the internet and set pretrain=True😂 " }, { "id": 570310, "postDate": "2019-07-08T05:41:54.673Z", "content": "@jesucristo To solve this kind of problem print the model that was trained and print the model before loading the weights.
\n\nIn this particular case, my guess is that you trained a model using torchvision which has fully connected layer named fc but are trying to load the model using pretrainedmodels where the fc layer is named last_linear . Changing the name of last layer to last_linear will work in this case. Let me know if that fixes your problem. :)
... solved ;) anoher thing I've learnt from you today.
", "rawMarkdown": "... solved ;) anoher thing I've learnt from you today.", "votes": 1 }, { "id": 608732, "postDate": "2019-08-27T05:46:21.650Z", "content": "Hi @jesucristo, I am trying to rename the last layer to last_linear, but i am not able to figure it out how. it would be helpful for me to know how to rename.My code here model.last_linear = nn.Sequential(\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.25),\n nn.Linear(in_features=2048, out_features=2048, bias=True),\n nn.ReLU(),\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.5),\n nn.Linear(in_features=2048, out_features=1, bias=True),\n )\n
Hello @abhishek, how do you infer that the unseen test set is 6 times greater than the public test set. If this confirms, I will be in trouble to get a final score.
\n\nThank you.\nJesús
", "rawMarkdown": "Hello @abhishek, how do you infer that the unseen test set is 6 times greater than the public test set. If this confirms, I will be in trouble to get a final score.\n\nThank you.\nJesús", "replies": [ { "id": 618814, "postDate": "2019-09-05T14:27:15.497Z", "content": "I'm sorry, I already saw in the notes provided.
\n\nThank you.
", "rawMarkdown": "I'm sorry, I already saw in the notes provided.\n\nThank you." } ] }, { "id": 579945, "postDate": "2019-07-19T12:28:11.310Z", "content": "Hello everyone, could someone tell me why my code is giving a error. The link to the kernel is \nHere\nAlso Im using fast ai package
", "rawMarkdown": "Hello everyone, could someone tell me why my code is giving a error. The link to the kernel is \n[Here](https://www.kaggle.com/saigautam/submition-ensembling-baseline)\nAlso Im using fast ai package", "replies": [ { "id": 580224, "postDate": "2019-07-19T21:25:31.957Z", "content": "Make sure your outputs are int type, that was the issue with my submission float type is not supported. See your submission file has float, just do something like astype(np.int32).
Wow it worked, thanks for pointing it out
", "rawMarkdown": "Wow it worked, thanks for pointing it out" } ] }, { "id": 578763, "postDate": "2019-07-18T05:50:46.367Z", "content": "Thanks for this helpful post! I am confused about the memory issue part. If we run the notebook manually, say with a image size 120 - batch size 32 and see that it works and doesn't exceed the memory limit, then why might it fail during submission? Might gpu specs be different? Thanks :)
", "rawMarkdown": "Thanks for this helpful post! I am confused about the memory issue part. If we run the notebook manually, say with a image size 120 - batch size 32 and see that it works and doesn't exceed the memory limit, then why might it fail during submission? Might gpu specs be different? Thanks :)" }, { "id": 574847, "postDate": "2019-07-14T14:57:00.617Z", "content": "Hi and thanks for the post!
\n\nI have never used Kaggle kernels before - and my question is very simple: \nI have augmented images in a custom way and I tried to upload the images - but it was seen as using an external data set. Hence I augmented the images within the kernel, which lead to a rather heavy image list. Still works, but what I would like to know is: Can I save the images on the disk and retrieve them with a data generator? Which reduce the runtime of the script immensly. Thanks for your answer,
\n\nbest regards,
\n\nWolfgang
", "rawMarkdown": "Hi and thanks for the post!\n\nI have never used Kaggle kernels before - and my question is very simple: \nI have augmented images in a custom way and I tried to upload the images - but it was seen as using an external data set. Hence I augmented the images within the kernel, which lead to a rather heavy image list. Still works, but what I would like to know is: Can I save the images on the disk and retrieve them with a data generator? Which reduce the runtime of the script immensly. Thanks for your answer, \n\nbest regards, \n\nWolfgang" }, { "id": 570988, "postDate": "2019-07-09T02:27:04.107Z", "content": "Big help!
", "rawMarkdown": "Big help!" }, { "id": 569889, "postDate": "2019-07-07T14:10:10.790Z", "content": "
Hi, Anyone knows why? I'm sure my submission file is in the right format and right row number. Committing a kernel almost cost 110 seconds, but when I click \"submit to competition\" button, it seems never got finished. I'm using GPU kernel and upload a well-trained model as a dataset, the model is used for inference, the prediction on the test dataset is expected to be finished within 30 mins. I checked my results and read the fine print and data description carefully, but still, don't know where is the problem. Any advice is appreciated!
", "rawMarkdown": "\n\nHi, Anyone knows why? I'm sure my submission file is in the right format and right row number. Committing a kernel almost cost 110 seconds, but when I click \"submit to competition\" button, it seems never got finished. I'm using GPU kernel and upload a well-trained model as a dataset, the model is used for inference, the prediction on the test dataset is expected to be finished within 30 mins. I checked my results and read the fine print and data description carefully, but still, don't know where is the problem. Any advice is appreciated!", "replies": [ { "id": 570723, "postDate": "2019-07-08T17:46:06.387Z", "content": "Apart from the public test set, there is also a much bigger private test set
", "rawMarkdown": "Apart from the public test set, there is also a much bigger private test set" } ] }, { "id": 569101, "postDate": "2019-07-06T04:06:58.837Z", "content": "Thank you @abhishek . you're one person whom i constantly admire about.
", "rawMarkdown": "Thank you @abhishek . you're one person whom i constantly admire about." }, { "id": 569073, "postDate": "2019-07-06T02:18:49.827Z", "content": "thx~
", "rawMarkdown": "thx~" }, { "id": 568558, "postDate": "2019-07-05T05:26:01.197Z", "content": "Thanks for making this kernel @abhishek! I am confused about the unseen test set. How can we ensure that the unseen test data goes through the same steps (e.g. preprocessing for image input, scaling and transformation to ordinal values for output)?
", "rawMarkdown": "Thanks for making this kernel @abhishek! I am confused about the unseen test set. How can we ensure that the unseen test data goes through the same steps (e.g. preprocessing for image input, scaling and transformation to ordinal values for output)?", "replies": [ { "id": 568673, "postDate": "2019-07-05T09:34:38.827Z", "content": "you have to handle all kinds of necessary steps in the inference kernel.
", "rawMarkdown": "you have to handle all kinds of necessary steps in the inference kernel." } ] }, { "id": 567585, "postDate": "2019-07-03T18:19:24.263Z", "content": "Thank you @abhishek , I appreciate that
", "rawMarkdown": "Thank you @abhishek , I appreciate that" }, { "id": 568818, "postDate": "2019-07-05T13:28:48.443Z", "rawMarkdown": "", "isDeleted": true }, { "id": 567157, "postDate": "2019-07-03T05:50:40.157Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 568656, "postDate": "2019-07-05T09:14:03.937Z", "content": "Quite helpful thanks for sharing...
", "rawMarkdown": "Quite helpful thanks for sharing...", "votes": 1 }, { "id": 566370, "postDate": "2019-07-02T05:23:33.317Z", "content": "Thank you so much! This is helpful!
", "rawMarkdown": "Thank you so much! This is helpful! ", "votes": 1 }, { "id": 980263, "postDate": "2020-08-21T12:51:04.033Z", "content": "Quite helpful ,Thank you sir…
", "rawMarkdown": "Quite helpful ,Thank you sir..." }, { "id": 576746, "postDate": "2019-07-16T00:48:38.370Z", "content": "great information ,thanks your sharing
", "rawMarkdown": "great information ,thanks your sharing" }, { "id": 567825, "postDate": "2019-07-04T03:43:32.150Z", "content": "Thanks for you sharing, very helpful.
", "rawMarkdown": "Thanks for you sharing, very helpful." } ], "comments": [ { "id": 569857, "author_name": "Vincoux", "author_url": "", "post_date": "2019-07-07T13:07:37.357000", "content": "I copy paste my post from my discussion just in case it can helps !
\n\nHello everyone, after few days of research, I finaly found the problem. It was not about memory, running time or whatever. The problem was nothing else than my preprocessing function. It was croping black border to reduce the noise and maximise the signal. The preprocessing function was working perfectly on public test set and training set. However, I truely believe that the private test set got fully dark / black images. Resulting in a full croping (image[0:0] or something like that). Therefore, the predict fail and to_csv also.
\n\nHope it will help ! GL
\n\nVincent
", "votes": 5, "replies": [] }, { "id": 567133, "author_name": "Tushar", "author_url": "", "post_date": "2019-07-03T05:12:20.960000", "content": "This is very helpful @abhishek . I face a little different problem with my kernel. During training, the weighted kappa score of the model is above ~0.7 but after I run predictions to submit the results, I get a leaderboard score of 0. Although my model is a little biased towards class 0 and 2, the score on leaderboard is still 0. Do you have any idea with what potentially could be wrong. I am using pytorch with fastai wrapper.
", "votes": 6, "replies": [ { "id": 567162, "author_name": "grantfrefg", "author_url": "", "post_date": "2019-07-03T06:05:03.903000", "content": "This happened to me as well. I managed to fix it by creating a new notebook with the exact same code and submitting it. I'm not exactly sure why it didn't work with my original notebook though.
", "votes": 1, "replies": [] }, { "id": 567643, "author_name": "Tushar", "author_url": "", "post_date": "2019-07-03T19:24:37.933000", "content": "Thanks @frefger, it did worked.
", "votes": 2, "replies": [] }, { "id": 569808, "author_name": "jxtrbtk", "author_url": "", "post_date": "2019-07-07T11:41:42.597000", "content": "I was also having a 0 score each time, but what has solved that was only to decrease the batchsize (in my case to 1). Was certainly a memory issue, but still hard to figure out without logs or even an error message. Thanks for this posts and comment that helped to solve !
", "votes": 0, "replies": [] } ] }, { "id": 567110, "author_name": "ilovescience", "author_url": "", "post_date": "2019-07-03T04:44:19.490000", "content": "\"your own beefy machine\" --> my beefy machine is Kaggle Kernels 😅
", "votes": 4, "replies": [ { "id": 568507, "author_name": "sh", "author_url": "", "post_date": "2019-07-05T03:15:11.683000", "content": "Kaggle kernels and google colab are very nice resources!\nThankfully its a synchronous KO competition, which allows you to train one model over multiple kernels so theoretically you could train your model for tens of hours in total.
\n\nTo do this, just download your model after running a kernel for the given 9 hours, then upload it as a custom dataset, import that dataset and continue training it for another 9 hours. Rinse and repeat until you are satisfied :)
", "votes": 0, "replies": [] } ] }, { "id": 570293, "author_name": "Nanashi", "author_url": "", "post_date": "2019-07-08T05:18:01.283000", "content": "Hi everyone, maybe you can help me, I've trained a model using this amazing kernel https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59
\n\nand when I try to load it:\n\nmodel.load_state_dict(torch.load(\"../input/aptos-pretrained-models/model.bin\"))\n\nI obtain this error:
\nError(s) in loading state_dict for ResNet:\n Missing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_li\n
I'm just trying the kernels from @abhishek , so I think this is not about the code :(
", "votes": 1, "replies": [ { "id": 570299, "author_name": "jionie", "author_url": "", "post_date": "2019-07-08T05:27:22.913000", "content": "load_state_dict(state_dict, strict=False) ?
", "votes": 0, "replies": [] }, { "id": 570303, "author_name": "Nanashi", "author_url": "", "post_date": "2019-07-08T05:30:09.380000", "content": "yeah, I tried but then the model doesn't work properly, I'll check it again ...\nI don't understand why abhishek doesn't have that error hahaha and I do using his own kernel\nanyway, thanks @jionie !
", "votes": 0, "replies": [] }, { "id": 570305, "author_name": "jionie", "author_url": "", "post_date": "2019-07-08T05:32:46.767000", "content": "Emmmm, interesting, I always open the internet and set pretrain=True😂
", "votes": 0, "replies": [] }, { "id": 570310, "author_name": "Abhishek Thakur", "author_url": "", "post_date": "2019-07-08T05:41:54.673000", "content": "@jesucristo To solve this kind of problem print the model that was trained and print the model before loading the weights.
\n\nIn this particular case, my guess is that you trained a model using torchvision which has fully connected layer named fc but are trying to load the model using pretrainedmodels where the fc layer is named last_linear . Changing the name of last layer to last_linear will work in this case. Let me know if that fixes your problem. :)
... solved ;) anoher thing I've learnt from you today.
", "votes": 1, "replies": [] }, { "id": 608732, "author_name": "purnasaiG", "author_url": "", "post_date": "2019-08-27T05:46:21.650000", "content": "Hi @jesucristo, I am trying to rename the last layer to last_linear, but i am not able to figure it out how. it would be helpful for me to know how to rename.My code here model.last_linear = nn.Sequential(\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.25),\n nn.Linear(in_features=2048, out_features=2048, bias=True),\n nn.ReLU(),\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.5),\n nn.Linear(in_features=2048, out_features=1, bias=True),\n )\n
Hello @abhishek, how do you infer that the unseen test set is 6 times greater than the public test set. If this confirms, I will be in trouble to get a final score.
\n\nThank you.\nJesús
", "votes": 0, "replies": [ { "id": 618814, "author_name": "Jesús MartÃn de la Sierra", "author_url": "", "post_date": "2019-09-05T14:27:15.497000", "content": "I'm sorry, I already saw in the notes provided.
\n\nThank you.
", "votes": 0, "replies": [] } ] }, { "id": 579945, "author_name": "saigautam", "author_url": "", "post_date": "2019-07-19T12:28:11.310000", "content": "Hello everyone, could someone tell me why my code is giving a error. The link to the kernel is \nHere\nAlso Im using fast ai package
", "votes": 0, "replies": [ { "id": 580224, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2019-07-19T21:25:31.957000", "content": "Make sure your outputs are int type, that was the issue with my submission float type is not supported. See your submission file has float, just do something like astype(np.int32).
Wow it worked, thanks for pointing it out
", "votes": 0, "replies": [] } ] }, { "id": 578763, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2019-07-18T05:50:46.367000", "content": "Thanks for this helpful post! I am confused about the memory issue part. If we run the notebook manually, say with a image size 120 - batch size 32 and see that it works and doesn't exceed the memory limit, then why might it fail during submission? Might gpu specs be different? Thanks :)
", "votes": 0, "replies": [] }, { "id": 574847, "author_name": "WolfgangReuter", "author_url": "", "post_date": "2019-07-14T14:57:00.617000", "content": "Hi and thanks for the post!
\n\nI have never used Kaggle kernels before - and my question is very simple: \nI have augmented images in a custom way and I tried to upload the images - but it was seen as using an external data set. Hence I augmented the images within the kernel, which lead to a rather heavy image list. Still works, but what I would like to know is: Can I save the images on the disk and retrieve them with a data generator? Which reduce the runtime of the script immensly. Thanks for your answer,
\n\nbest regards,
\n\nWolfgang
", "votes": 0, "replies": [] }, { "id": 570988, "author_name": "Drei", "author_url": "", "post_date": "2019-07-09T02:27:04.107000", "content": "Big help!
", "votes": 0, "replies": [] }, { "id": 569889, "author_name": "Jun Liu", "author_url": "", "post_date": "2019-07-07T14:10:10.790000", "content": "
Hi, Anyone knows why? I'm sure my submission file is in the right format and right row number. Committing a kernel almost cost 110 seconds, but when I click \"submit to competition\" button, it seems never got finished. I'm using GPU kernel and upload a well-trained model as a dataset, the model is used for inference, the prediction on the test dataset is expected to be finished within 30 mins. I checked my results and read the fine print and data description carefully, but still, don't know where is the problem. Any advice is appreciated!
", "votes": 0, "replies": [ { "id": 570723, "author_name": "Ken Ho", "author_url": "", "post_date": "2019-07-08T17:46:06.387000", "content": "Apart from the public test set, there is also a much bigger private test set
", "votes": 0, "replies": [] } ] }, { "id": 569101, "author_name": "Saravana kumar", "author_url": "", "post_date": "2019-07-06T04:06:58.837000", "content": "Thank you @abhishek . you're one person whom i constantly admire about.
", "votes": 0, "replies": [] }, { "id": 569073, "author_name": "ZNing", "author_url": "", "post_date": "2019-07-06T02:18:49.827000", "content": "thx~
", "votes": 0, "replies": [] }, { "id": 568558, "author_name": "Ken Ho", "author_url": "", "post_date": "2019-07-05T05:26:01.197000", "content": "Thanks for making this kernel @abhishek! I am confused about the unseen test set. How can we ensure that the unseen test data goes through the same steps (e.g. preprocessing for image input, scaling and transformation to ordinal values for output)?
", "votes": 0, "replies": [ { "id": 568673, "author_name": "Abhishek Thakur", "author_url": "", "post_date": "2019-07-05T09:34:38.827000", "content": "you have to handle all kinds of necessary steps in the inference kernel.
", "votes": 0, "replies": [] } ] }, { "id": 567585, "author_name": "Ibrahim Chaoudi ", "author_url": "", "post_date": "2019-07-03T18:19:24.263000", "content": "Thank you @abhishek , I appreciate that
", "votes": 0, "replies": [] }, { "id": 568818, "author_name": "", "author_url": "", "post_date": "2019-07-05T13:28:48.443000", "content": "", "votes": 0, "replies": [] }, { "id": 567157, "author_name": "", "author_url": "", "post_date": "2019-07-03T05:50:40.157000", "content": "", "votes": 1, "replies": [] }, { "id": 568656, "author_name": "Shubham Singh", "author_url": "", "post_date": "2019-07-05T09:14:03.937000", "content": "Quite helpful thanks for sharing...
", "votes": 1, "replies": [] }, { "id": 566370, "author_name": "rahul", "author_url": "", "post_date": "2019-07-02T05:23:33.317000", "content": "Thank you so much! This is helpful!
", "votes": 1, "replies": [] }, { "id": 980263, "author_name": "Sai Sudheer Vishnumolakala", "author_url": "", "post_date": "2020-08-21T12:51:04.033000", "content": "Quite helpful ,Thank you sir…
", "votes": 0, "replies": [] }, { "id": 576746, "author_name": "daniel chiang", "author_url": "", "post_date": "2019-07-16T00:48:38.370000", "content": "great information ,thanks your sharing
", "votes": 0, "replies": [] }, { "id": 567825, "author_name": "ynhuhu", "author_url": "", "post_date": "2019-07-04T03:43:32.150000", "content": "Thanks for you sharing, very helpful.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "566062": "A lot of people seem to be having problems with submitting in this competition, so I'm creating this guide.\n\nThis is Synchronous Kernels Only competition where your kernel will run against an unseen dataset in addition to the visible test set!\n\n\n## The training phase\n\nFirst of all, you need to train the model. In this kernel here: https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59 I have shown how to train a simple high scoring model using pytorch.\n\nIn the \"Get the model\" section, you can find the model definition which is a simple ResNet101.\n\n\n```\nmodel = torchvision.models.resnet101(pretrained=False)\nmodel.load_state_dict(torch.load(\"../input/pytorch-pretrained-models/resnet101-5d3b4d8f.pth\"))\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\n```\n\n\nSince pretrained weights are allowed, we will use pre-trained imagenet weights as the starting point.\n\nOnce the model is trained and you are happy with the loss/accuracy, save the model weights.\n\nIf you are using keras, you can do:\n\n```\nfrom keras.models import load_model\nmodel.save('model.h5')\n```\n\nIf you are using pytorch, look at the kernel mentioned above. It saves a \"model.bin\":\n\n\n```\ntorch.save(model.state_dict(), \"model.bin\")\n```\n\nNow remember 2 things. The model and the preprocessing! For example, if you resized training images to 128x128, you need to do the same for test images (most of the time).\n\nSo, once you have the model saved, click on datasets on the top of the page, and save you model in kaggle datasets.\n\n\nThis step does not need to run on kaggle kernels, ofcourse you can use it but if you prefer your own beefy machine, feel free to do so.\n\n\nNow moving to the next and final step.\n\n\n## The inference phase\n\nDuring inference, you will make your predictions on the unseen test images. Unseen? YES! We have two sets of test images, one that you can download and other one you cannot. The inference step must run in a Kaggle Kernel.\n\nAfter you start the kernel, add the dataset you uploaded in the previous step.\n\nWhat's different when your kernel runs on unseen test images:\n\n- sample_submission.csv\n- test.csv\n- test.zip\n\nKeep in mind that we dont know how many test images are there in the second stage. \n\n\nYou can base your inference kernel on: https://www.kaggle.com/abhishek/pytorch-inference-kernel-lazy-tta\n\nNow you need the same model again!\n\npytorch:\n\n```\nmodel = torchvision.models.resnet101(pretrained=False)\nnum_features = model.fc.in_features\nmodel.fc = nn.Linear(2048, 1)\nmodel.load_state_dict(torch.load(\"../input/model/model.bin\"))\n```\n\nin keras, you can do:\n\n```\nfrom keras.models import load_model\nmodel = load_model('../model/model.h5')\n```\n\nOnce you have the model in place, start generating predictions for test images using either sample_submission.csv or test.csv. You need to use the same pre-processing as you did for training.!\n\n\nYou need to take care of the following things:\n- kernels dont have a lot of memory, use small images\n- use small batches\n- do not hardcode number of test images! you dont know how many are there!\n- unseen test images are approximately 6 times the test images that you see.\n\nStart with very simple inference kernel with 16 batch size and 128x128 images. Once that works, improve it!\n\nOnce the kernel is working fine, commit it to run it from top to bottom. When the commit is ready, select the submission.csv in output tab and submit to the competition and wait.\n\n\n*NOTE: If you make a mistake and kernel fails, you will lose a submission. So, submit only when you are sure that you are not using anything that will break!!*\n\n---\n\n\nIn case of any questions, ask here and ill try my best to answer :)\n\n\n\n\n\n\n\n", "569857": "I copy paste my post from my discussion just in case it can helps ! \n\nHello everyone, after few days of research, I finaly found the problem. It was not about memory, running time or whatever. The problem was nothing else than my preprocessing function. It was croping black border to reduce the noise and maximise the signal. The preprocessing function was working perfectly on public test set and training set. However, I truely believe that the private test set got fully dark / black images. Resulting in a full croping (image[0:0] or something like that). Therefore, the predict fail and to_csv also.\n\nHope it will help ! GL\n\nVincent", "567133": "This is very helpful @abhishek . I face a little different problem with my kernel. During training, the weighted kappa score of the model is above ~0.7 but after I run predictions to submit the results, I get a leaderboard score of 0. Although my model is a little biased towards class 0 and 2, the score on leaderboard is still 0. Do you have any idea with what potentially could be wrong. I am using _pytorch_ with _fastai_ wrapper.", "567110": "\"your own beefy machine\" --> my beefy machine is Kaggle Kernels 😅 ", "570293": "Hi everyone, maybe you can help me, I've trained a model using this amazing kernel https://www.kaggle.com/abhishek/very-simple-pytorch-training-0-59\n\nand when I try to load it:\n``` \nmodel.load_state_dict(torch.load(\"../input/aptos-pretrained-models/model.bin\"))\n```\nI obtain this error:\n\n```\nError(s) in loading state_dict for ResNet:\n\tMissing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_li\n```\n\nI'm just trying the kernels from @abhishek , so I think this is not about the code :(", "618339": "Hello @abhishek, how do you infer that the unseen test set is 6 times greater than the public test set. If this confirms, I will be in trouble to get a final score.\n\nThank you.\nJesús", "579945": "Hello everyone, could someone tell me why my code is giving a error. The link to the kernel is \n[Here](https://www.kaggle.com/saigautam/submition-ensembling-baseline)\nAlso Im using fast ai package", "578763": "Thanks for this helpful post! I am confused about the memory issue part. If we run the notebook manually, say with a image size 120 - batch size 32 and see that it works and doesn't exceed the memory limit, then why might it fail during submission? Might gpu specs be different? Thanks :)", "574847": "Hi and thanks for the post!\n\nI have never used Kaggle kernels before - and my question is very simple: \nI have augmented images in a custom way and I tried to upload the images - but it was seen as using an external data set. Hence I augmented the images within the kernel, which lead to a rather heavy image list. Still works, but what I would like to know is: Can I save the images on the disk and retrieve them with a data generator? Which reduce the runtime of the script immensly. Thanks for your answer, \n\nbest regards, \n\nWolfgang", "570988": "Big help!", "569889": "\n\nHi, Anyone knows why? I'm sure my submission file is in the right format and right row number. Committing a kernel almost cost 110 seconds, but when I click \"submit to competition\" button, it seems never got finished. I'm using GPU kernel and upload a well-trained model as a dataset, the model is used for inference, the prediction on the test dataset is expected to be finished within 30 mins. I checked my results and read the fine print and data description carefully, but still, don't know where is the problem. Any advice is appreciated!", "569101": "Thank you @abhishek . you're one person whom i constantly admire about.", "569073": "thx~", "568558": "Thanks for making this kernel @abhishek! I am confused about the unseen test set. How can we ensure that the unseen test data goes through the same steps (e.g. preprocessing for image input, scaling and transformation to ordinal values for output)?", "567585": "Thank you @abhishek , I appreciate that", "568818": "", "567157": "", "568656": "Quite helpful thanks for sharing...", "566370": "Thank you so much! This is helpful! ", "980263": "Quite helpful ,Thank you sir...", "576746": "great information ,thanks your sharing", "567825": "Thanks for you sharing, very helpful." } }