When I try to load the model like shown below:
\n\nmodel = pretrainedmodels.__dict__['resnet101'](pretrained=None)\nmodel.avg_pool = nn.AdaptiveAvgPool2d(1)\nmodel.last_linear = nn.Sequential(\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.25),\n nn.Linear(in_features=2048, out_features=2048, bias=True),\n nn.ReLU(),\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.5),\n nn.Linear(in_features=2048, out_features=1, bias=True),\n )\nmodel.load_state_dict(torch.load(\"../input/myfile/newmodel.bin\"))\nmodel = model.to(device)
Throws me an error something like
\n\nRuntimeError: Error(s) in loading state_dict for ResNet:\n Missing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_linear.4.running_mean\", \"last_linear.4.running_var\", \"last_linear.6.weight\", \"last_linear.6.bias\". \n Unexpected key(s) in state_dict: \"last_linear.weight\", \"last_linear.bias\".
I found the solution here in discussion thread as
\n\n`To solve this kind of problem print the model that was trained and print the model before loading the weights.
\n\nIn this particular case, my guess is that you trained a model using torchvision which has fully connected layer named fc but are trying to load the model using pretrainedmodels where the fc layer is named last_linear . Changing the name of last layer to last_linear will work in this case. Let me know if that fixes your problem.`
\n\nbut i am not able to rename last layer to last_linear. It would be helpful for me if anyone could find out.
", "messages": [ { "id": "617288", "postDate": "09/04/2019 01:24:08", "content": "When I try to load the model like shown below:
\n\nmodel = pretrainedmodels.__dict__['resnet101'](pretrained=None)\nmodel.avg_pool = nn.AdaptiveAvgPool2d(1)\nmodel.last_linear = nn.Sequential(\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.25),\n nn.Linear(in_features=2048, out_features=2048, bias=True),\n nn.ReLU(),\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.5),\n nn.Linear(in_features=2048, out_features=1, bias=True),\n )\nmodel.load_state_dict(torch.load(\"../input/myfile/newmodel.bin\"))\nmodel = model.to(device)
Throws me an error something like
\n\nRuntimeError: Error(s) in loading state_dict for ResNet:\n Missing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_linear.4.running_mean\", \"last_linear.4.running_var\", \"last_linear.6.weight\", \"last_linear.6.bias\". \n Unexpected key(s) in state_dict: \"last_linear.weight\", \"last_linear.bias\".
I found the solution here in discussion thread as
\n\n`To solve this kind of problem print the model that was trained and print the model before loading the weights.
\n\nIn this particular case, my guess is that you trained a model using torchvision which has fully connected layer named fc but are trying to load the model using pretrainedmodels where the fc layer is named last_linear . Changing the name of last layer to last_linear will work in this case. Let me know if that fixes your problem.`
\n\nbut i am not able to rename last layer to last_linear. It would be helpful for me if anyone could find out.
", "rawMarkdown": "When I try to load the model like shown below:\n\n`model = pretrainedmodels.__dict__['resnet101'](pretrained=None)\nmodel.avg_pool = nn.AdaptiveAvgPool2d(1)\nmodel.last_linear = nn.Sequential(\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.25),\n nn.Linear(in_features=2048, out_features=2048, bias=True),\n nn.ReLU(),\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.5),\n nn.Linear(in_features=2048, out_features=1, bias=True),\n )\nmodel.load_state_dict(torch.load(\"../input/myfile/newmodel.bin\"))\nmodel = model.to(device)`\n\nThrows me an error something like \n\n`RuntimeError: Error(s) in loading state_dict for ResNet:\n\tMissing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_linear.4.running_mean\", \"last_linear.4.running_var\", \"last_linear.6.weight\", \"last_linear.6.bias\". \n\tUnexpected key(s) in state_dict: \"last_linear.weight\", \"last_linear.bias\". `\n\nI found the solution [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98168) in discussion thread as \n\n`To solve this kind of problem print the model that was trained and print the model before loading the weights.\n\nIn this particular case, my guess is that you trained a model using torchvision which has fully connected layer named fc but are trying to load the model using pretrainedmodels where the fc layer is named last_linear . Changing the name of last layer to last_linear will work in this case. Let me know if that fixes your problem.` \n\nbut i am not able to rename last layer to last_linear. It would be helpful for me if anyone could find out.", "votes": null }, { "id": "617367", "postDate": "09/04/2019 03:52:15", "content": "Save the weights for fc layer of pytorch pretrained model first. \nMake a new nn.Sequential named last_linear and assign its weight from the saved weight.\nAlso look at pytorch forum posts : \n1. https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113\n2. https://discuss.pytorch.org/t/load-part-of-pretrained-model-with-strict-false-in-load-state-dict/21337
", "rawMarkdown": "Save the weights for fc layer of pytorch pretrained model first. \nMake a new nn.Sequential named last_linear and assign its weight from the saved weight.\nAlso look at pytorch forum posts : \n1. https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113\n2. https://discuss.pytorch.org/t/load-part-of-pretrained-model-with-strict-false-in-load-state-dict/21337", "votes": null }, { "id": "617397", "postDate": "09/04/2019 04:59:51", "content": "Thank you for the response @abyaadrafid. I tried as shown in the link2, but my model is not working correctly. it gives me '0' as prediction for the 1920 test set. I dont know why?
", "rawMarkdown": "Thank you for the response @abyaadrafid. I tried as shown in the [link2](https://discuss.pytorch.org/t/load-part-of-pretrained-model-with-strict-false-in-load-state-dict/21337), but my model is not working correctly. it gives me '0' as prediction for the 1920 test set. I dont know why?", "votes": null }, { "id": "617578", "postDate": "09/04/2019 09:19:26", "content": "Did you try with other models? If there is no error, it should work.\nAlso remember to load weights for your new layer from the old one, if you are using the model without training to make predictions.
", "rawMarkdown": "Did you try with other models? If there is no error, it should work.\nAlso remember to load weights for your new layer from the old one, if you are using the model without training to make predictions.", "votes": null } ], "comments": [ { "id": 617367, "author_name": "abyaadrafid", "author_url": "", "post_date": "09/04/2019 03:52:15", "content": "Save the weights for fc layer of pytorch pretrained model first. \nMake a new nn.Sequential named last_linear and assign its weight from the saved weight.\nAlso look at pytorch forum posts : \n1. https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113\n2. https://discuss.pytorch.org/t/load-part-of-pretrained-model-with-strict-false-in-load-state-dict/21337
", "votes": null, "replies": [ { "id": 617397, "author_name": "purnasaig", "author_url": "", "post_date": "09/04/2019 04:59:51", "content": "Thank you for the response @abyaadrafid. I tried as shown in the link2, but my model is not working correctly. it gives me '0' as prediction for the 1920 test set. I dont know why?
", "votes": null, "replies": [] }, { "id": 617578, "author_name": "abyaadrafid", "author_url": "", "post_date": "09/04/2019 09:19:26", "content": "Did you try with other models? If there is no error, it should work.\nAlso remember to load weights for your new layer from the old one, if you are using the model without training to make predictions.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "617288": "When I try to load the model like shown below:\n\n`model = pretrainedmodels.__dict__['resnet101'](pretrained=None)\nmodel.avg_pool = nn.AdaptiveAvgPool2d(1)\nmodel.last_linear = nn.Sequential(\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.25),\n nn.Linear(in_features=2048, out_features=2048, bias=True),\n nn.ReLU(),\n nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),\n nn.Dropout(p=0.5),\n nn.Linear(in_features=2048, out_features=1, bias=True),\n )\nmodel.load_state_dict(torch.load(\"../input/myfile/newmodel.bin\"))\nmodel = model.to(device)`\n\nThrows me an error something like \n\n`RuntimeError: Error(s) in loading state_dict for ResNet:\n\tMissing key(s) in state_dict: \"last_linear.0.weight\", \"last_linear.0.bias\", \"last_linear.0.running_mean\", \"last_linear.0.running_var\", \"last_linear.2.weight\", \"last_linear.2.bias\", \"last_linear.4.weight\", \"last_linear.4.bias\", \"last_linear.4.running_mean\", \"last_linear.4.running_var\", \"last_linear.6.weight\", \"last_linear.6.bias\". \n\tUnexpected key(s) in state_dict: \"last_linear.weight\", \"last_linear.bias\". `\n\nI found the solution [here](https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/98168) in discussion thread as \n\n`To solve this kind of problem print the model that was trained and print the model before loading the weights.\n\nIn this particular case, my guess is that you trained a model using torchvision which has fully connected layer named fc but are trying to load the model using pretrainedmodels where the fc layer is named last_linear . Changing the name of last layer to last_linear will work in this case. Let me know if that fixes your problem.` \n\nbut i am not able to rename last layer to last_linear. It would be helpful for me if anyone could find out.", "617367": "Save the weights for fc layer of pytorch pretrained model first. \nMake a new nn.Sequential named last_linear and assign its weight from the saved weight.\nAlso look at pytorch forum posts : \n1. https://discuss.pytorch.org/t/how-to-load-part-of-pre-trained-model/1113\n2. https://discuss.pytorch.org/t/load-part-of-pretrained-model-with-strict-false-in-load-state-dict/21337", "617397": "Thank you for the response @abyaadrafid. I tried as shown in the [link2](https://discuss.pytorch.org/t/load-part-of-pretrained-model-with-strict-false-in-load-state-dict/21337), but my model is not working correctly. it gives me '0' as prediction for the 1920 test set. I dont know why?", "617578": "Did you try with other models? If there is no error, it should work.\nAlso remember to load weights for your new layer from the old one, if you are using the model without training to make predictions." }, "source": "meta" }