![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3778743%2Fd9f332e4ac2a74310f165487df7203c1%2FBildschirmfoto%202022-08-13%20um%2021.16.12.png?generation=1660419572592580&alt=media\")
Hello everyone, I just stumbled upon something unexpected: setting model.train() instead of model.eval() results in a higher leaderboard score:
The only difference in these notebooks is the line model.train() vs. model.eval() right before inference. Both notebooks use a UNet with efficientnet encoder from the segmentation-models-pytorch library, the same weights and a (inference) batch size of 1.
Is this common behaviour?
\nMaybe it's worth trying out to get some improvements :)
", "messages": [ { "id": "1897502", "postDate": "08/13/2022 19:46:05", "content": "Hello everyone, I just stumbled upon something unexpected: setting model.train() instead of model.eval() results in a higher leaderboard score:
The only difference in these notebooks is the line model.train() vs. model.eval() right before inference. Both notebooks use a UNet with efficientnet encoder from the segmentation-models-pytorch library, the same weights and a (inference) batch size of 1.
Is this common behaviour?
\nMaybe it's worth trying out to get some improvements :)
", "rawMarkdown": "Hello everyone, I just stumbled upon something unexpected: setting `model.train()` instead of `model.eval()` results in a higher leaderboard score:\n\n\n\nThe only difference in these notebooks is the line `model.train()` vs. `model.eval()` right before inference. Both notebooks use a UNet with efficientnet encoder from the [segmentation-models-pytorch](https://github.com/qubvel/segmentation_models.pytorch) library, the same weights and a (inference) batch size of 1. \n\nIs this common behaviour?\n\nMaybe it's worth trying out to get some improvements :)", "votes": null }, { "id": "1897943", "postDate": "08/14/2022 06:57:05", "content": "it's werid.. model.eval() turns off batch norm or dropout when applied during the training phase. model.eval() is correct in the case of inference.
", "rawMarkdown": "it's werid.. model.eval() turns off batch norm or dropout when applied during the training phase. model.eval() is correct in the case of inference.", "votes": null }, { "id": "1898110", "postDate": "08/14/2022 10:00:07", "content": "Didn't work for me though 🤒
", "rawMarkdown": "Didn't work for me though 🤒", "votes": null }, { "id": "1900192", "postDate": "08/15/2022 19:06:17", "content": "Thats so interesting! I think perhaps similar to @deepkim said, using model.train() uses batch norm or dropout, so perhaps the model learned how to use those to its advantage somehow?
", "rawMarkdown": "Thats so interesting! I think perhaps similar to @deepkim said, using model.train() uses batch norm or dropout, so perhaps the model learned how to use those to its advantage somehow?", "votes": null }, { "id": "1901713", "postDate": "08/16/2022 20:38:05", "content": "This isn't expected behavior. I could imagine if your model was overfit it might actually benefit from having dropout enabled at inference time, definitely hard to say though
", "rawMarkdown": "This isn't expected behavior. I could imagine if your model was overfit it might actually benefit from having dropout enabled at inference time, definitely hard to say though", "votes": null } ], "comments": [ { "id": 1897943, "author_name": "deepkim", "author_url": "", "post_date": "08/14/2022 06:57:05", "content": "it's werid.. model.eval() turns off batch norm or dropout when applied during the training phase. model.eval() is correct in the case of inference.
", "votes": null, "replies": [] }, { "id": 1898110, "author_name": "kfk42kfk", "author_url": "", "post_date": "08/14/2022 10:00:07", "content": "Didn't work for me though 🤒
", "votes": null, "replies": [] }, { "id": 1900192, "author_name": "aadityaagnihotri", "author_url": "", "post_date": "08/15/2022 19:06:17", "content": "Thats so interesting! I think perhaps similar to @deepkim said, using model.train() uses batch norm or dropout, so perhaps the model learned how to use those to its advantage somehow?
", "votes": null, "replies": [] }, { "id": 1901713, "author_name": "maxvandijck", "author_url": "", "post_date": "08/16/2022 20:38:05", "content": "This isn't expected behavior. I could imagine if your model was overfit it might actually benefit from having dropout enabled at inference time, definitely hard to say though
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1897502": "Hello everyone, I just stumbled upon something unexpected: setting `model.train()` instead of `model.eval()` results in a higher leaderboard score:\n\n\n\nThe only difference in these notebooks is the line `model.train()` vs. `model.eval()` right before inference. Both notebooks use a UNet with efficientnet encoder from the [segmentation-models-pytorch](https://github.com/qubvel/segmentation_models.pytorch) library, the same weights and a (inference) batch size of 1. \n\nIs this common behaviour?\n\nMaybe it's worth trying out to get some improvements :)", "1897943": "it's werid.. model.eval() turns off batch norm or dropout when applied during the training phase. model.eval() is correct in the case of inference.", "1898110": "Didn't work for me though 🤒", "1900192": "Thats so interesting! I think perhaps similar to @deepkim said, using model.train() uses batch norm or dropout, so perhaps the model learned how to use those to its advantage somehow?", "1901713": "This isn't expected behavior. I could imagine if your model was overfit it might actually benefit from having dropout enabled at inference time, definitely hard to say though" }, "source": "meta" }