{
  "id": 559863,
  "title": "About making u-net model",
  "url": "/competitions/czii-cryo-et-object-identification/discussion/559863",
  "author_name": "",
  "post_date": "2025-01-28T07:58:12.730692600Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hello everyone! Thank you always for your support.</p>\n<p>I have a question regarding the weight structure of U-Net models in PyTorch versus PyTorch Lightning.<br>\nI’m currently working on implementing a U-Net model, and I came across this baseline code (<a href=\"https://www.kaggle.com/code/hideyukizushi/czii-yolo11-unet3d-monai-lb-707/notebook)that\" target=\"_blank\">https://www.kaggle.com/code/hideyukizushi/czii-yolo11-unet3d-monai-lb-707/notebook)that</a> loads pre-trained weights directly. However, I couldn’t figure out how the training process was conducted in the reference implementation.</p>\n<p>Specifically</p>\n<p>Are there structural differences in model weights between PyTorch and PyTorch Lightning implementations?</p>\n<p>Should I build my model using pure PyTorch or PyTorch Lightning to ensure compatibility when loading weights?</p>\n<p>If anyone has insights or experience with this, I’d greatly appreciate your guidance!</p>\n<p>Thank you in advance for your help! 🙏</p>",
  "messages": [
    {
      "id": "3108880",
      "postDate": "01/28/2025 07:58:12",
      "content": "<p>Hello everyone! Thank you always for your support.</p>\n<p>I have a question regarding the weight structure of U-Net models in PyTorch versus PyTorch Lightning.<br>\nI’m currently working on implementing a U-Net model, and I came across this baseline code (<a href=\"https://www.kaggle.com/code/hideyukizushi/czii-yolo11-unet3d-monai-lb-707/notebook)that\" target=\"_blank\">https://www.kaggle.com/code/hideyukizushi/czii-yolo11-unet3d-monai-lb-707/notebook)that</a> loads pre-trained weights directly. However, I couldn’t figure out how the training process was conducted in the reference implementation.</p>\n<p>Specifically</p>\n<p>Are there structural differences in model weights between PyTorch and PyTorch Lightning implementations?</p>\n<p>Should I build my model using pure PyTorch or PyTorch Lightning to ensure compatibility when loading weights?</p>\n<p>If anyone has insights or experience with this, I’d greatly appreciate your guidance!</p>\n<p>Thank you in advance for your help! 🙏</p>",
      "rawMarkdown": "Hello everyone! Thank you always for your support.\n\nI have a question regarding the weight structure of U-Net models in PyTorch versus PyTorch Lightning.\nI’m currently working on implementing a U-Net model, and I came across this baseline code (https://www.kaggle.com/code/hideyukizushi/czii-yolo11-unet3d-monai-lb-707/notebook)that loads pre-trained weights directly. However, I couldn’t figure out how the training process was conducted in the reference implementation.\n\nSpecifically\n\n   Are there structural differences in model weights between PyTorch and PyTorch Lightning implementations?\n\n   Should I build my model using pure PyTorch or PyTorch Lightning to ensure compatibility when loading weights?\n\nIf anyone has insights or experience with this, I’d greatly appreciate your guidance!\n\nThank you in advance for your help! 🙏",
      "votes": null
    },
    {
      "id": "3109131",
      "postDate": "01/28/2025 14:24:27",
      "content": "<p>The choice of PyTorch vs PyTorch Lightning is entirely based on your preference.  Both will run native PyTorch code, PyTorch Lightning simply abstracts certain steps.  For example, in PyTorch you have to set the model to train, calculate the loss, set the optimizer gradient to zero, run back propagation, take the optimization step whereas in PyTorch Lightning it will do this for you after running training_step() and all you have to do is handle the loss calculation and model prediction.  PyTorch Lightning has ckpt files which save everything that you would need to continue training the model, however you can also save a .pt file just like you would in native PyTorch.  Your results in both will be identical so it just comes down to what you prefer.  Another benefit to PyTorch Lightning is that it has classes for things like saving the best k models, stopping training after the model stops performing better on the validation set, and so on.  </p>\n<p>In the end, it truly comes down to whatever you are most comfortable with.  I personally alternate between the two based on how much custom code I think I will be writing.  For your submission notebook I do recommend using just PyTorch because installing PyTorch lightning without internet just takes another step but for training use whatever you find more useful :)</p>",
      "rawMarkdown": "The choice of PyTorch vs PyTorch Lightning is entirely based on your preference.  Both will run native PyTorch code, PyTorch Lightning simply abstracts certain steps.  For example, in PyTorch you have to set the model to train, calculate the loss, set the optimizer gradient to zero, run back propagation, take the optimization step whereas in PyTorch Lightning it will do this for you after running training_step() and all you have to do is handle the loss calculation and model prediction.  PyTorch Lightning has ckpt files which save everything that you would need to continue training the model, however you can also save a .pt file just like you would in native PyTorch.  Your results in both will be identical so it just comes down to what you prefer.  Another benefit to PyTorch Lightning is that it has classes for things like saving the best k models, stopping training after the model stops performing better on the validation set, and so on.  \n\nIn the end, it truly comes down to whatever you are most comfortable with.  I personally alternate between the two based on how much custom code I think I will be writing.  For your submission notebook I do recommend using just PyTorch because installing PyTorch lightning without internet just takes another step but for training use whatever you find more useful :)",
      "votes": null
    },
    {
      "id": "3109450",
      "postDate": "01/28/2025 20:57:27",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/hikarukai\" target=\"_blank\">@hikarukai</a> , There are no structural differences in the model weights themselves between PyTorch and PyTorch Lightning implementations. Both frameworks store the same weight tensors in the same format because PyTorch Lightning is built on top of PyTorch. </p>",
      "rawMarkdown": "Hi @hikarukai , There are no structural differences in the model weights themselves between PyTorch and PyTorch Lightning implementations. Both frameworks store the same weight tensors in the same format because PyTorch Lightning is built on top of PyTorch.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3109131,
      "author_name": "connorjd",
      "author_url": "",
      "post_date": "01/28/2025 14:24:27",
      "content": "<p>The choice of PyTorch vs PyTorch Lightning is entirely based on your preference.  Both will run native PyTorch code, PyTorch Lightning simply abstracts certain steps.  For example, in PyTorch you have to set the model to train, calculate the loss, set the optimizer gradient to zero, run back propagation, take the optimization step whereas in PyTorch Lightning it will do this for you after running training_step() and all you have to do is handle the loss calculation and model prediction.  PyTorch Lightning has ckpt files which save everything that you would need to continue training the model, however you can also save a .pt file just like you would in native PyTorch.  Your results in both will be identical so it just comes down to what you prefer.  Another benefit to PyTorch Lightning is that it has classes for things like saving the best k models, stopping training after the model stops performing better on the validation set, and so on.  </p>\n<p>In the end, it truly comes down to whatever you are most comfortable with.  I personally alternate between the two based on how much custom code I think I will be writing.  For your submission notebook I do recommend using just PyTorch because installing PyTorch lightning without internet just takes another step but for training use whatever you find more useful :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3109450,
      "author_name": "sergiosaharovskiy",
      "author_url": "",
      "post_date": "01/28/2025 20:57:27",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/hikarukai\" target=\"_blank\">@hikarukai</a> , There are no structural differences in the model weights themselves between PyTorch and PyTorch Lightning implementations. Both frameworks store the same weight tensors in the same format because PyTorch Lightning is built on top of PyTorch. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3108880": "Hello everyone! Thank you always for your support.\n\nI have a question regarding the weight structure of U-Net models in PyTorch versus PyTorch Lightning.\nI’m currently working on implementing a U-Net model, and I came across this baseline code (https://www.kaggle.com/code/hideyukizushi/czii-yolo11-unet3d-monai-lb-707/notebook)that loads pre-trained weights directly. However, I couldn’t figure out how the training process was conducted in the reference implementation.\n\nSpecifically\n\n   Are there structural differences in model weights between PyTorch and PyTorch Lightning implementations?\n\n   Should I build my model using pure PyTorch or PyTorch Lightning to ensure compatibility when loading weights?\n\nIf anyone has insights or experience with this, I’d greatly appreciate your guidance!\n\nThank you in advance for your help! 🙏",
    "3109131": "The choice of PyTorch vs PyTorch Lightning is entirely based on your preference.  Both will run native PyTorch code, PyTorch Lightning simply abstracts certain steps.  For example, in PyTorch you have to set the model to train, calculate the loss, set the optimizer gradient to zero, run back propagation, take the optimization step whereas in PyTorch Lightning it will do this for you after running training_step() and all you have to do is handle the loss calculation and model prediction.  PyTorch Lightning has ckpt files which save everything that you would need to continue training the model, however you can also save a .pt file just like you would in native PyTorch.  Your results in both will be identical so it just comes down to what you prefer.  Another benefit to PyTorch Lightning is that it has classes for things like saving the best k models, stopping training after the model stops performing better on the validation set, and so on.  \n\nIn the end, it truly comes down to whatever you are most comfortable with.  I personally alternate between the two based on how much custom code I think I will be writing.  For your submission notebook I do recommend using just PyTorch because installing PyTorch lightning without internet just takes another step but for training use whatever you find more useful :)",
    "3109450": "Hi @hikarukai , There are no structural differences in the model weights themselves between PyTorch and PyTorch Lightning implementations. Both frameworks store the same weight tensors in the same format because PyTorch Lightning is built on top of PyTorch."
  },
  "source": "meta"
}