{
  "id": 268621,
  "title": "Freeze or No Freeze in Transfer Learning ? ",
  "url": "/competitions/rsna-miccai-brain-tumor-radiogenomic-classification/discussion/268621",
  "author_name": "",
  "post_date": "2021-08-28T05:08:39.147320600Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi All , <br>\nIt seems that people have been using efficient net mostly but now a simple question comes to me my mind , the pretrained networks are huge and i have seen people not freezing anything and running to the update the weights again , is this approach good ,because the original method may have used a different loss function to reach the minima , also I understand doing this will improve the performance but is it substantial ? </p>",
  "messages": [
    {
      "id": "1493631",
      "postDate": "08/28/2021 05:08:39",
      "content": "<p>Hi All , <br>\nIt seems that people have been using efficient net mostly but now a simple question comes to me my mind , the pretrained networks are huge and i have seen people not freezing anything and running to the update the weights again , is this approach good ,because the original method may have used a different loss function to reach the minima , also I understand doing this will improve the performance but is it substantial ? </p>",
      "rawMarkdown": "Hi All , \nIt seems that people have been using efficient net mostly but now a simple question comes to me my mind , the pretrained networks are huge and i have seen people not freezing anything and running to the update the weights again , is this approach good ,because the original method may have used a different loss function to reach the minima , also I understand doing this will improve the performance but is it substantial ?",
      "votes": null
    },
    {
      "id": "1494566",
      "postDate": "08/28/2021 19:20:33",
      "content": "<p>I usually experiment with both the ideas and use that method giving better CV score</p>",
      "rawMarkdown": "I usually experiment with both the ideas and use that method giving better CV score",
      "votes": null
    },
    {
      "id": "1495628",
      "postDate": "08/29/2021 16:54:48",
      "content": "<p>But is there any rule of thumb we can follow ??</p>",
      "rawMarkdown": "But is there any rule of thumb we can follow ??",
      "votes": null
    },
    {
      "id": "1495806",
      "postDate": "08/29/2021 19:57:37",
      "content": "<p>I think most people are using Efficientnet3d and I believe that is not pretrained, so freezing wouldn't make sense. Freezing only makes sense if you already have a pretrained model. But most people on this website don't freeze their models. For example, if you look at the top solutions for NLP competitions, people re-train all of the layers of BERT. So I would not recommend freezing unless the pretrained task is very similar to our task</p>",
      "rawMarkdown": "I think most people are using Efficientnet3d and I believe that is not pretrained, so freezing wouldn't make sense. Freezing only makes sense if you already have a pretrained model. But most people on this website don't freeze their models. For example, if you look at the top solutions for NLP competitions, people re-train all of the layers of BERT. So I would not recommend freezing unless the pretrained task is very similar to our task",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1494566,
      "author_name": "mrinath",
      "author_url": "",
      "post_date": "08/28/2021 19:20:33",
      "content": "<p>I usually experiment with both the ideas and use that method giving better CV score</p>",
      "votes": null,
      "replies": [
        {
          "id": 1495628,
          "author_name": "avikrams",
          "author_url": "",
          "post_date": "08/29/2021 16:54:48",
          "content": "<p>But is there any rule of thumb we can follow ??</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1495806,
      "author_name": "returnofsputnik",
      "author_url": "",
      "post_date": "08/29/2021 19:57:37",
      "content": "<p>I think most people are using Efficientnet3d and I believe that is not pretrained, so freezing wouldn't make sense. Freezing only makes sense if you already have a pretrained model. But most people on this website don't freeze their models. For example, if you look at the top solutions for NLP competitions, people re-train all of the layers of BERT. So I would not recommend freezing unless the pretrained task is very similar to our task</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1493631": "Hi All , \nIt seems that people have been using efficient net mostly but now a simple question comes to me my mind , the pretrained networks are huge and i have seen people not freezing anything and running to the update the weights again , is this approach good ,because the original method may have used a different loss function to reach the minima , also I understand doing this will improve the performance but is it substantial ?",
    "1494566": "I usually experiment with both the ideas and use that method giving better CV score",
    "1495628": "But is there any rule of thumb we can follow ??",
    "1495806": "I think most people are using Efficientnet3d and I believe that is not pretrained, so freezing wouldn't make sense. Freezing only makes sense if you already have a pretrained model. But most people on this website don't freeze their models. For example, if you look at the top solutions for NLP competitions, people re-train all of the layers of BERT. So I would not recommend freezing unless the pretrained task is very similar to our task"
  },
  "source": "meta"
}