{
  "id": 78296,
  "title": "Question about feature engineering",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/78296",
  "author_name": "",
  "post_date": "2019-01-22T04:15:48.654651900Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi all</p>\n\n<p>I am new in neural network and probably the question is late, but I still have some questions. The question is </p>\n\n<p>Eg. I fine tuned my Resnet model and get the first effective weight-1. If I want to test the effect of different aug, should I <strong>repeat the procedures of fine tune to get a new weight</strong> or I can <strong>use the weight-1 as the initial weight then train the resnet</strong>?</p>\n\n<p>Specific example:\nWhen I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1. Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.</p>\n\n<p>My guess is that if I use the imagenet weight each time, it will be very time-consuming.\nBut if I use the weight-1 as initial weight, the speed of training will be faster but it may stuck in local minimum.</p>\n\n<p>If you know about this, can you give me some instructions? Thanks</p>",
  "messages": [
    {
      "id": "459604",
      "postDate": "01/22/2019 04:15:48",
      "content": "<p>Hi all</p>\n\n<p>I am new in neural network and probably the question is late, but I still have some questions. The question is </p>\n\n<p>Eg. I fine tuned my Resnet model and get the first effective weight-1. If I want to test the effect of different aug, should I <strong>repeat the procedures of fine tune to get a new weight</strong> or I can <strong>use the weight-1 as the initial weight then train the resnet</strong>?</p>\n\n<p>Specific example:\nWhen I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1. Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.</p>\n\n<p>My guess is that if I use the imagenet weight each time, it will be very time-consuming.\nBut if I use the weight-1 as initial weight, the speed of training will be faster but it may stuck in local minimum.</p>\n\n<p>If you know about this, can you give me some instructions? Thanks</p>",
      "rawMarkdown": "Hi all\n\nI am new in neural network and probably the question is late, but I still have some questions. The question is \n\nEg. I fine tuned my Resnet model and get the first effective weight-1. If I want to test the effect of different aug, should I **repeat the procedures of fine tune to get a new weight** or I can **use the weight-1 as the initial weight then train the resnet**?\n\nSpecific example:\nWhen I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1. Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.\n\n\nMy guess is that if I use the imagenet weight each time, it will be very time-consuming.\nBut if I use the weight-1 as initial weight, the speed of training will be faster but it may stuck in local minimum.\n\nIf you know about this, can you give me some instructions? Thanks",
      "votes": null
    },
    {
      "id": "459650",
      "postDate": "01/22/2019 06:08:13",
      "content": "<p>You'd better start with the initial weight, otherwise you won't be able to tell whether your result change is due to augment or weight-1.</p>",
      "rawMarkdown": "You'd better start with the initial weight, otherwise you won't be able to tell whether your result change is due to augment or weight-1.",
      "votes": null
    },
    {
      "id": "459666",
      "postDate": "01/22/2019 06:38:13",
      "content": "<p>i would say yes, you can. to check it it's actually the augmentation and not the fine tuning, do the same training with the old aug as well and monitor the difference.</p>",
      "rawMarkdown": "i would say yes, you can. to check it it's actually the augmentation and not the fine tuning, do the same training with the old aug as well and monitor the difference.",
      "votes": null
    },
    {
      "id": "459706",
      "postDate": "01/22/2019 08:05:32",
      "content": "<p>Sorry, I'm just want to make sure I clarify the question.</p>\n\n<p>When I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1.  Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.</p>\n\n<p>Sorry I didn't understand your point that result change by weight-1. Did you mean use the weight-1 and imagenet weight as initial will affect the results? What I think is that for this case, it's only about the speed of training.</p>\n\n<p>Thanks for reply.</p>",
      "rawMarkdown": "Sorry, I'm just want to make sure I clarify the question.\n\nWhen I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1.  Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.\n\nSorry I didn't understand your point that result change by weight-1. Did you mean use the weight-1 and imagenet weight as initial will affect the results? What I think is that for this case, it's only about the speed of training.\n\nThanks for reply.",
      "votes": null
    },
    {
      "id": "459709",
      "postDate": "01/22/2019 08:12:14",
      "content": "<p>Hi, thanks for reply.\nDid you mean each time I can use the weight-1 as initial weight rather than imagenet (pretrained) weight to get a faster training time?</p>",
      "rawMarkdown": "Hi, thanks for reply.\nDid you mean each time I can use the weight-1 as initial weight rather than imagenet (pretrained) weight to get a faster training time?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 459650,
      "author_name": "zjuyang",
      "author_url": "",
      "post_date": "01/22/2019 06:08:13",
      "content": "<p>You'd better start with the initial weight, otherwise you won't be able to tell whether your result change is due to augment or weight-1.</p>",
      "votes": null,
      "replies": [
        {
          "id": 459706,
          "author_name": "kaka2nd",
          "author_url": "",
          "post_date": "01/22/2019 08:05:32",
          "content": "<p>Sorry, I'm just want to make sure I clarify the question.</p>\n\n<p>When I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1.  Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.</p>\n\n<p>Sorry I didn't understand your point that result change by weight-1. Did you mean use the weight-1 and imagenet weight as initial will affect the results? What I think is that for this case, it's only about the speed of training.</p>\n\n<p>Thanks for reply.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 459666,
      "author_name": "moshel",
      "author_url": "",
      "post_date": "01/22/2019 06:38:13",
      "content": "<p>i would say yes, you can. to check it it's actually the augmentation and not the fine tuning, do the same training with the old aug as well and monitor the difference.</p>",
      "votes": null,
      "replies": [
        {
          "id": 459709,
          "author_name": "kaka2nd",
          "author_url": "",
          "post_date": "01/22/2019 08:12:14",
          "content": "<p>Hi, thanks for reply.\nDid you mean each time I can use the weight-1 as initial weight rather than imagenet (pretrained) weight to get a faster training time?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "459604": "Hi all\n\nI am new in neural network and probably the question is late, but I still have some questions. The question is \n\nEg. I fine tuned my Resnet model and get the first effective weight-1. If I want to test the effect of different aug, should I **repeat the procedures of fine tune to get a new weight** or I can **use the weight-1 as the initial weight then train the resnet**?\n\nSpecific example:\nWhen I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1. Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.\n\n\nMy guess is that if I use the imagenet weight each time, it will be very time-consuming.\nBut if I use the weight-1 as initial weight, the speed of training will be faster but it may stuck in local minimum.\n\nIf you know about this, can you give me some instructions? Thanks",
    "459650": "You'd better start with the initial weight, otherwise you won't be able to tell whether your result change is due to augment or weight-1.",
    "459666": "i would say yes, you can. to check it it's actually the augmentation and not the fine tuning, do the same training with the old aug as well and monitor the difference.",
    "459706": "Sorry, I'm just want to make sure I clarify the question.\n\nWhen I want to test the effect of aug and use the Resnet models, I have two choices of initial weight.\n1. imagenet weight, fine tune to train the model. I think this is always workable but maybe time-consuming\n2. weight-1.  Trained weight for Resnet but without aug. And I use this as initial weight, which I guess this will be faster than using imagenet weight.\n\nSorry I didn't understand your point that result change by weight-1. Did you mean use the weight-1 and imagenet weight as initial will affect the results? What I think is that for this case, it's only about the speed of training.\n\nThanks for reply.",
    "459709": "Hi, thanks for reply.\nDid you mean each time I can use the weight-1 as initial weight rather than imagenet (pretrained) weight to get a faster training time?"
  },
  "source": "meta"
}