{
  "id": 174027,
  "title": "Which custom head to use for EffficientNet?",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/174027",
  "author_name": "",
  "post_date": "2020-08-11T21:37:12.686094300Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hi guys,</p>\n<p>if you are using EfficientNet, which kind of custom-head are you using?</p>\n<p>I am currently using a Dense Layer with LeakyRelu another Dropout, but that's it. As EffNet is quite powerful itself, it does not feel like we need a big head. <br>\nAny opinions?</p>\n<pre><code>def build_model(dim=128, ef=0):\n    inp = tf.keras.layers.Input(shape=(dim,dim,3))\n    base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)\n    x = base(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    x = tf.keras.layers.Dense(64,activation='linear')(x)\n    x = LeakyReLU(alpha=0.2)(x)\n    x = tf.keras.layers.Dropout(0.2)(x)\n    x = tf.keras.layers.Dense(1,activation='sigmoid')(x)\n    model = tf.keras.Model(inputs=inp,outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate=0.001)\n    loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05) \n    model.compile(optimizer=opt,loss=loss,metrics=['AUC'])\n    return model\n</code></pre>",
  "messages": [
    {
      "id": "967051",
      "postDate": "08/11/2020 21:37:12",
      "content": "<p>Hi guys,</p>\n<p>if you are using EfficientNet, which kind of custom-head are you using?</p>\n<p>I am currently using a Dense Layer with LeakyRelu another Dropout, but that's it. As EffNet is quite powerful itself, it does not feel like we need a big head. <br>\nAny opinions?</p>\n<pre><code>def build_model(dim=128, ef=0):\n    inp = tf.keras.layers.Input(shape=(dim,dim,3))\n    base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)\n    x = base(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    x = tf.keras.layers.Dense(64,activation='linear')(x)\n    x = LeakyReLU(alpha=0.2)(x)\n    x = tf.keras.layers.Dropout(0.2)(x)\n    x = tf.keras.layers.Dense(1,activation='sigmoid')(x)\n    model = tf.keras.Model(inputs=inp,outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate=0.001)\n    loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05) \n    model.compile(optimizer=opt,loss=loss,metrics=['AUC'])\n    return model\n</code></pre>",
      "rawMarkdown": "Hi guys,\n\nif you are using EfficientNet, which kind of custom-head are you using?\n\nI am currently using a Dense Layer with LeakyRelu another Dropout, but that's it. As EffNet is quite powerful itself, it does not feel like we need a big head. \nAny opinions?\n\n```\ndef build_model(dim=128, ef=0):\n    inp = tf.keras.layers.Input(shape=(dim,dim,3))\n    base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)\n    x = base(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    x = tf.keras.layers.Dense(64,activation='linear')(x)\n    x = LeakyReLU(alpha=0.2)(x)\n    x = tf.keras.layers.Dropout(0.2)(x)\n    x = tf.keras.layers.Dense(1,activation='sigmoid')(x)\n    model = tf.keras.Model(inputs=inp,outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate=0.001)\n    loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05) \n    model.compile(optimizer=opt,loss=loss,metrics=['AUC'])\n    return model\n```",
      "votes": null
    },
    {
      "id": "967272",
      "postDate": "08/12/2020 06:02:57",
      "content": "<p>For image only data I think what you are doing is much the same as most people.  I think some use a single dense layer and just take it to 1 .</p>",
      "rawMarkdown": "For image only data I think what you are doing is much the same as most people.  I think some use a single dense layer and just take it to 1 .",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 967272,
      "author_name": "brianfeeny",
      "author_url": "",
      "post_date": "08/12/2020 06:02:57",
      "content": "<p>For image only data I think what you are doing is much the same as most people.  I think some use a single dense layer and just take it to 1 .</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "967051": "Hi guys,\n\nif you are using EfficientNet, which kind of custom-head are you using?\n\nI am currently using a Dense Layer with LeakyRelu another Dropout, but that's it. As EffNet is quite powerful itself, it does not feel like we need a big head. \nAny opinions?\n\n```\ndef build_model(dim=128, ef=0):\n    inp = tf.keras.layers.Input(shape=(dim,dim,3))\n    base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)\n    x = base(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n    x = tf.keras.layers.Dense(64,activation='linear')(x)\n    x = LeakyReLU(alpha=0.2)(x)\n    x = tf.keras.layers.Dropout(0.2)(x)\n    x = tf.keras.layers.Dense(1,activation='sigmoid')(x)\n    model = tf.keras.Model(inputs=inp,outputs=x)\n    opt = tf.keras.optimizers.Adam(learning_rate=0.001)\n    loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05) \n    model.compile(optimizer=opt,loss=loss,metrics=['AUC'])\n    return model\n```",
    "967272": "For image only data I think what you are doing is much the same as most people.  I think some use a single dense layer and just take it to 1 ."
  },
  "source": "meta"
}