{
  "id": 253125,
  "title": "Larger network helps?",
  "url": "/competitions/google-smartphone-decimeter-challenge/discussion/253125",
  "author_name": "Yogesh Haribhau Kulkarni",
  "post_date": "2021-07-15T05:38:57.028000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>For Google Smartphone Decimeter challenge, started with model as defined by <a href=\"https://www.kaggle.com/jeongyoonlee\" target=\"_blank\">@jeongyoonlee</a>.<br>\nAdding layers to it was only a bit helpful and not a whole lot.</p>\n<p>Also lower lr and batch-size also has marginal effect.</p>\n<pre><code>lrate = 0.01 #0.001\nbatch_size = 128 #1024\nepochs = 2000 #100\n</code></pre>\n<p>My current model is :</p>\n<pre><code>    inputs = keras.layers.Input((input_dim,))\n    x = keras.layers.Dense(128, activation='relu')(inputs)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n\n    ox = x\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n\n    x1 = x\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n\n    x2 = x\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n\n    x = keras.layers.Add()([x, x2, x1, ox])\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n\n    outputs = keras.layers.Dense(output_dim, activation='linear')(x)\n</code></pre>\n<p>Any specific ideas on which specific parameters or metrics can suggest number of layers, for better accuracy?</p>",
  "messages": [
    {
      "id": 1388616,
      "postDate": "2021-07-15T05:38:57.030Z",
      "content": "<p>For Google Smartphone Decimeter challenge, started with model as defined by <a href=\"https://www.kaggle.com/jeongyoonlee\" target=\"_blank\">@jeongyoonlee</a>.<br>\nAdding layers to it was only a bit helpful and not a whole lot.</p>\n<p>Also lower lr and batch-size also has marginal effect.</p>\n<pre><code>lrate = 0.01 #0.001\nbatch_size = 128 #1024\nepochs = 2000 #100\n</code></pre>\n<p>My current model is :</p>\n<pre><code>    inputs = keras.layers.Input((input_dim,))\n    x = keras.layers.Dense(128, activation='relu')(inputs)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n\n    ox = x\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n\n    x1 = x\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n\n    x2 = x\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n\n    x = keras.layers.Add()([x, x2, x1, ox])\n\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n\n    outputs = keras.layers.Dense(output_dim, activation='linear')(x)\n</code></pre>\n<p>Any specific ideas on which specific parameters or metrics can suggest number of layers, for better accuracy?</p>",
      "rawMarkdown": "For Google Smartphone Decimeter challenge, started with model as defined by @jeongyoonlee.\nAdding layers to it was only a bit helpful and not a whole lot.\n\nAlso lower lr and batch-size also has marginal effect.\n\n```\nlrate = 0.01 #0.001\nbatch_size = 128 #1024\nepochs = 2000 #100\n```\n\nMy current model is :\n\n```\n    inputs = keras.layers.Input((input_dim,))\n    x = keras.layers.Dense(128, activation='relu')(inputs)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n    \n    ox = x\n    \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n    \n    x1 = x\n    \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n    \n    x2 = x\n        \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n    \n    x = keras.layers.Add()([x, x2, x1, ox])\n    \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n    \n    outputs = keras.layers.Dense(output_dim, activation='linear')(x)\n\n```\n\nAny specific ideas on which specific parameters or metrics can suggest number of layers, for better accuracy?",
      "votes": 7
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1388616": "For Google Smartphone Decimeter challenge, started with model as defined by @jeongyoonlee.\nAdding layers to it was only a bit helpful and not a whole lot.\n\nAlso lower lr and batch-size also has marginal effect.\n\n```\nlrate = 0.01 #0.001\nbatch_size = 128 #1024\nepochs = 2000 #100\n```\n\nMy current model is :\n\n```\n    inputs = keras.layers.Input((input_dim,))\n    x = keras.layers.Dense(128, activation='relu')(inputs)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n    \n    ox = x\n    \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n    \n    x1 = x\n    \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n    \n    x2 = x\n        \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)    \n    \n    x = keras.layers.Add()([x, x2, x1, ox])\n    \n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.BatchNormalization()(x)\n    x = keras.layers.Dense(128, activation='relu')(x)\n    x = keras.layers.Dropout(.3)(x)\n    \n    outputs = keras.layers.Dense(output_dim, activation='linear')(x)\n\n```\n\nAny specific ideas on which specific parameters or metrics can suggest number of layers, for better accuracy?"
  }
}