{
  "id": 410969,
  "title": "training problem with UNet",
  "url": "/competitions/vesuvius-challenge-ink-detection/discussion/410969",
  "author_name": "",
  "post_date": "2023-05-17T07:35:00.755610Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I wonder why a simple UNet does not work(recall and precision drops to zero). I am a new Kaggler, waiting for help sincerely.</p>\n<pre><code>def unet(pretrained_weights = None,input_size = (128,128,48)):\n    inputs = Input(input_size)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)\n    drop4 = Dropout(0.5)(conv4)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)\n\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)\n    drop5 = Dropout(0.5)(conv5)\n\n    up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))\n    merge6 = concatenate([drop4,up6], axis = 3)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)\n\n    up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))\n    merge7 = concatenate([conv3,up7], axis = 3)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)\n\n    up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))\n    merge8 = concatenate([conv2,up8], axis = 3)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)\n\n    up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))\n    merge9 = concatenate([conv1,up9], axis = 3)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)\n\n    model = Model(inputs=inputs, outputs = conv10)\n    clr = tfa.optimizers.CyclicalLearningRate(initial_learning_rate=0.1,\n    maximal_learning_rate=0.1,\n    scale_fn=lambda x: 1/(2.**(x-1)),\n    step_size=2 * 100)\n    sgd = optimizers.SGD(clr)\n    def recall(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        possible_positives = keras.sum(keras.round(keras.clip(y_true, 0, 1)))\n        recall = true_positives / (possible_positives + keras.epsilon())\n        return recall\n\n    def precision(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        predicted_positives = keras.sum(keras.round(keras.clip(y_pred, 0, 1)))\n        precision = true_positives / (predicted_positives + keras.epsilon())\n        return precision\n\n    def fbeta_score(y_true, y_pred, beta=0.5):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        # Calculates the F score, the weighted harmonic mean of precision and recall.\n        if beta &lt; 0:\n            raise ValueError('The lowest choosable beta is zero (only precision).')\n        # If there are no true positives, fix the F score at 0 like sklearn.\n        if keras.sum(keras.round(keras.clip(y_true, 0, 1))) == 0:\n            return 0.\n\n        p = precision(y_true, y_pred)\n        r = recall(y_true, y_pred)\n        bb = beta ** 2\n        fbeta_score = (1 + bb) * (p * r) / (bb * p + r + keras.epsilon())\n        return fbeta_score\n\n    model.compile(optimizer = sgd, loss = 'binary_crossentropy', metrics = ['accuracy', fbeta_score, precision, recall])\n\n    #model.summary()\n\n    if(pretrained_weights):\n        model.load_weights(pretrained_weights)\n\n    return model\n</code></pre>",
  "messages": [
    {
      "id": "2262895",
      "postDate": "05/17/2023 07:35:00",
      "content": "<p>I wonder why a simple UNet does not work(recall and precision drops to zero). I am a new Kaggler, waiting for help sincerely.</p>\n<pre><code>def unet(pretrained_weights = None,input_size = (128,128,48)):\n    inputs = Input(input_size)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)\n    drop4 = Dropout(0.5)(conv4)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)\n\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)\n    drop5 = Dropout(0.5)(conv5)\n\n    up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))\n    merge6 = concatenate([drop4,up6], axis = 3)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)\n\n    up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))\n    merge7 = concatenate([conv3,up7], axis = 3)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)\n\n    up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))\n    merge8 = concatenate([conv2,up8], axis = 3)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)\n\n    up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))\n    merge9 = concatenate([conv1,up9], axis = 3)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)\n\n    model = Model(inputs=inputs, outputs = conv10)\n    clr = tfa.optimizers.CyclicalLearningRate(initial_learning_rate=0.1,\n    maximal_learning_rate=0.1,\n    scale_fn=lambda x: 1/(2.**(x-1)),\n    step_size=2 * 100)\n    sgd = optimizers.SGD(clr)\n    def recall(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        possible_positives = keras.sum(keras.round(keras.clip(y_true, 0, 1)))\n        recall = true_positives / (possible_positives + keras.epsilon())\n        return recall\n\n    def precision(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        predicted_positives = keras.sum(keras.round(keras.clip(y_pred, 0, 1)))\n        precision = true_positives / (predicted_positives + keras.epsilon())\n        return precision\n\n    def fbeta_score(y_true, y_pred, beta=0.5):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        # Calculates the F score, the weighted harmonic mean of precision and recall.\n        if beta &lt; 0:\n            raise ValueError('The lowest choosable beta is zero (only precision).')\n        # If there are no true positives, fix the F score at 0 like sklearn.\n        if keras.sum(keras.round(keras.clip(y_true, 0, 1))) == 0:\n            return 0.\n\n        p = precision(y_true, y_pred)\n        r = recall(y_true, y_pred)\n        bb = beta ** 2\n        fbeta_score = (1 + bb) * (p * r) / (bb * p + r + keras.epsilon())\n        return fbeta_score\n\n    model.compile(optimizer = sgd, loss = 'binary_crossentropy', metrics = ['accuracy', fbeta_score, precision, recall])\n\n    #model.summary()\n\n    if(pretrained_weights):\n        model.load_weights(pretrained_weights)\n\n    return model\n</code></pre>",
      "rawMarkdown": "I wonder why a simple UNet does not work(recall and precision drops to zero). I am a new Kaggler, waiting for help sincerely.\n\n```\ndef unet(pretrained_weights = None,input_size = (128,128,48)):\n    inputs = Input(input_size)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)\n    drop4 = Dropout(0.5)(conv4)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)\n\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)\n    drop5 = Dropout(0.5)(conv5)\n\n    up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))\n    merge6 = concatenate([drop4,up6], axis = 3)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)\n\n    up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))\n    merge7 = concatenate([conv3,up7], axis = 3)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)\n\n    up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))\n    merge8 = concatenate([conv2,up8], axis = 3)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)\n\n    up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))\n    merge9 = concatenate([conv1,up9], axis = 3)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)\n\n    model = Model(inputs=inputs, outputs = conv10)\n    clr = tfa.optimizers.CyclicalLearningRate(initial_learning_rate=0.1,\n    maximal_learning_rate=0.1,\n    scale_fn=lambda x: 1/(2.**(x-1)),\n    step_size=2 * 100)\n    sgd = optimizers.SGD(clr)\n    def recall(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        possible_positives = keras.sum(keras.round(keras.clip(y_true, 0, 1)))\n        recall = true_positives / (possible_positives + keras.epsilon())\n        return recall\n    \n    def precision(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        predicted_positives = keras.sum(keras.round(keras.clip(y_pred, 0, 1)))\n        precision = true_positives / (predicted_positives + keras.epsilon())\n        return precision\n\n    def fbeta_score(y_true, y_pred, beta=0.5):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        # Calculates the F score, the weighted harmonic mean of precision and recall.\n        if beta < 0:\n            raise ValueError('The lowest choosable beta is zero (only precision).')\n        # If there are no true positives, fix the F score at 0 like sklearn.\n        if keras.sum(keras.round(keras.clip(y_true, 0, 1))) == 0:\n            return 0.\n\n        p = precision(y_true, y_pred)\n        r = recall(y_true, y_pred)\n        bb = beta ** 2\n        fbeta_score = (1 + bb) * (p * r) / (bb * p + r + keras.epsilon())\n        return fbeta_score\n        \n    model.compile(optimizer = sgd, loss = 'binary_crossentropy', metrics = ['accuracy', fbeta_score, precision, recall])\n    \n    #model.summary()\n\n    if(pretrained_weights):\n        model.load_weights(pretrained_weights)\n\n    return model\n\n```",
      "votes": null
    },
    {
      "id": "2263084",
      "postDate": "05/17/2023 10:20:20",
      "content": "<p>It could be due to couple of things. <br>\nHow do your output probabilities look like? <br>\nAlso, is your input data properly normalized?<br>\nI see that you don't have batch norm layers implemented, it could also be due to vanishing/exploding gradient problem.. </p>",
      "rawMarkdown": "It could be due to couple of things. \nHow do your output probabilities look like? \nAlso, is your input data properly normalized?\nI see that you don't have batch norm layers implemented, it could also be due to vanishing/exploding gradient problem..",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2263084,
      "author_name": "viktorcikojevic",
      "author_url": "",
      "post_date": "05/17/2023 10:20:20",
      "content": "<p>It could be due to couple of things. <br>\nHow do your output probabilities look like? <br>\nAlso, is your input data properly normalized?<br>\nI see that you don't have batch norm layers implemented, it could also be due to vanishing/exploding gradient problem.. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2262895": "I wonder why a simple UNet does not work(recall and precision drops to zero). I am a new Kaggler, waiting for help sincerely.\n\n```\ndef unet(pretrained_weights = None,input_size = (128,128,48)):\n    inputs = Input(input_size)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)\n    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)\n    conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)\n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)\n    conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)\n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)\n    conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)\n    drop4 = Dropout(0.5)(conv4)\n    pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)\n\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)\n    conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)\n    drop5 = Dropout(0.5)(conv5)\n\n    up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))\n    merge6 = concatenate([drop4,up6], axis = 3)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)\n    conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)\n\n    up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))\n    merge7 = concatenate([conv3,up7], axis = 3)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)\n    conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)\n\n    up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))\n    merge8 = concatenate([conv2,up8], axis = 3)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)\n    conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)\n\n    up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))\n    merge9 = concatenate([conv1,up9], axis = 3)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)\n    conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)\n    conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)\n\n    model = Model(inputs=inputs, outputs = conv10)\n    clr = tfa.optimizers.CyclicalLearningRate(initial_learning_rate=0.1,\n    maximal_learning_rate=0.1,\n    scale_fn=lambda x: 1/(2.**(x-1)),\n    step_size=2 * 100)\n    sgd = optimizers.SGD(clr)\n    def recall(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        possible_positives = keras.sum(keras.round(keras.clip(y_true, 0, 1)))\n        recall = true_positives / (possible_positives + keras.epsilon())\n        return recall\n    \n    def precision(y_true, y_pred):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        true_positives = keras.sum(keras.round(keras.clip(y_true * y_pred, 0, 1)))\n        predicted_positives = keras.sum(keras.round(keras.clip(y_pred, 0, 1)))\n        precision = true_positives / (predicted_positives + keras.epsilon())\n        return precision\n\n    def fbeta_score(y_true, y_pred, beta=0.5):\n        y_pred = tf.cast(tf.greater(y_pred,0.6),tf.float32)\n        # Calculates the F score, the weighted harmonic mean of precision and recall.\n        if beta < 0:\n            raise ValueError('The lowest choosable beta is zero (only precision).')\n        # If there are no true positives, fix the F score at 0 like sklearn.\n        if keras.sum(keras.round(keras.clip(y_true, 0, 1))) == 0:\n            return 0.\n\n        p = precision(y_true, y_pred)\n        r = recall(y_true, y_pred)\n        bb = beta ** 2\n        fbeta_score = (1 + bb) * (p * r) / (bb * p + r + keras.epsilon())\n        return fbeta_score\n        \n    model.compile(optimizer = sgd, loss = 'binary_crossentropy', metrics = ['accuracy', fbeta_score, precision, recall])\n    \n    #model.summary()\n\n    if(pretrained_weights):\n        model.load_weights(pretrained_weights)\n\n    return model\n\n```",
    "2263084": "It could be due to couple of things. \nHow do your output probabilities look like? \nAlso, is your input data properly normalized?\nI see that you don't have batch norm layers implemented, it could also be due to vanishing/exploding gradient problem.."
  },
  "source": "meta"
}