{
  "id": 174321,
  "title": "2 input with custom Generator facing this issue ",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/174321",
  "author_name": "",
  "post_date": "2020-08-13T06:29:39.255211700Z",
  "votes": 1,
  "comment_count": 7,
  "views": 0,
  "content": "<p>InvalidArgumentError:  assertion failed: [predictions must be &gt;= 0] [Condition x &gt;= y did not hold element-wise:] [x (aNetwork/Effnet0/Sigmoid:0) = ] [[nan][nan]] [y (Cast<em>14/x:0) = ] [0]\n     [[{{node assert</em>greater<em>equal/Assert/AssertGuard/else/</em>1/assert<em>greater</em>equal/Assert/AssertGuard/Assert}}]] [Op:_<em>inference</em>train<em>function</em>353274]</p>\n<p>Function call stack:<br>\ntrain_function</p>\n<p>please refer to code here : <a href=\"https://www.kaggle.com/kunduruanil/tf-img-table-effnetall\" target=\"_blank\">https://www.kaggle.com/kunduruanil/tf-img-table-effnetall</a></p>",
  "messages": [
    {
      "id": "968624",
      "postDate": "08/13/2020 06:29:39",
      "content": "<p>InvalidArgumentError:  assertion failed: [predictions must be &gt;= 0] [Condition x &gt;= y did not hold element-wise:] [x (aNetwork/Effnet0/Sigmoid:0) = ] [[nan][nan]] [y (Cast<em>14/x:0) = ] [0]\n     [[{{node assert</em>greater<em>equal/Assert/AssertGuard/else/</em>1/assert<em>greater</em>equal/Assert/AssertGuard/Assert}}]] [Op:_<em>inference</em>train<em>function</em>353274]</p>\n<p>Function call stack:<br>\ntrain_function</p>\n<p>please refer to code here : <a href=\"https://www.kaggle.com/kunduruanil/tf-img-table-effnetall\" target=\"_blank\">https://www.kaggle.com/kunduruanil/tf-img-table-effnetall</a></p>",
      "rawMarkdown": "InvalidArgumentError:  assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (aNetwork/Effnet0/Sigmoid:0) = ] [[nan][nan]] [y (Cast_14/x:0) = ] [0]\n\t [[{{node assert_greater_equal/Assert/AssertGuard/else/_1/assert_greater_equal/Assert/AssertGuard/Assert}}]] [Op:__inference_train_function_353274]\n\nFunction call stack:\ntrain_function\n\nplease refer to code here : https://www.kaggle.com/kunduruanil/tf-img-table-effnetall",
      "votes": null
    },
    {
      "id": "968630",
      "postDate": "08/13/2020 06:34:44",
      "content": "<p>Your script is not readable, so I will just guess. If you use AUC metric, make sure that your final output is sigmoid. If you try to evaluate AUC with TensorFlow and your don't output between 0 and 1, you get that error.</p>",
      "rawMarkdown": "Your script is not readable, so I will just guess. If you use AUC metric, make sure that your final output is sigmoid. If you try to evaluate AUC with TensorFlow and your don't output between 0 and 1, you get that error.",
      "votes": null
    },
    {
      "id": "968894",
      "postDate": "08/13/2020 10:21:26",
      "content": "<p>please find the model i am using :</p>\n<p>def get<em>model():\n    model</em>input = tf.keras.Input(shape=(*size, 3), name='imgIn')<br>\n    tab<em>input = tf.keras.Input(shape=(3,),name=\"tabIn\")\n    dummy = tf.keras.layers.Lambda(lambda x:x)(model</em>input)<br>\n    outputs = []    <br>\n    for i in range(8):<br>\n        constructor = getattr(efn, f'EfficientNetB{i}')</p>\n<pre><code>    x = constructor(include_top=False, weights='imagenet', \n                    input_shape=(*size, 3), \n                    pooling='avg')(dummy)\n    y = tf.keras.layers.Dense(100)(tab_input)\n    y = tf.keras.layers.BatchNormalization()(y)\n    y = tf.keras.layers.Activation(\"relu\")(y)\n    y = tf.keras.layers.Dropout(0.4)(y)\n    y = tf.keras.layers.Dense(100)(y)\n    y = tf.keras.layers.BatchNormalization()(y)\n    y = tf.keras.layers.Activation(\"relu\")(y)\n    y = tf.keras.layers.Dropout(0.4)(y)\n    concatenated = tf.keras.layers.concatenate([x, y], axis=-1)\n    con =  tf.keras.layers.Dense(100, activation='relu')(concatenated)\n    con = tf.keras.layers.BatchNormalization()(con)\n    con = tf.keras.layers.Activation(\"relu\")(con)\n    con = tf.keras.layers.Dropout(0.4)(con)\n    output = tf.keras.layers.Dense(1,name=f'Effnet{i}')(con)\n    output = tf.keras.layers.Activation(\"sigmoid\")(output)\n    outputs.append(output)\n\nmodel = tf.keras.Model([model_input,tab_input], outputs, name='aNetwork')\nmodel.compile(optimizer='adam',loss = tf.keras.losses.BinaryCrossentropy(\nlabel_smoothing = 0.05),metrics=[tf.keras.metrics.Accuracy(),tf.keras.metrics.AUC(name='auc')])\n#tf.keras.metrics.AUC(name='auc')\nreturn model\n</code></pre>\n<p>model = get_model()<br>\nmodel.summary()</p>\n<p>i am using activation as sigmoid at output layer.</p>",
      "rawMarkdown": "please find the model i am using :\n\ndef get_model():\n    model_input = tf.keras.Input(shape=(*size, 3), name='imgIn')\n    tab_input = tf.keras.Input(shape=(3,),name=\"tabIn\")\n    dummy = tf.keras.layers.Lambda(lambda x:x)(model_input)\n    outputs = []    \n    for i in range(8):\n        constructor = getattr(efn, f'EfficientNetB{i}')\n \n        x = constructor(include_top=False, weights='imagenet', \n                        input_shape=(*size, 3), \n                        pooling='avg')(dummy)\n        y = tf.keras.layers.Dense(100)(tab_input)\n        y = tf.keras.layers.BatchNormalization()(y)\n        y = tf.keras.layers.Activation(\"relu\")(y)\n        y = tf.keras.layers.Dropout(0.4)(y)\n        y = tf.keras.layers.Dense(100)(y)\n        y = tf.keras.layers.BatchNormalization()(y)\n        y = tf.keras.layers.Activation(\"relu\")(y)\n        y = tf.keras.layers.Dropout(0.4)(y)\n        concatenated = tf.keras.layers.concatenate([x, y], axis=-1)\n        con =  tf.keras.layers.Dense(100, activation='relu')(concatenated)\n        con = tf.keras.layers.BatchNormalization()(con)\n        con = tf.keras.layers.Activation(\"relu\")(con)\n        con = tf.keras.layers.Dropout(0.4)(con)\n        output = tf.keras.layers.Dense(1,name=f'Effnet{i}')(con)\n        output = tf.keras.layers.Activation(\"sigmoid\")(output)\n        outputs.append(output)\n \n    model = tf.keras.Model([model_input,tab_input], outputs, name='aNetwork')\n    model.compile(optimizer='adam',loss = tf.keras.losses.BinaryCrossentropy(\n    label_smoothing = 0.05),metrics=[tf.keras.metrics.Accuracy(),tf.keras.metrics.AUC(name='auc')])\n    #tf.keras.metrics.AUC(name='auc')\n    return model\nmodel = get_model()\nmodel.summary()\n\ni am using activation as sigmoid at output layer.",
      "votes": null
    },
    {
      "id": "968897",
      "postDate": "08/13/2020 10:23:21",
      "content": "<p>i created a custom Image Generator :</p>\n<p>class Mygenarator(tf.keras.utils.Sequence):</p>\n<pre><code>def __init__(self,df,td,x_col,y_col=None,batch_size=2,num_classes=None,size=(224,224,3),shuffle=True):\n    self.df = df\n    self.td = td\n    self.x_col = x_col\n    self.y_col = y_col\n    self.size = size\n    self.indices = df.index.tolist()\n    self.batch_size = batch_size\n    self.num_classes = num_classes\n    self.shuffle = shuffle\n    self.on_epoch_end()\n\ndef on_epoch_end(self):\n    self.index = np.arange(len(self.indices))\n    if self.shuffle == True:\n        np.random.shuffle(self.index)\n\ndef __len__(self):\n # Denotes the number of batches per epoch\n    return len(self.indices) // self.batch_size\n\n\ndef __getitem__(self, index):\n    # Generate one batch of data\n    # Generate indices of the batch\n    index = self.index[index * self.batch_size:(index + 1) * self.batch_size]\n    # Find list of IDs\n    batch = [self.indices[k] for k in index]\n    # Generate data\n    X, y = self.__get_data(batch)\n    return X, y\n\ndef __get_data(self, batch):\n    # X.shape : (batch_size, *dim)\n    # We can have multiple Xs and can return them as a list\n    X1 = np.empty((self.batch_size,*self.size))\n    X2 = np.empty((self.batch_size,3))\n    y = np.empty((self.batch_size), dtype=int)\n    # Generate data\n    for i, id in enumerate(batch):\n     # Store sample\n        X1[i,] = self.read_img(self.df.loc[id,self.x_col])\n        X2[i,] = self.td.loc[id,:].values\n        y[i] = self.df.loc[id,self.y_col]\n\n    return {\"imgIn\":X1,\"tabIn\":X2}, y\n\ndef hair_removal(self,image):\n  grayScale = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)\n  # kernel for morphologyEx\n  kernel = cv2.getStructuringElement(1,(17,17))\n  # apply MORPH_BLACKHAT to grayScale image\n  blackhat = cv2.morphologyEx(grayScale, cv2.MORPH_BLACKHAT, kernel)\n  # apply thresholding to blackhat\n  _,threshold = cv2.threshold(blackhat,10,255,cv2.THRESH_BINARY)\n\n  # inpaint with original image and threshold image\n  final_image = cv2.inpaint(image,threshold,1,cv2.INPAINT_TELEA)\n  final_image = cv2.cvtColor(final_image,cv2.COLOR_BGR2RGB)\n  return final_image\n\ndef read_img(self,file):\n  #image = cv2.imread(file)\n  #image = cv2.resize(image,self.size[:-1])\n  im = np.array(Image.open(file).resize(size))/255.0\n  #return self.hair_removal(image)\n  return im \n</code></pre>",
      "rawMarkdown": "i created a custom Image Generator :\n\nclass Mygenarator(tf.keras.utils.Sequence):\n    \n    def __init__(self,df,td,x_col,y_col=None,batch_size=2,num_classes=None,size=(224,224,3),shuffle=True):\n        self.df = df\n        self.td = td\n        self.x_col = x_col\n        self.y_col = y_col\n        self.size = size\n        self.indices = df.index.tolist()\n        self.batch_size = batch_size\n        self.num_classes = num_classes\n        self.shuffle = shuffle\n        self.on_epoch_end()\n        \n    def on_epoch_end(self):\n        self.index = np.arange(len(self.indices))\n        if self.shuffle == True:\n            np.random.shuffle(self.index)\n            \n    def __len__(self):\n     # Denotes the number of batches per epoch\n        return len(self.indices) // self.batch_size\n    \n    \n    def __getitem__(self, index):\n        # Generate one batch of data\n        # Generate indices of the batch\n        index = self.index[index * self.batch_size:(index + 1) * self.batch_size]\n        # Find list of IDs\n        batch = [self.indices[k] for k in index]\n        # Generate data\n        X, y = self.__get_data(batch)\n        return X, y\n    \n    def __get_data(self, batch):\n        # X.shape : (batch_size, *dim)\n        # We can have multiple Xs and can return them as a list\n        X1 = np.empty((self.batch_size,*self.size))\n        X2 = np.empty((self.batch_size,3))\n        y = np.empty((self.batch_size), dtype=int)\n        # Generate data\n        for i, id in enumerate(batch):\n         # Store sample\n            X1[i,] = self.read_img(self.df.loc[id,self.x_col])\n            X2[i,] = self.td.loc[id,:].values\n            y[i] = self.df.loc[id,self.y_col]\n            \n        return {\"imgIn\":X1,\"tabIn\":X2}, y\n    \n    def hair_removal(self,image):\n      grayScale = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)\n      # kernel for morphologyEx\n      kernel = cv2.getStructuringElement(1,(17,17))\n      # apply MORPH_BLACKHAT to grayScale image\n      blackhat = cv2.morphologyEx(grayScale, cv2.MORPH_BLACKHAT, kernel)\n      # apply thresholding to blackhat\n      _,threshold = cv2.threshold(blackhat,10,255,cv2.THRESH_BINARY)\n\n      # inpaint with original image and threshold image\n      final_image = cv2.inpaint(image,threshold,1,cv2.INPAINT_TELEA)\n      final_image = cv2.cvtColor(final_image,cv2.COLOR_BGR2RGB)\n      return final_image\n\n    def read_img(self,file):\n      #image = cv2.imread(file)\n      #image = cv2.resize(image,self.size[:-1])\n      im = np.array(Image.open(file).resize(size))/255.0\n      #return self.hair_removal(image)\n      return im",
      "votes": null
    },
    {
      "id": "968907",
      "postDate": "08/13/2020 10:32:53",
      "content": "<p>i created new version : <a href=\"https://www.kaggle.com/kunduruanil/tf-img-table-effnetall\" target=\"_blank\">https://www.kaggle.com/kunduruanil/tf-img-table-effnetall</a> </p>",
      "rawMarkdown": "i created new version : https://www.kaggle.com/kunduruanil/tf-img-table-effnetall",
      "votes": null
    },
    {
      "id": "969050",
      "postDate": "08/13/2020 12:34:44",
      "content": "<p>Also all your accuracies are 0 which is impossible unless you perfectly predict everything wrong. I think your model is outputting NaN. Not sure why. Perhaps your learning rate is too large.</p>\n<p>Try using a learning rate scheduler or use a lower LR as is <code>opt = tf.keras.optimizers.Adam(lr=0.00001)</code> then <code>model.compile(optimizer=opt, ...)</code>. The default LR for Adam is 0.001. That is too large.</p>",
      "rawMarkdown": "Also all your accuracies are 0 which is impossible unless you perfectly predict everything wrong. I think your model is outputting NaN. Not sure why. Perhaps your learning rate is too large.\n\nTry using a learning rate scheduler or use a lower LR as is `opt = tf.keras.optimizers.Adam(lr=0.00001)` then `model.compile(optimizer=opt, ...)`. The default LR for Adam is 0.001. That is too large.",
      "votes": null
    },
    {
      "id": "970351",
      "postDate": "08/14/2020 11:31:28",
      "content": "<p>i used tf.keras.optimizers.Adam(lr=0.00001) as optimizer still same error , so i just checked prediction with random this is the output , yeah you are correct!!!!!!  i am just wondering whats wrong with this model ?</p>\n<p>model.predict([np.random.randint(0,1,(2,224,224,3)),np.random.randint(0,2,(2,3))])</p>\n<p>[array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32)]</p>",
      "rawMarkdown": "i used tf.keras.optimizers.Adam(lr=0.00001) as optimizer still same error , so i just checked prediction with random this is the output , yeah you are correct!!!!!!  i am just wondering whats wrong with this model ?\n\nmodel.predict([np.random.randint(0,1,(2,224,224,3)),np.random.randint(0,2,(2,3))])\n\n[array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32)]",
      "votes": null
    },
    {
      "id": "970395",
      "postDate": "08/14/2020 12:17:13",
      "content": "<p>i tried adding this tab = tf.keras.layers.Lambda(lambda x:x)(tab_input) in line 4th line in get_model() function still same nan !!</p>",
      "rawMarkdown": "i tried adding this tab = tf.keras.layers.Lambda(lambda x:x)(tab_input) in line 4th line in get_model() function still same nan !!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 968630,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "08/13/2020 06:34:44",
      "content": "<p>Your script is not readable, so I will just guess. If you use AUC metric, make sure that your final output is sigmoid. If you try to evaluate AUC with TensorFlow and your don't output between 0 and 1, you get that error.</p>",
      "votes": null,
      "replies": [
        {
          "id": 968894,
          "author_name": "kunduruanil",
          "author_url": "",
          "post_date": "08/13/2020 10:21:26",
          "content": "<p>please find the model i am using :</p>\n<p>def get<em>model():\n    model</em>input = tf.keras.Input(shape=(*size, 3), name='imgIn')<br>\n    tab<em>input = tf.keras.Input(shape=(3,),name=\"tabIn\")\n    dummy = tf.keras.layers.Lambda(lambda x:x)(model</em>input)<br>\n    outputs = []    <br>\n    for i in range(8):<br>\n        constructor = getattr(efn, f'EfficientNetB{i}')</p>\n<pre><code>    x = constructor(include_top=False, weights='imagenet', \n                    input_shape=(*size, 3), \n                    pooling='avg')(dummy)\n    y = tf.keras.layers.Dense(100)(tab_input)\n    y = tf.keras.layers.BatchNormalization()(y)\n    y = tf.keras.layers.Activation(\"relu\")(y)\n    y = tf.keras.layers.Dropout(0.4)(y)\n    y = tf.keras.layers.Dense(100)(y)\n    y = tf.keras.layers.BatchNormalization()(y)\n    y = tf.keras.layers.Activation(\"relu\")(y)\n    y = tf.keras.layers.Dropout(0.4)(y)\n    concatenated = tf.keras.layers.concatenate([x, y], axis=-1)\n    con =  tf.keras.layers.Dense(100, activation='relu')(concatenated)\n    con = tf.keras.layers.BatchNormalization()(con)\n    con = tf.keras.layers.Activation(\"relu\")(con)\n    con = tf.keras.layers.Dropout(0.4)(con)\n    output = tf.keras.layers.Dense(1,name=f'Effnet{i}')(con)\n    output = tf.keras.layers.Activation(\"sigmoid\")(output)\n    outputs.append(output)\n\nmodel = tf.keras.Model([model_input,tab_input], outputs, name='aNetwork')\nmodel.compile(optimizer='adam',loss = tf.keras.losses.BinaryCrossentropy(\nlabel_smoothing = 0.05),metrics=[tf.keras.metrics.Accuracy(),tf.keras.metrics.AUC(name='auc')])\n#tf.keras.metrics.AUC(name='auc')\nreturn model\n</code></pre>\n<p>model = get_model()<br>\nmodel.summary()</p>\n<p>i am using activation as sigmoid at output layer.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 968897,
          "author_name": "kunduruanil",
          "author_url": "",
          "post_date": "08/13/2020 10:23:21",
          "content": "<p>i created a custom Image Generator :</p>\n<p>class Mygenarator(tf.keras.utils.Sequence):</p>\n<pre><code>def __init__(self,df,td,x_col,y_col=None,batch_size=2,num_classes=None,size=(224,224,3),shuffle=True):\n    self.df = df\n    self.td = td\n    self.x_col = x_col\n    self.y_col = y_col\n    self.size = size\n    self.indices = df.index.tolist()\n    self.batch_size = batch_size\n    self.num_classes = num_classes\n    self.shuffle = shuffle\n    self.on_epoch_end()\n\ndef on_epoch_end(self):\n    self.index = np.arange(len(self.indices))\n    if self.shuffle == True:\n        np.random.shuffle(self.index)\n\ndef __len__(self):\n # Denotes the number of batches per epoch\n    return len(self.indices) // self.batch_size\n\n\ndef __getitem__(self, index):\n    # Generate one batch of data\n    # Generate indices of the batch\n    index = self.index[index * self.batch_size:(index + 1) * self.batch_size]\n    # Find list of IDs\n    batch = [self.indices[k] for k in index]\n    # Generate data\n    X, y = self.__get_data(batch)\n    return X, y\n\ndef __get_data(self, batch):\n    # X.shape : (batch_size, *dim)\n    # We can have multiple Xs and can return them as a list\n    X1 = np.empty((self.batch_size,*self.size))\n    X2 = np.empty((self.batch_size,3))\n    y = np.empty((self.batch_size), dtype=int)\n    # Generate data\n    for i, id in enumerate(batch):\n     # Store sample\n        X1[i,] = self.read_img(self.df.loc[id,self.x_col])\n        X2[i,] = self.td.loc[id,:].values\n        y[i] = self.df.loc[id,self.y_col]\n\n    return {\"imgIn\":X1,\"tabIn\":X2}, y\n\ndef hair_removal(self,image):\n  grayScale = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)\n  # kernel for morphologyEx\n  kernel = cv2.getStructuringElement(1,(17,17))\n  # apply MORPH_BLACKHAT to grayScale image\n  blackhat = cv2.morphologyEx(grayScale, cv2.MORPH_BLACKHAT, kernel)\n  # apply thresholding to blackhat\n  _,threshold = cv2.threshold(blackhat,10,255,cv2.THRESH_BINARY)\n\n  # inpaint with original image and threshold image\n  final_image = cv2.inpaint(image,threshold,1,cv2.INPAINT_TELEA)\n  final_image = cv2.cvtColor(final_image,cv2.COLOR_BGR2RGB)\n  return final_image\n\ndef read_img(self,file):\n  #image = cv2.imread(file)\n  #image = cv2.resize(image,self.size[:-1])\n  im = np.array(Image.open(file).resize(size))/255.0\n  #return self.hair_removal(image)\n  return im \n</code></pre>",
          "votes": null,
          "replies": []
        },
        {
          "id": 968907,
          "author_name": "kunduruanil",
          "author_url": "",
          "post_date": "08/13/2020 10:32:53",
          "content": "<p>i created new version : <a href=\"https://www.kaggle.com/kunduruanil/tf-img-table-effnetall\" target=\"_blank\">https://www.kaggle.com/kunduruanil/tf-img-table-effnetall</a> </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 969050,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "08/13/2020 12:34:44",
          "content": "<p>Also all your accuracies are 0 which is impossible unless you perfectly predict everything wrong. I think your model is outputting NaN. Not sure why. Perhaps your learning rate is too large.</p>\n<p>Try using a learning rate scheduler or use a lower LR as is <code>opt = tf.keras.optimizers.Adam(lr=0.00001)</code> then <code>model.compile(optimizer=opt, ...)</code>. The default LR for Adam is 0.001. That is too large.</p>",
          "votes": null,
          "replies": [
            {
              "id": 970351,
              "author_name": "kunduruanil",
              "author_url": "",
              "post_date": "08/14/2020 11:31:28",
              "content": "<p>i used tf.keras.optimizers.Adam(lr=0.00001) as optimizer still same error , so i just checked prediction with random this is the output , yeah you are correct!!!!!!  i am just wondering whats wrong with this model ?</p>\n<p>model.predict([np.random.randint(0,1,(2,224,224,3)),np.random.randint(0,2,(2,3))])</p>\n<p>[array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32), array([[nan],<br>\n        [nan]], dtype=float32)]</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 970395,
              "author_name": "kunduruanil",
              "author_url": "",
              "post_date": "08/14/2020 12:17:13",
              "content": "<p>i tried adding this tab = tf.keras.layers.Lambda(lambda x:x)(tab_input) in line 4th line in get_model() function still same nan !!</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "968624": "InvalidArgumentError:  assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (aNetwork/Effnet0/Sigmoid:0) = ] [[nan][nan]] [y (Cast_14/x:0) = ] [0]\n\t [[{{node assert_greater_equal/Assert/AssertGuard/else/_1/assert_greater_equal/Assert/AssertGuard/Assert}}]] [Op:__inference_train_function_353274]\n\nFunction call stack:\ntrain_function\n\nplease refer to code here : https://www.kaggle.com/kunduruanil/tf-img-table-effnetall",
    "968630": "Your script is not readable, so I will just guess. If you use AUC metric, make sure that your final output is sigmoid. If you try to evaluate AUC with TensorFlow and your don't output between 0 and 1, you get that error.",
    "968894": "please find the model i am using :\n\ndef get_model():\n    model_input = tf.keras.Input(shape=(*size, 3), name='imgIn')\n    tab_input = tf.keras.Input(shape=(3,),name=\"tabIn\")\n    dummy = tf.keras.layers.Lambda(lambda x:x)(model_input)\n    outputs = []    \n    for i in range(8):\n        constructor = getattr(efn, f'EfficientNetB{i}')\n \n        x = constructor(include_top=False, weights='imagenet', \n                        input_shape=(*size, 3), \n                        pooling='avg')(dummy)\n        y = tf.keras.layers.Dense(100)(tab_input)\n        y = tf.keras.layers.BatchNormalization()(y)\n        y = tf.keras.layers.Activation(\"relu\")(y)\n        y = tf.keras.layers.Dropout(0.4)(y)\n        y = tf.keras.layers.Dense(100)(y)\n        y = tf.keras.layers.BatchNormalization()(y)\n        y = tf.keras.layers.Activation(\"relu\")(y)\n        y = tf.keras.layers.Dropout(0.4)(y)\n        concatenated = tf.keras.layers.concatenate([x, y], axis=-1)\n        con =  tf.keras.layers.Dense(100, activation='relu')(concatenated)\n        con = tf.keras.layers.BatchNormalization()(con)\n        con = tf.keras.layers.Activation(\"relu\")(con)\n        con = tf.keras.layers.Dropout(0.4)(con)\n        output = tf.keras.layers.Dense(1,name=f'Effnet{i}')(con)\n        output = tf.keras.layers.Activation(\"sigmoid\")(output)\n        outputs.append(output)\n \n    model = tf.keras.Model([model_input,tab_input], outputs, name='aNetwork')\n    model.compile(optimizer='adam',loss = tf.keras.losses.BinaryCrossentropy(\n    label_smoothing = 0.05),metrics=[tf.keras.metrics.Accuracy(),tf.keras.metrics.AUC(name='auc')])\n    #tf.keras.metrics.AUC(name='auc')\n    return model\nmodel = get_model()\nmodel.summary()\n\ni am using activation as sigmoid at output layer.",
    "968897": "i created a custom Image Generator :\n\nclass Mygenarator(tf.keras.utils.Sequence):\n    \n    def __init__(self,df,td,x_col,y_col=None,batch_size=2,num_classes=None,size=(224,224,3),shuffle=True):\n        self.df = df\n        self.td = td\n        self.x_col = x_col\n        self.y_col = y_col\n        self.size = size\n        self.indices = df.index.tolist()\n        self.batch_size = batch_size\n        self.num_classes = num_classes\n        self.shuffle = shuffle\n        self.on_epoch_end()\n        \n    def on_epoch_end(self):\n        self.index = np.arange(len(self.indices))\n        if self.shuffle == True:\n            np.random.shuffle(self.index)\n            \n    def __len__(self):\n     # Denotes the number of batches per epoch\n        return len(self.indices) // self.batch_size\n    \n    \n    def __getitem__(self, index):\n        # Generate one batch of data\n        # Generate indices of the batch\n        index = self.index[index * self.batch_size:(index + 1) * self.batch_size]\n        # Find list of IDs\n        batch = [self.indices[k] for k in index]\n        # Generate data\n        X, y = self.__get_data(batch)\n        return X, y\n    \n    def __get_data(self, batch):\n        # X.shape : (batch_size, *dim)\n        # We can have multiple Xs and can return them as a list\n        X1 = np.empty((self.batch_size,*self.size))\n        X2 = np.empty((self.batch_size,3))\n        y = np.empty((self.batch_size), dtype=int)\n        # Generate data\n        for i, id in enumerate(batch):\n         # Store sample\n            X1[i,] = self.read_img(self.df.loc[id,self.x_col])\n            X2[i,] = self.td.loc[id,:].values\n            y[i] = self.df.loc[id,self.y_col]\n            \n        return {\"imgIn\":X1,\"tabIn\":X2}, y\n    \n    def hair_removal(self,image):\n      grayScale = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)\n      # kernel for morphologyEx\n      kernel = cv2.getStructuringElement(1,(17,17))\n      # apply MORPH_BLACKHAT to grayScale image\n      blackhat = cv2.morphologyEx(grayScale, cv2.MORPH_BLACKHAT, kernel)\n      # apply thresholding to blackhat\n      _,threshold = cv2.threshold(blackhat,10,255,cv2.THRESH_BINARY)\n\n      # inpaint with original image and threshold image\n      final_image = cv2.inpaint(image,threshold,1,cv2.INPAINT_TELEA)\n      final_image = cv2.cvtColor(final_image,cv2.COLOR_BGR2RGB)\n      return final_image\n\n    def read_img(self,file):\n      #image = cv2.imread(file)\n      #image = cv2.resize(image,self.size[:-1])\n      im = np.array(Image.open(file).resize(size))/255.0\n      #return self.hair_removal(image)\n      return im",
    "968907": "i created new version : https://www.kaggle.com/kunduruanil/tf-img-table-effnetall",
    "969050": "Also all your accuracies are 0 which is impossible unless you perfectly predict everything wrong. I think your model is outputting NaN. Not sure why. Perhaps your learning rate is too large.\n\nTry using a learning rate scheduler or use a lower LR as is `opt = tf.keras.optimizers.Adam(lr=0.00001)` then `model.compile(optimizer=opt, ...)`. The default LR for Adam is 0.001. That is too large.",
    "970351": "i used tf.keras.optimizers.Adam(lr=0.00001) as optimizer still same error , so i just checked prediction with random this is the output , yeah you are correct!!!!!!  i am just wondering whats wrong with this model ?\n\nmodel.predict([np.random.randint(0,1,(2,224,224,3)),np.random.randint(0,2,(2,3))])\n\n[array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32), array([[nan],\n        [nan]], dtype=float32)]",
    "970395": "i tried adding this tab = tf.keras.layers.Lambda(lambda x:x)(tab_input) in line 4th line in get_model() function still same nan !!"
  },
  "source": "meta"
}