{ "id": 57136, "title": "NAN in loss of Keras NN? help!", "url": "/competitions/avito-demand-prediction/discussion/57136", "author_name": "AmirH", "post_date": "2018-05-19T19:33:14.652000", "votes": 7, "comment_count": 18, "views": 0, "content": "

Hi guys and gals,\nI'm trying to run a keras NN on the data and my loss and metric data is NAN:

\n\n

This is my keras code (In R), Any ideas?\n(i know i should use RMSE and not MSE, i'm just testing things out, as RMSE requires a side function)\nThanks!!!

\n\n

library(\"keras\")

\n\n

model <- keras_model_sequential()

\n\n

model %>%

\n\n

layer_dense(units=43,activation=\"relu\",input_shape=c(43)) %>%

\n\n

layer_dropout(rate=0.2) %>%

\n\n

layer_dense(units = 22,activation = \"relu\") %>%

\n\n

layer_dropout(rate=0.2) %>%

\n\n

layer_dense(units=1,activation=\"relu\")

\n\n

summary(model)

\n\n

model %>%\n compile(loss =\"mean_squared_error\",\n optimizer = \"adam\",\n metrics= c(\"mse\"))

\n\n

history<-model %>% fit(TrainData, DealProbs, epochs = 1,\n callbacks = callback_tensorboard(log_dir = \"logs/run_b\"),\n validation_split = 0.2) #train on 80% of train set and will evaluate

\n\n

The result (I ran it on small part of the data to test)\nEpoch 1/1\n80000/80000 [==============================] - 5s 60us/step - loss: nan - mean_squared_error: nan - val_loss: nan - val_mean_squared_error: nan

", "messages": [ { "id": 539860, "postDate": "2019-05-30T16:07:36.577Z", "content": "

It is not because of having missing values in the data. It is because the activation function is out of range for handling the input weight vector. Change the activation to softmax in the output layer and try,

", "rawMarkdown": "It is not because of having missing values in the data. It is because the activation function is out of range for handling the input weight vector. Change the activation to softmax in the output layer and try,", "votes": 13, "replies": [ { "id": 665795, "postDate": "2019-11-05T12:06:49.213Z", "content": "

Your answer solve my issue. Thanks

", "rawMarkdown": "Your answer solve my issue. Thanks" }, { "id": 808077, "postDate": "2020-04-15T06:07:06.347Z", "content": "

thank you, it solved my issue too, but in my case sigmoid seems to be more suitable

", "rawMarkdown": "thank you, it solved my issue too, but in my case sigmoid seems to be more suitable" } ] }, { "id": 330915, "postDate": "2018-05-20T02:55:56.597Z", "content": "

I'v faced this problem before. Its definitely NaN's in the input. Printing out DataFrame.isnull().any() will do the trick.

", "rawMarkdown": "I'v faced this problem before. Its definitely NaN's in the input. Printing out DataFrame.isnull().any() will do the trick.", "votes": 13 }, { "id": 330820, "postDate": "2018-05-19T19:33:14.653Z", "content": "

Hi guys and gals,\nI'm trying to run a keras NN on the data and my loss and metric data is NAN:

\n\n

This is my keras code (In R), Any ideas?\n(i know i should use RMSE and not MSE, i'm just testing things out, as RMSE requires a side function)\nThanks!!!

\n\n

library(\"keras\")

\n\n

model <- keras_model_sequential()

\n\n

model %>%

\n\n

layer_dense(units=43,activation=\"relu\",input_shape=c(43)) %>%

\n\n

layer_dropout(rate=0.2) %>%

\n\n

layer_dense(units = 22,activation = \"relu\") %>%

\n\n

layer_dropout(rate=0.2) %>%

\n\n

layer_dense(units=1,activation=\"relu\")

\n\n

summary(model)

\n\n

model %>%\n compile(loss =\"mean_squared_error\",\n optimizer = \"adam\",\n metrics= c(\"mse\"))

\n\n

history<-model %>% fit(TrainData, DealProbs, epochs = 1,\n callbacks = callback_tensorboard(log_dir = \"logs/run_b\"),\n validation_split = 0.2) #train on 80% of train set and will evaluate

\n\n

The result (I ran it on small part of the data to test)\nEpoch 1/1\n80000/80000 [==============================] - 5s 60us/step - loss: nan - mean_squared_error: nan - val_loss: nan - val_mean_squared_error: nan

", "rawMarkdown": "Hi guys and gals,\nI'm trying to run a keras NN on the data and my loss and metric data is NAN:\n\nThis is my keras code (In R), Any ideas?\n(i know i should use RMSE and not MSE, i'm just testing things out, as RMSE requires a side function)\nThanks!!!\n\n\nlibrary(\"keras\")\n\nmodel <- keras_model_sequential()\n\nmodel %>% \n\n layer_dense(units=43,activation=\"relu\",input_shape=c(43)) %>%\n\n layer_dropout(rate=0.2) %>%\n\n layer_dense(units = 22,activation = \"relu\") %>%\n\n layer_dropout(rate=0.2) %>%\n \n\n layer_dense(units=1,activation=\"relu\") \n\nsummary(model)\n\nmodel %>%\n compile(loss =\"mean_squared_error\",\n optimizer = \"adam\",\n metrics= c(\"mse\"))\n\nhistory<-model %>% fit(TrainData, DealProbs, epochs = 1,\n callbacks = callback_tensorboard(log_dir = \"logs/run_b\"),\n validation_split = 0.2) #train on 80% of train set and will evaluate \n\nThe result (I ran it on small part of the data to test)\nEpoch 1/1\n80000/80000 [==============================] - 5s 60us/step - loss: nan - mean_squared_error: nan - val_loss: nan - val_mean_squared_error: nan", "votes": 7 }, { "id": 330824, "postDate": "2018-05-19T19:37:27.707Z", "content": "

You for sure have some nans in your input. Check your continuous variables, especially the ones you've FE'd (if you do FE for your NNet?) and make sure there aren't any NANs in there.

", "rawMarkdown": "You for sure have some nans in your input. Check your continuous variables, especially the ones you've FE'd (if you do FE for your NNet?) and make sure there aren't any NANs in there.", "votes": 5, "replies": [ { "id": 330825, "postDate": "2018-05-19T19:39:00.443Z", "content": "

Thanks my friend, i will check that out!\nI handled only NAs, not NANs

", "rawMarkdown": "Thanks my friend, i will check that out!\nI handled only NAs, not NANs" }, { "id": 580923, "postDate": "2019-07-21T04:32:21.853Z", "content": "

Handling nan in input solved the issue... Thanks!

", "rawMarkdown": "Handling nan in input solved the issue... Thanks!", "votes": 3 }, { "id": 1616331, "postDate": "2021-12-13T10:52:42.493Z", "content": "

Solved my problem. Had issues with NaNs.

", "rawMarkdown": "Solved my problem. Had issues with NaNs." } ] }, { "id": 1495197, "postDate": "2021-08-29T11:36:22.047Z", "content": "

I had the nan loss problem in image data, I was using np.empty for generating the batches of images and looks like it was the reason of the nan loss, changed np.empty to np.zeros and problem solved.

", "rawMarkdown": "I had the nan loss problem in image data, I was using **np.empty** for generating the batches of images and looks like it was the reason of the nan loss, changed **np.empty** to **np.zeros** and problem solved." }, { "id": 959399, "postDate": "2020-08-05T14:53:11.320Z", "content": "

Guys even i tried making a an ANN with keras .....i had double checked but there r no NaN's in my input this is my code...model = tf.keras.Sequential([\n tf.keras.layers.Dense(200, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(125, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(625, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(100, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(175, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(750, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(600, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(25, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(2, activation='softmax'),\n tf.keras.layers.BatchNormalization(),\nIn my compilation i made sure loss was binary crossentropy loss becuz i hv probability distributions....but still i get loss alone as nan while accuracy as 0.2724 like here....in my first epoch....Epoch 1/20\n15/15 [==============================] - 0s 17ms/step - loss: 'nan' - accuracy: 0.3841\nHow can i fix this issue??

", "rawMarkdown": "Guys even i tried making a an ANN with keras .....i had double checked but there r no NaN's in my input this is my code...model = tf.keras.Sequential([\n tf.keras.layers.Dense(200, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(125, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(625, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(100, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(175, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(750, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(600, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(25, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(2, activation='softmax'),\n tf.keras.layers.BatchNormalization(),\nIn my compilation i made sure loss was binary crossentropy loss becuz i hv probability distributions....but still i get loss alone as nan while accuracy as 0.2724 like here....in my first epoch....Epoch 1/20\n15/15 [==============================] - 0s 17ms/step - loss: 'nan' - accuracy: 0.3841\nHow can i fix this issue??", "replies": [ { "id": 960421, "postDate": "2020-08-06T11:41:04.723Z", "content": "

@sidhivinayak \nHi,\nnot sure about that,\nbut maybe try removing the excess BatchNormalization you have after your softmax?

", "rawMarkdown": "@sidhivinayak \nHi,\nnot sure about that,\nbut maybe try removing the excess BatchNormalization you have after your softmax?\n", "votes": 1 }, { "id": 960592, "postDate": "2020-08-06T14:24:06.020Z", "content": "

ok lemme try that thank you!

", "rawMarkdown": "ok lemme try that thank you!" } ] }, { "id": 805629, "postDate": "2020-04-12T21:19:34.157Z", "content": "

model = Sequential()\nmodel.add(Conv2D(16,(3,3),activation='relu' , input_shape = X_train[0].shape))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.1))

\n\n

model.add(Conv2D(32,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.2))

\n\n

model.add(Conv2D(64,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))

\n\n

model.add(Conv2D(128,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))

\n\n

model.add(Flatten())

\n\n

model.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))

\n\n

model.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))

\n\n

model.add(Dense(5, activation='softmax'))

\n\n

i am using this and getting loss:Nan

", "rawMarkdown": "\nmodel = Sequential()\nmodel.add(Conv2D(16,(3,3),activation='relu' , input_shape = X_train[0].shape))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.1))\n\n\n\nmodel.add(Conv2D(32,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.2))\n\n\n\n\nmodel.add(Conv2D(64,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))\n\n\nmodel.add(Conv2D(128,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))\n\nmodel.add(Flatten())\n\nmodel.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))\n\nmodel.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))\n\nmodel.add(Dense(5, activation='softmax'))\n\ni am using this and getting loss:Nan", "replies": [ { "id": 825134, "postDate": "2020-04-28T20:00:12.040Z", "content": "

How did you compile the model ?

", "rawMarkdown": "How did you compile the model ? " } ] }, { "id": 546366, "postDate": "2019-06-06T14:16:52.513Z", "content": "

NaN's in the input , this is my case.

", "rawMarkdown": " NaN's in the input , this is my case." }, { "id": 393944, "postDate": "2018-09-26T05:20:11.983Z", "content": "

Hi there,

\n\n

I am also facing this issue. I am creating AutoEncoder (DAE).\nStructure is

\n\n
cols = X.shape[1]\n
\n\n

xt, xval = train_test_split(X, test_size = 0.1, random_state = 7)

\n\n

inp = Input(shape=(cols,))

\n\n

encoded = Dense(cols * 2, activation='relu')(inp)

\n\n

encoded = Dropout(0.5)(encoded)

\n\n

encoded = Dense(cols , activation='relu')(encoded)

\n\n

encoded = Dropout(0.2)(encoded)

\n\n

encoded = Dense(cols // 4, activation='relu')(encoded)

\n\n

decoded = Dense(cols // 4, activation='relu')(encoded)

\n\n

encoded = Dropout(0.2)(encoded)

\n\n

decoded = Dense(cols, activation='relu')(decoded)

\n\n

encoded = Dropout(0.5)(encoded)

\n\n

decoded = Dense(cols * 2, activation='relu')(decoded)

\n\n

decoded = Dense(cols, activation='sigmoid')(decoded)

\n\n

autoencoder = Model(inp, decoded)

\n\n

autoencoder.compile(optimizer='adam', loss='mse')

\n\n

I also tried following to fillna.

\n\n

X = X.fillna(0)

\n\n

X[np.isnan(X)] = 0

\n\n

for i in X.columns: X[i] = X[i].fillna(0)

\n\n

Then also I am getting nan as train & val loss\nCan anyone please help..

\n\n

Thanks

", "rawMarkdown": "Hi there,\n\nI am also facing this issue. I am creating AutoEncoder (DAE).\nStructure is\n\n cols = X.shape[1]\n\nxt, xval = train_test_split(X, test_size = 0.1, random_state = 7)\n\ninp = Input(shape=(cols,))\n\nencoded = Dense(cols * 2, activation='relu')(inp)\n\nencoded = Dropout(0.5)(encoded)\n\nencoded = Dense(cols , activation='relu')(encoded)\n\nencoded = Dropout(0.2)(encoded)\n\nencoded = Dense(cols // 4, activation='relu')(encoded)\n\n\ndecoded = Dense(cols // 4, activation='relu')(encoded)\n\nencoded = Dropout(0.2)(encoded)\n\ndecoded = Dense(cols, activation='relu')(decoded)\n\nencoded = Dropout(0.5)(encoded)\n\ndecoded = Dense(cols * 2, activation='relu')(decoded)\n\ndecoded = Dense(cols, activation='sigmoid')(decoded)\n\nautoencoder = Model(inp, decoded)\n\nautoencoder.compile(optimizer='adam', loss='mse')\n\nI also tried following to fillna.\n\nX = X.fillna(0)\n\nX[np.isnan(X)] = 0\n\nfor i in X.columns: X[i] = X[i].fillna(0)\n\nThen also I am getting nan as train & val loss\nCan anyone please help..\n\nThanks", "replies": [ { "id": 660664, "postDate": "2019-10-29T12:49:43.093Z", "content": "

Hi Prashant,

\n\n

in my case it was the exact 0 values for NaN. Maybe one of both:\n```

\n\n

Not exact 0 value

\n\n

X.fillna(1e-10)\n\n

\n\n

Forward fill last known value

\n\n

X.fillna(method='ffill')\n```\nThanks,\nDennis

", "rawMarkdown": "Hi Prashant,\n\nin my case it was the exact 0 values for NaN. Maybe one of both:\n```\n#Not exact 0 value\nX.fillna(1e-10)\n```\n```\n#Forward fill last known value\nX.fillna(method='ffill')\n```\nThanks,\nDennis" } ] }, { "id": 1090900, "postDate": "2020-11-25T16:53:06.187Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 539860, "author_name": "TejaChebrolu", "author_url": "", "post_date": "2019-05-30T16:07:36.577000", "content": "

It is not because of having missing values in the data. It is because the activation function is out of range for handling the input weight vector. Change the activation to softmax in the output layer and try,

", "votes": 13, "replies": [ { "id": 665795, "author_name": "Serkan Peldek", "author_url": "", "post_date": "2019-11-05T12:06:49.213000", "content": "

Your answer solve my issue. Thanks

", "votes": 0, "replies": [] }, { "id": 808077, "author_name": "Vadim Placinta", "author_url": "", "post_date": "2020-04-15T06:07:06.347000", "content": "

thank you, it solved my issue too, but in my case sigmoid seems to be more suitable

", "votes": 0, "replies": [] } ] }, { "id": 330915, "author_name": "Shanth", "author_url": "", "post_date": "2018-05-20T02:55:56.597000", "content": "

I'v faced this problem before. Its definitely NaN's in the input. Printing out DataFrame.isnull().any() will do the trick.

", "votes": 13, "replies": [] }, { "id": 330824, "author_name": "عثمان", "author_url": "", "post_date": "2018-05-19T19:37:27.707000", "content": "

You for sure have some nans in your input. Check your continuous variables, especially the ones you've FE'd (if you do FE for your NNet?) and make sure there aren't any NANs in there.

", "votes": 5, "replies": [ { "id": 330825, "author_name": "AmirH", "author_url": "", "post_date": "2018-05-19T19:39:00.443000", "content": "

Thanks my friend, i will check that out!\nI handled only NAs, not NANs

", "votes": 0, "replies": [] }, { "id": 580923, "author_name": "Kranthi Kumar", "author_url": "", "post_date": "2019-07-21T04:32:21.853000", "content": "

Handling nan in input solved the issue... Thanks!

", "votes": 3, "replies": [] }, { "id": 1616331, "author_name": "Himanshu", "author_url": "", "post_date": "2021-12-13T10:52:42.493000", "content": "

Solved my problem. Had issues with NaNs.

", "votes": 0, "replies": [] } ] }, { "id": 1495197, "author_name": "Samyar Rahimi", "author_url": "", "post_date": "2021-08-29T11:36:22.047000", "content": "

I had the nan loss problem in image data, I was using np.empty for generating the batches of images and looks like it was the reason of the nan loss, changed np.empty to np.zeros and problem solved.

", "votes": 0, "replies": [] }, { "id": 959399, "author_name": "Sidhivinayak", "author_url": "", "post_date": "2020-08-05T14:53:11.320000", "content": "

Guys even i tried making a an ANN with keras .....i had double checked but there r no NaN's in my input this is my code...model = tf.keras.Sequential([\n tf.keras.layers.Dense(200, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(125, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(625, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(100, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(175, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(750, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(600, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(25, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(2, activation='softmax'),\n tf.keras.layers.BatchNormalization(),\nIn my compilation i made sure loss was binary crossentropy loss becuz i hv probability distributions....but still i get loss alone as nan while accuracy as 0.2724 like here....in my first epoch....Epoch 1/20\n15/15 [==============================] - 0s 17ms/step - loss: 'nan' - accuracy: 0.3841\nHow can i fix this issue??

", "votes": 0, "replies": [ { "id": 960421, "author_name": "AmirH", "author_url": "", "post_date": "2020-08-06T11:41:04.723000", "content": "

@sidhivinayak \nHi,\nnot sure about that,\nbut maybe try removing the excess BatchNormalization you have after your softmax?

", "votes": 1, "replies": [] }, { "id": 960592, "author_name": "Sidhivinayak", "author_url": "", "post_date": "2020-08-06T14:24:06.020000", "content": "

ok lemme try that thank you!

", "votes": 0, "replies": [] } ] }, { "id": 805629, "author_name": "Nikunj Pithwa", "author_url": "", "post_date": "2020-04-12T21:19:34.157000", "content": "

model = Sequential()\nmodel.add(Conv2D(16,(3,3),activation='relu' , input_shape = X_train[0].shape))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.1))

\n\n

model.add(Conv2D(32,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.2))

\n\n

model.add(Conv2D(64,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))

\n\n

model.add(Conv2D(128,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))

\n\n

model.add(Flatten())

\n\n

model.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))

\n\n

model.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))

\n\n

model.add(Dense(5, activation='softmax'))

\n\n

i am using this and getting loss:Nan

", "votes": 0, "replies": [ { "id": 825134, "author_name": "Aakarsh Yadav", "author_url": "", "post_date": "2020-04-28T20:00:12.040000", "content": "

How did you compile the model ?

", "votes": 0, "replies": [] } ] }, { "id": 546366, "author_name": "Yang Yang", "author_url": "", "post_date": "2019-06-06T14:16:52.513000", "content": "

NaN's in the input , this is my case.

", "votes": 0, "replies": [] }, { "id": 393944, "author_name": "Prashant Kikani", "author_url": "", "post_date": "2018-09-26T05:20:11.983000", "content": "

Hi there,

\n\n

I am also facing this issue. I am creating AutoEncoder (DAE).\nStructure is

\n\n
cols = X.shape[1]\n
\n\n

xt, xval = train_test_split(X, test_size = 0.1, random_state = 7)

\n\n

inp = Input(shape=(cols,))

\n\n

encoded = Dense(cols * 2, activation='relu')(inp)

\n\n

encoded = Dropout(0.5)(encoded)

\n\n

encoded = Dense(cols , activation='relu')(encoded)

\n\n

encoded = Dropout(0.2)(encoded)

\n\n

encoded = Dense(cols // 4, activation='relu')(encoded)

\n\n

decoded = Dense(cols // 4, activation='relu')(encoded)

\n\n

encoded = Dropout(0.2)(encoded)

\n\n

decoded = Dense(cols, activation='relu')(decoded)

\n\n

encoded = Dropout(0.5)(encoded)

\n\n

decoded = Dense(cols * 2, activation='relu')(decoded)

\n\n

decoded = Dense(cols, activation='sigmoid')(decoded)

\n\n

autoencoder = Model(inp, decoded)

\n\n

autoencoder.compile(optimizer='adam', loss='mse')

\n\n

I also tried following to fillna.

\n\n

X = X.fillna(0)

\n\n

X[np.isnan(X)] = 0

\n\n

for i in X.columns: X[i] = X[i].fillna(0)

\n\n

Then also I am getting nan as train & val loss\nCan anyone please help..

\n\n

Thanks

", "votes": 0, "replies": [ { "id": 660664, "author_name": "Dennis_AFK", "author_url": "", "post_date": "2019-10-29T12:49:43.093000", "content": "

Hi Prashant,

\n\n

in my case it was the exact 0 values for NaN. Maybe one of both:\n```

\n\n

Not exact 0 value

\n\n

X.fillna(1e-10)\n\n

\n\n

Forward fill last known value

\n\n

X.fillna(method='ffill')\n```\nThanks,\nDennis

", "votes": 0, "replies": [] } ] }, { "id": 1090900, "author_name": "", "author_url": "", "post_date": "2020-11-25T16:53:06.187000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "539860": "It is not because of having missing values in the data. It is because the activation function is out of range for handling the input weight vector. Change the activation to softmax in the output layer and try,", "330915": "I'v faced this problem before. Its definitely NaN's in the input. Printing out DataFrame.isnull().any() will do the trick.", "330820": "Hi guys and gals,\nI'm trying to run a keras NN on the data and my loss and metric data is NAN:\n\nThis is my keras code (In R), Any ideas?\n(i know i should use RMSE and not MSE, i'm just testing things out, as RMSE requires a side function)\nThanks!!!\n\n\nlibrary(\"keras\")\n\nmodel <- keras_model_sequential()\n\nmodel %>% \n\n layer_dense(units=43,activation=\"relu\",input_shape=c(43)) %>%\n\n layer_dropout(rate=0.2) %>%\n\n layer_dense(units = 22,activation = \"relu\") %>%\n\n layer_dropout(rate=0.2) %>%\n \n\n layer_dense(units=1,activation=\"relu\") \n\nsummary(model)\n\nmodel %>%\n compile(loss =\"mean_squared_error\",\n optimizer = \"adam\",\n metrics= c(\"mse\"))\n\nhistory<-model %>% fit(TrainData, DealProbs, epochs = 1,\n callbacks = callback_tensorboard(log_dir = \"logs/run_b\"),\n validation_split = 0.2) #train on 80% of train set and will evaluate \n\nThe result (I ran it on small part of the data to test)\nEpoch 1/1\n80000/80000 [==============================] - 5s 60us/step - loss: nan - mean_squared_error: nan - val_loss: nan - val_mean_squared_error: nan", "330824": "You for sure have some nans in your input. Check your continuous variables, especially the ones you've FE'd (if you do FE for your NNet?) and make sure there aren't any NANs in there.", "1495197": "I had the nan loss problem in image data, I was using **np.empty** for generating the batches of images and looks like it was the reason of the nan loss, changed **np.empty** to **np.zeros** and problem solved.", "959399": "Guys even i tried making a an ANN with keras .....i had double checked but there r no NaN's in my input this is my code...model = tf.keras.Sequential([\n tf.keras.layers.Dense(200, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(125, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(625, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(100, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(175, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(750, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(600, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(25, activation='relu', kernel_initializer='he_uniform'),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(2, activation='softmax'),\n tf.keras.layers.BatchNormalization(),\nIn my compilation i made sure loss was binary crossentropy loss becuz i hv probability distributions....but still i get loss alone as nan while accuracy as 0.2724 like here....in my first epoch....Epoch 1/20\n15/15 [==============================] - 0s 17ms/step - loss: 'nan' - accuracy: 0.3841\nHow can i fix this issue??", "805629": "\nmodel = Sequential()\nmodel.add(Conv2D(16,(3,3),activation='relu' , input_shape = X_train[0].shape))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.1))\n\n\n\nmodel.add(Conv2D(32,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.2))\n\n\n\n\nmodel.add(Conv2D(64,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))\n\n\nmodel.add(Conv2D(128,(3,3),activation='relu' ))\nmodel.add(BatchNormalization())\nmodel.add(MaxPool2D(2,2))\nmodel.add(Dropout(0.3))\n\nmodel.add(Flatten())\n\nmodel.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))\n\nmodel.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.3))\n\nmodel.add(Dense(5, activation='softmax'))\n\ni am using this and getting loss:Nan", "546366": " NaN's in the input , this is my case.", "393944": "Hi there,\n\nI am also facing this issue. I am creating AutoEncoder (DAE).\nStructure is\n\n cols = X.shape[1]\n\nxt, xval = train_test_split(X, test_size = 0.1, random_state = 7)\n\ninp = Input(shape=(cols,))\n\nencoded = Dense(cols * 2, activation='relu')(inp)\n\nencoded = Dropout(0.5)(encoded)\n\nencoded = Dense(cols , activation='relu')(encoded)\n\nencoded = Dropout(0.2)(encoded)\n\nencoded = Dense(cols // 4, activation='relu')(encoded)\n\n\ndecoded = Dense(cols // 4, activation='relu')(encoded)\n\nencoded = Dropout(0.2)(encoded)\n\ndecoded = Dense(cols, activation='relu')(decoded)\n\nencoded = Dropout(0.5)(encoded)\n\ndecoded = Dense(cols * 2, activation='relu')(decoded)\n\ndecoded = Dense(cols, activation='sigmoid')(decoded)\n\nautoencoder = Model(inp, decoded)\n\nautoencoder.compile(optimizer='adam', loss='mse')\n\nI also tried following to fillna.\n\nX = X.fillna(0)\n\nX[np.isnan(X)] = 0\n\nfor i in X.columns: X[i] = X[i].fillna(0)\n\nThen also I am getting nan as train & val loss\nCan anyone please help..\n\nThanks", "1090900": "" } }