{
  "id": 77554,
  "title": "keras batch normalization produce NAN",
  "url": "/competitions/humpback-whale-identification/discussion/77554",
  "author_name": "",
  "post_date": "2019-01-14T08:39:54.654689800Z",
  "votes": 4,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I come into a very strange problem.I train a siamese model with keras. During the training stage , everything seems fine. However, in eval stage,the model produce the same results everytime. Then i print all the mid layers' output, it turns out to be that the batch normalization layer produce the NAN while the layer before batch normalization seems fine.Help..This really confuses me</p>",
  "messages": [
    {
      "id": "455606",
      "postDate": "01/14/2019 08:39:54",
      "content": "<p>I come into a very strange problem.I train a siamese model with keras. During the training stage , everything seems fine. However, in eval stage,the model produce the same results everytime. Then i print all the mid layers' output, it turns out to be that the batch normalization layer produce the NAN while the layer before batch normalization seems fine.Help..This really confuses me</p>",
      "rawMarkdown": "I come into a very strange problem.I train a siamese model with keras. During the training stage , everything seems fine. However, in eval stage,the model produce the same results everytime. Then i print all the mid layers' output, it turns out to be that the batch normalization layer produce the NAN while the layer before batch normalization seems fine.Help..This really confuses me",
      "votes": null
    },
    {
      "id": "953051",
      "postDate": "07/31/2020 13:30:59",
      "content": "<p>In case you are still wondering what was wrong :\nMaybe all your inputs inside one batch had the same value, it happened to me. It is explained here <a href=\"https://stackoverflow.com/questions/42333163/keras-nan-training-loss-after-introducing-batch-normalization\">https://stackoverflow.com/questions/42333163/keras-nan-training-loss-after-introducing-batch-normalization</a></p>",
      "rawMarkdown": "In case you are still wondering what was wrong :\nMaybe all your inputs inside one batch had the same value, it happened to me. It is explained here https://stackoverflow.com/questions/42333163/keras-nan-training-loss-after-introducing-batch-normalization",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 953051,
      "author_name": "skanderbeg",
      "author_url": "",
      "post_date": "07/31/2020 13:30:59",
      "content": "<p>In case you are still wondering what was wrong :\nMaybe all your inputs inside one batch had the same value, it happened to me. It is explained here <a href=\"https://stackoverflow.com/questions/42333163/keras-nan-training-loss-after-introducing-batch-normalization\">https://stackoverflow.com/questions/42333163/keras-nan-training-loss-after-introducing-batch-normalization</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "455606": "I come into a very strange problem.I train a siamese model with keras. During the training stage , everything seems fine. However, in eval stage,the model produce the same results everytime. Then i print all the mid layers' output, it turns out to be that the batch normalization layer produce the NAN while the layer before batch normalization seems fine.Help..This really confuses me",
    "953051": "In case you are still wondering what was wrong :\nMaybe all your inputs inside one batch had the same value, it happened to me. It is explained here https://stackoverflow.com/questions/42333163/keras-nan-training-loss-after-introducing-batch-normalization"
  },
  "source": "meta"
}