{
  "id": 73526,
  "title": "Validation set categorical accuracy stuck at 0.3594",
  "url": "/competitions/humpback-whale-identification/discussion/73526",
  "author_name": "",
  "post_date": "2018-12-03T19:56:01.271409Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Recently i have been getting my val_categorical_accuracy stuck on a value on all my models, weird because loss still decreases with each epoch.\n<a href=\"https://www.kaggle.com/amneves/humpkeras-whale-identification\">My whale kernel</a></p>\n\n<p>You can see by this plots that i might have a stupid bug somewhere, it used to ran fine on my other kernels, now i got two of those like this, if someone could help me identify the problem i would give them a kudos. I can't seem to identify the change that broke this.</p>\n\n<p><img src=\"https://media.discordapp.net/attachments/91848983500029952/519233611719901208/unknown.png\" alt=\"Stuck accuracy\">\n<img src=\"https://media.discordapp.net/attachments/91848983500029952/519233769648029696/unknown.png\" alt=\"Decreasing loss\"></p>",
  "messages": [
    {
      "id": "432396",
      "postDate": "12/03/2018 19:56:01",
      "content": "<p>Recently i have been getting my val_categorical_accuracy stuck on a value on all my models, weird because loss still decreases with each epoch.\n<a href=\"https://www.kaggle.com/amneves/humpkeras-whale-identification\">My whale kernel</a></p>\n\n<p>You can see by this plots that i might have a stupid bug somewhere, it used to ran fine on my other kernels, now i got two of those like this, if someone could help me identify the problem i would give them a kudos. I can't seem to identify the change that broke this.</p>\n\n<p><img src=\"https://media.discordapp.net/attachments/91848983500029952/519233611719901208/unknown.png\" alt=\"Stuck accuracy\">\n<img src=\"https://media.discordapp.net/attachments/91848983500029952/519233769648029696/unknown.png\" alt=\"Decreasing loss\"></p>",
      "rawMarkdown": "Recently i have been getting my val_categorical_accuracy stuck on a value on all my models, weird because loss still decreases with each epoch.\n[My whale kernel][1]\n\nYou can see by this plots that i might have a stupid bug somewhere, it used to ran fine on my other kernels, now i got two of those like this, if someone could help me identify the problem i would give them a kudos. I can't seem to identify the change that broke this.\n\n![Stuck accuracy][2]\n![Decreasing loss][3]\n\n\n  [1]: https://www.kaggle.com/amneves/humpkeras-whale-identification\n  [2]: https://media.discordapp.net/attachments/91848983500029952/519233611719901208/unknown.png\n  [3]: https://media.discordapp.net/attachments/91848983500029952/519233769648029696/unknown.png",
      "votes": null
    },
    {
      "id": "432439",
      "postDate": "12/03/2018 21:49:36",
      "content": "<p>I've had a look at your kernels and I won't pretend to understand what is going on there, but I have spotted a few differences:\n 1. In the first version, you have a Flatten part in ln 16 that doesn't seem to be there in the second version. Perhaps you've replaced this with the GlobalAveragePooling2D part?\n 2. Your first version has 20 epochs, while your second version has only 10. Should not make such a big difference, but it might be worth a shot to run more epochs and see what happens.\n 3. Your activation functions are completely different. In the first version, you use softmax. In the second version, you use a sigmoid. Maybe read up on the documentation for these and decide if it's what you want to do here. I'm guessing you might want to use softmax instead, but I am not too confident that I understand your code.\n 4. Version two has a Dropout rate of 0.5. You could try to make it 0.2 or even remove it and see what happens. Your first version does not seem to have a dropout rate involved.</p>\n\n<p>Hope this helps!</p>",
      "rawMarkdown": "I've had a look at your kernels and I won't pretend to understand what is going on there, but I have spotted a few differences:\n 1. In the first version, you have a Flatten part in ln 16 that doesn't seem to be there in the second version. Perhaps you've replaced this with the GlobalAveragePooling2D part?\n 2. Your first version has 20 epochs, while your second version has only 10. Should not make such a big difference, but it might be worth a shot to run more epochs and see what happens.\n 3. Your activation functions are completely different. In the first version, you use softmax. In the second version, you use a sigmoid. Maybe read up on the documentation for these and decide if it's what you want to do here. I'm guessing you might want to use softmax instead, but I am not too confident that I understand your code.\n 4. Version two has a Dropout rate of 0.5. You could try to make it 0.2 or even remove it and see what happens. Your first version does not seem to have a dropout rate involved.\n\nHope this helps!",
      "votes": null
    },
    {
      "id": "432515",
      "postDate": "12/04/2018 01:31:25",
      "content": "<p>Thanks! Didn't notice the activation function difference, that was my bad! I need a softmax for this problem.</p>\n\n<p>The GlobalAveragePooling2D should perform better in theory in this dataset, but i will try with flatten again.</p>\n\n<p>I think there's a bug somewhere that makes the code behave weirdly, my google doodle kernel uses the same principles as i applied here and works fine</p>",
      "rawMarkdown": "Thanks! Didn't notice the activation function difference, that was my bad! I need a softmax for this problem.\n\nThe GlobalAveragePooling2D should perform better in theory in this dataset, but i will try with flatten again.\n\nI think there's a bug somewhere that makes the code behave weirdly, my google doodle kernel uses the same principles as i applied here and works fine",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 432439,
      "author_name": "devilears",
      "author_url": "",
      "post_date": "12/03/2018 21:49:36",
      "content": "<p>I've had a look at your kernels and I won't pretend to understand what is going on there, but I have spotted a few differences:\n 1. In the first version, you have a Flatten part in ln 16 that doesn't seem to be there in the second version. Perhaps you've replaced this with the GlobalAveragePooling2D part?\n 2. Your first version has 20 epochs, while your second version has only 10. Should not make such a big difference, but it might be worth a shot to run more epochs and see what happens.\n 3. Your activation functions are completely different. In the first version, you use softmax. In the second version, you use a sigmoid. Maybe read up on the documentation for these and decide if it's what you want to do here. I'm guessing you might want to use softmax instead, but I am not too confident that I understand your code.\n 4. Version two has a Dropout rate of 0.5. You could try to make it 0.2 or even remove it and see what happens. Your first version does not seem to have a dropout rate involved.</p>\n\n<p>Hope this helps!</p>",
      "votes": null,
      "replies": [
        {
          "id": 432515,
          "author_name": "amneves",
          "author_url": "",
          "post_date": "12/04/2018 01:31:25",
          "content": "<p>Thanks! Didn't notice the activation function difference, that was my bad! I need a softmax for this problem.</p>\n\n<p>The GlobalAveragePooling2D should perform better in theory in this dataset, but i will try with flatten again.</p>\n\n<p>I think there's a bug somewhere that makes the code behave weirdly, my google doodle kernel uses the same principles as i applied here and works fine</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "432396": "Recently i have been getting my val_categorical_accuracy stuck on a value on all my models, weird because loss still decreases with each epoch.\n[My whale kernel][1]\n\nYou can see by this plots that i might have a stupid bug somewhere, it used to ran fine on my other kernels, now i got two of those like this, if someone could help me identify the problem i would give them a kudos. I can't seem to identify the change that broke this.\n\n![Stuck accuracy][2]\n![Decreasing loss][3]\n\n\n  [1]: https://www.kaggle.com/amneves/humpkeras-whale-identification\n  [2]: https://media.discordapp.net/attachments/91848983500029952/519233611719901208/unknown.png\n  [3]: https://media.discordapp.net/attachments/91848983500029952/519233769648029696/unknown.png",
    "432439": "I've had a look at your kernels and I won't pretend to understand what is going on there, but I have spotted a few differences:\n 1. In the first version, you have a Flatten part in ln 16 that doesn't seem to be there in the second version. Perhaps you've replaced this with the GlobalAveragePooling2D part?\n 2. Your first version has 20 epochs, while your second version has only 10. Should not make such a big difference, but it might be worth a shot to run more epochs and see what happens.\n 3. Your activation functions are completely different. In the first version, you use softmax. In the second version, you use a sigmoid. Maybe read up on the documentation for these and decide if it's what you want to do here. I'm guessing you might want to use softmax instead, but I am not too confident that I understand your code.\n 4. Version two has a Dropout rate of 0.5. You could try to make it 0.2 or even remove it and see what happens. Your first version does not seem to have a dropout rate involved.\n\nHope this helps!",
    "432515": "Thanks! Didn't notice the activation function difference, that was my bad! I need a softmax for this problem.\n\nThe GlobalAveragePooling2D should perform better in theory in this dataset, but i will try with flatten again.\n\nI think there's a bug somewhere that makes the code behave weirdly, my google doodle kernel uses the same principles as i applied here and works fine"
  },
  "source": "meta"
}