{
  "id": 217145,
  "title": "How to interpret increase in both val_loss and val_auc?",
  "url": "/competitions/ranzcr-clip-catheter-line-classification/discussion/217145",
  "author_name": "",
  "post_date": "2021-02-05T13:42:05.152503500Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I train an EfficientNet-B7 model. It is easy to happen when <code>weights='noisy-student'</code>, but <code>weights='imagenet'</code>will not. How can I do with my model?</p>\n<pre><code>188/188 - 255s - loss: 0.7037 - auc: 0.9103 - val_loss: 0.8523 - val_auc: 0.8683 - lr: 0.0010\nEpoch 11/35\n188/188 - 254s - loss: 0.6741 - auc: 0.9192 - val_loss: 0.7844 - val_auc: 0.8791 - lr: 0.0010\nEpoch 12/35\n188/188 - 254s - loss: 0.6439 - auc: 0.9293 - val_loss: 0.7232 - val_auc: 0.8944 - lr: 0.0010\nEpoch 13/35\n188/188 - 264s - loss: 0.5469 - auc: 0.9502 - val_loss: 0.6718 - val_auc: 0.9088 - lr: 1.0000e-04\nEpoch 14/35\n188/188 - 254s - loss: 0.5104 - auc: 0.9575 - val_loss: 0.6832 - val_auc: 0.9076 - lr: 1.0000e-04\nEpoch 15/35\n188/188 - 265s - loss: 0.4851 - auc: 0.9624 - val_loss: 0.6772 - val_auc: 0.9121 - lr: 1.0000e-04\nEpoch 16/35\n188/188 - 265s - loss: 0.4700 - auc: 0.9648 - val_loss: 0.6832 - val_auc: 0.9171 - lr: 1.0000e-04\nEpoch 17/35\n188/188 - 253s - loss: 0.4480 - auc: 0.9662 - val_loss: 0.6927 - val_auc: 0.9166 - lr: 1.0000e-04\nEpoch 18/35\n188/188 - 265s - loss: 0.4318 - auc: 0.9657 - val_loss: 0.6998 - val_auc: 0.9186 - lr: 1.0000e-04\nEpoch 19/35\n188/188 - 265s - loss: 0.4218 - auc: 0.9703 - val_loss: 0.7038 - val_auc: 0.9215 - lr: 1.0000e-04\nEpoch 20/35\n188/188 - 254s - loss: 0.4038 - auc: 0.9706 - val_loss: 0.7292 - val_auc: 0.9152 - lr: 1.0000e-04\nEpoch 21/35\n188/188 - 255s - loss: 0.3943 - auc: 0.9752 - val_loss: 0.7277 - val_auc: 0.9190 - lr: 1.0000e-04\nEpoch 22/35\n188/188 - 255s - loss: 0.3806 - auc: 0.9742 - val_loss: 0.7359 - val_auc: 0.9214 - lr: 1.0000e-04\nEpoch 23/35\n188/188 - 254s - loss: 0.3538 - auc: 0.9797 - val_loss: 0.7410 - val_auc: 0.9210 - lr: 1.0000e-05\nEpoch 24/35\n188/188 - 254s - loss: 0.3543 - auc: 0.9793 - val_loss: 0.7445 - val_auc: 0.9211 - lr: 1.0000e-05\nEpoch 25/35\n188/188 - 268s - loss: 0.3492 - auc: 0.9793 - val_loss: 0.7466 - val_auc: 0.9219 - lr: 1.0000e-05\nEpoch 26/35\n188/188 - 254s - loss: 0.3452 - auc: 0.9791 - val_loss: 0.7514 - val_auc: 0.9206 - lr: 1.0000e-05\nEpoch 27/35\n188/188 - 255s - loss: 0.3404 - auc: 0.9813 - val_loss: 0.7524 - val_auc: 0.9198 - lr: 1.0000e-05\nEpoch 28/35\n188/188 - 254s - loss: 0.3429 - auc: 0.9795 - val_loss: 0.7544 - val_auc: 0.9194 - lr: 1.0000e-05\nEpoch 29/35\n188/188 - 254s - loss: 0.3377 - auc: 0.9803 - val_loss: 0.7546 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 30/35\n188/188 - 255s - loss: 0.3363 - auc: 0.9812 - val_loss: 0.7548 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 31/35\n188/188 - 254s - loss: 0.3407 - auc: 0.9805 - val_loss: 0.7554 - val_auc: 0.9195 - lr: 1.0000e-06\nEpoch 32/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9806 - val_loss: 0.7565 - val_auc: 0.9186 - lr: 1.0000e-06\nEpoch 33/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9812 - val_loss: 0.7569 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 34/35\n188/188 - 255s - loss: 0.3388 - auc: 0.9791 - val_loss: 0.7553 - val_auc: 0.9194 - lr: 1.0000e-06\nEpoch 35/35\n188/188 - 254s - loss: 0.3417 - auc: 0.9796 - val_loss: 0.7566 - val_auc: 0.9205 - lr: 1.0000e-06\n</code></pre>",
  "messages": [
    {
      "id": "1187500",
      "postDate": "02/05/2021 13:42:05",
      "content": "<p>I train an EfficientNet-B7 model. It is easy to happen when <code>weights='noisy-student'</code>, but <code>weights='imagenet'</code>will not. How can I do with my model?</p>\n<pre><code>188/188 - 255s - loss: 0.7037 - auc: 0.9103 - val_loss: 0.8523 - val_auc: 0.8683 - lr: 0.0010\nEpoch 11/35\n188/188 - 254s - loss: 0.6741 - auc: 0.9192 - val_loss: 0.7844 - val_auc: 0.8791 - lr: 0.0010\nEpoch 12/35\n188/188 - 254s - loss: 0.6439 - auc: 0.9293 - val_loss: 0.7232 - val_auc: 0.8944 - lr: 0.0010\nEpoch 13/35\n188/188 - 264s - loss: 0.5469 - auc: 0.9502 - val_loss: 0.6718 - val_auc: 0.9088 - lr: 1.0000e-04\nEpoch 14/35\n188/188 - 254s - loss: 0.5104 - auc: 0.9575 - val_loss: 0.6832 - val_auc: 0.9076 - lr: 1.0000e-04\nEpoch 15/35\n188/188 - 265s - loss: 0.4851 - auc: 0.9624 - val_loss: 0.6772 - val_auc: 0.9121 - lr: 1.0000e-04\nEpoch 16/35\n188/188 - 265s - loss: 0.4700 - auc: 0.9648 - val_loss: 0.6832 - val_auc: 0.9171 - lr: 1.0000e-04\nEpoch 17/35\n188/188 - 253s - loss: 0.4480 - auc: 0.9662 - val_loss: 0.6927 - val_auc: 0.9166 - lr: 1.0000e-04\nEpoch 18/35\n188/188 - 265s - loss: 0.4318 - auc: 0.9657 - val_loss: 0.6998 - val_auc: 0.9186 - lr: 1.0000e-04\nEpoch 19/35\n188/188 - 265s - loss: 0.4218 - auc: 0.9703 - val_loss: 0.7038 - val_auc: 0.9215 - lr: 1.0000e-04\nEpoch 20/35\n188/188 - 254s - loss: 0.4038 - auc: 0.9706 - val_loss: 0.7292 - val_auc: 0.9152 - lr: 1.0000e-04\nEpoch 21/35\n188/188 - 255s - loss: 0.3943 - auc: 0.9752 - val_loss: 0.7277 - val_auc: 0.9190 - lr: 1.0000e-04\nEpoch 22/35\n188/188 - 255s - loss: 0.3806 - auc: 0.9742 - val_loss: 0.7359 - val_auc: 0.9214 - lr: 1.0000e-04\nEpoch 23/35\n188/188 - 254s - loss: 0.3538 - auc: 0.9797 - val_loss: 0.7410 - val_auc: 0.9210 - lr: 1.0000e-05\nEpoch 24/35\n188/188 - 254s - loss: 0.3543 - auc: 0.9793 - val_loss: 0.7445 - val_auc: 0.9211 - lr: 1.0000e-05\nEpoch 25/35\n188/188 - 268s - loss: 0.3492 - auc: 0.9793 - val_loss: 0.7466 - val_auc: 0.9219 - lr: 1.0000e-05\nEpoch 26/35\n188/188 - 254s - loss: 0.3452 - auc: 0.9791 - val_loss: 0.7514 - val_auc: 0.9206 - lr: 1.0000e-05\nEpoch 27/35\n188/188 - 255s - loss: 0.3404 - auc: 0.9813 - val_loss: 0.7524 - val_auc: 0.9198 - lr: 1.0000e-05\nEpoch 28/35\n188/188 - 254s - loss: 0.3429 - auc: 0.9795 - val_loss: 0.7544 - val_auc: 0.9194 - lr: 1.0000e-05\nEpoch 29/35\n188/188 - 254s - loss: 0.3377 - auc: 0.9803 - val_loss: 0.7546 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 30/35\n188/188 - 255s - loss: 0.3363 - auc: 0.9812 - val_loss: 0.7548 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 31/35\n188/188 - 254s - loss: 0.3407 - auc: 0.9805 - val_loss: 0.7554 - val_auc: 0.9195 - lr: 1.0000e-06\nEpoch 32/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9806 - val_loss: 0.7565 - val_auc: 0.9186 - lr: 1.0000e-06\nEpoch 33/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9812 - val_loss: 0.7569 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 34/35\n188/188 - 255s - loss: 0.3388 - auc: 0.9791 - val_loss: 0.7553 - val_auc: 0.9194 - lr: 1.0000e-06\nEpoch 35/35\n188/188 - 254s - loss: 0.3417 - auc: 0.9796 - val_loss: 0.7566 - val_auc: 0.9205 - lr: 1.0000e-06\n</code></pre>",
      "rawMarkdown": "I train an EfficientNet-B7 model. It is easy to happen when `weights='noisy-student'`, but `weights='imagenet' `will not. How can I do with my model?\n\n\n```\n188/188 - 255s - loss: 0.7037 - auc: 0.9103 - val_loss: 0.8523 - val_auc: 0.8683 - lr: 0.0010\nEpoch 11/35\n188/188 - 254s - loss: 0.6741 - auc: 0.9192 - val_loss: 0.7844 - val_auc: 0.8791 - lr: 0.0010\nEpoch 12/35\n188/188 - 254s - loss: 0.6439 - auc: 0.9293 - val_loss: 0.7232 - val_auc: 0.8944 - lr: 0.0010\nEpoch 13/35\n188/188 - 264s - loss: 0.5469 - auc: 0.9502 - val_loss: 0.6718 - val_auc: 0.9088 - lr: 1.0000e-04\nEpoch 14/35\n188/188 - 254s - loss: 0.5104 - auc: 0.9575 - val_loss: 0.6832 - val_auc: 0.9076 - lr: 1.0000e-04\nEpoch 15/35\n188/188 - 265s - loss: 0.4851 - auc: 0.9624 - val_loss: 0.6772 - val_auc: 0.9121 - lr: 1.0000e-04\nEpoch 16/35\n188/188 - 265s - loss: 0.4700 - auc: 0.9648 - val_loss: 0.6832 - val_auc: 0.9171 - lr: 1.0000e-04\nEpoch 17/35\n188/188 - 253s - loss: 0.4480 - auc: 0.9662 - val_loss: 0.6927 - val_auc: 0.9166 - lr: 1.0000e-04\nEpoch 18/35\n188/188 - 265s - loss: 0.4318 - auc: 0.9657 - val_loss: 0.6998 - val_auc: 0.9186 - lr: 1.0000e-04\nEpoch 19/35\n188/188 - 265s - loss: 0.4218 - auc: 0.9703 - val_loss: 0.7038 - val_auc: 0.9215 - lr: 1.0000e-04\nEpoch 20/35\n188/188 - 254s - loss: 0.4038 - auc: 0.9706 - val_loss: 0.7292 - val_auc: 0.9152 - lr: 1.0000e-04\nEpoch 21/35\n188/188 - 255s - loss: 0.3943 - auc: 0.9752 - val_loss: 0.7277 - val_auc: 0.9190 - lr: 1.0000e-04\nEpoch 22/35\n188/188 - 255s - loss: 0.3806 - auc: 0.9742 - val_loss: 0.7359 - val_auc: 0.9214 - lr: 1.0000e-04\nEpoch 23/35\n188/188 - 254s - loss: 0.3538 - auc: 0.9797 - val_loss: 0.7410 - val_auc: 0.9210 - lr: 1.0000e-05\nEpoch 24/35\n188/188 - 254s - loss: 0.3543 - auc: 0.9793 - val_loss: 0.7445 - val_auc: 0.9211 - lr: 1.0000e-05\nEpoch 25/35\n188/188 - 268s - loss: 0.3492 - auc: 0.9793 - val_loss: 0.7466 - val_auc: 0.9219 - lr: 1.0000e-05\nEpoch 26/35\n188/188 - 254s - loss: 0.3452 - auc: 0.9791 - val_loss: 0.7514 - val_auc: 0.9206 - lr: 1.0000e-05\nEpoch 27/35\n188/188 - 255s - loss: 0.3404 - auc: 0.9813 - val_loss: 0.7524 - val_auc: 0.9198 - lr: 1.0000e-05\nEpoch 28/35\n188/188 - 254s - loss: 0.3429 - auc: 0.9795 - val_loss: 0.7544 - val_auc: 0.9194 - lr: 1.0000e-05\nEpoch 29/35\n188/188 - 254s - loss: 0.3377 - auc: 0.9803 - val_loss: 0.7546 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 30/35\n188/188 - 255s - loss: 0.3363 - auc: 0.9812 - val_loss: 0.7548 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 31/35\n188/188 - 254s - loss: 0.3407 - auc: 0.9805 - val_loss: 0.7554 - val_auc: 0.9195 - lr: 1.0000e-06\nEpoch 32/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9806 - val_loss: 0.7565 - val_auc: 0.9186 - lr: 1.0000e-06\nEpoch 33/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9812 - val_loss: 0.7569 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 34/35\n188/188 - 255s - loss: 0.3388 - auc: 0.9791 - val_loss: 0.7553 - val_auc: 0.9194 - lr: 1.0000e-06\nEpoch 35/35\n188/188 - 254s - loss: 0.3417 - auc: 0.9796 - val_loss: 0.7566 - val_auc: 0.9205 - lr: 1.0000e-06\n```",
      "votes": null
    },
    {
      "id": "1194572",
      "postDate": "02/10/2021 08:48:19",
      "content": "<p>Meaning: lower loss != lower auc. But there's no direct way to optimize AUC since it's not differentiable.</p>",
      "rawMarkdown": "Meaning: lower loss != lower auc. But there's no direct way to optimize AUC since it's not differentiable.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1194572,
      "author_name": "underwearfitting",
      "author_url": "",
      "post_date": "02/10/2021 08:48:19",
      "content": "<p>Meaning: lower loss != lower auc. But there's no direct way to optimize AUC since it's not differentiable.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1187500": "I train an EfficientNet-B7 model. It is easy to happen when `weights='noisy-student'`, but `weights='imagenet' `will not. How can I do with my model?\n\n\n```\n188/188 - 255s - loss: 0.7037 - auc: 0.9103 - val_loss: 0.8523 - val_auc: 0.8683 - lr: 0.0010\nEpoch 11/35\n188/188 - 254s - loss: 0.6741 - auc: 0.9192 - val_loss: 0.7844 - val_auc: 0.8791 - lr: 0.0010\nEpoch 12/35\n188/188 - 254s - loss: 0.6439 - auc: 0.9293 - val_loss: 0.7232 - val_auc: 0.8944 - lr: 0.0010\nEpoch 13/35\n188/188 - 264s - loss: 0.5469 - auc: 0.9502 - val_loss: 0.6718 - val_auc: 0.9088 - lr: 1.0000e-04\nEpoch 14/35\n188/188 - 254s - loss: 0.5104 - auc: 0.9575 - val_loss: 0.6832 - val_auc: 0.9076 - lr: 1.0000e-04\nEpoch 15/35\n188/188 - 265s - loss: 0.4851 - auc: 0.9624 - val_loss: 0.6772 - val_auc: 0.9121 - lr: 1.0000e-04\nEpoch 16/35\n188/188 - 265s - loss: 0.4700 - auc: 0.9648 - val_loss: 0.6832 - val_auc: 0.9171 - lr: 1.0000e-04\nEpoch 17/35\n188/188 - 253s - loss: 0.4480 - auc: 0.9662 - val_loss: 0.6927 - val_auc: 0.9166 - lr: 1.0000e-04\nEpoch 18/35\n188/188 - 265s - loss: 0.4318 - auc: 0.9657 - val_loss: 0.6998 - val_auc: 0.9186 - lr: 1.0000e-04\nEpoch 19/35\n188/188 - 265s - loss: 0.4218 - auc: 0.9703 - val_loss: 0.7038 - val_auc: 0.9215 - lr: 1.0000e-04\nEpoch 20/35\n188/188 - 254s - loss: 0.4038 - auc: 0.9706 - val_loss: 0.7292 - val_auc: 0.9152 - lr: 1.0000e-04\nEpoch 21/35\n188/188 - 255s - loss: 0.3943 - auc: 0.9752 - val_loss: 0.7277 - val_auc: 0.9190 - lr: 1.0000e-04\nEpoch 22/35\n188/188 - 255s - loss: 0.3806 - auc: 0.9742 - val_loss: 0.7359 - val_auc: 0.9214 - lr: 1.0000e-04\nEpoch 23/35\n188/188 - 254s - loss: 0.3538 - auc: 0.9797 - val_loss: 0.7410 - val_auc: 0.9210 - lr: 1.0000e-05\nEpoch 24/35\n188/188 - 254s - loss: 0.3543 - auc: 0.9793 - val_loss: 0.7445 - val_auc: 0.9211 - lr: 1.0000e-05\nEpoch 25/35\n188/188 - 268s - loss: 0.3492 - auc: 0.9793 - val_loss: 0.7466 - val_auc: 0.9219 - lr: 1.0000e-05\nEpoch 26/35\n188/188 - 254s - loss: 0.3452 - auc: 0.9791 - val_loss: 0.7514 - val_auc: 0.9206 - lr: 1.0000e-05\nEpoch 27/35\n188/188 - 255s - loss: 0.3404 - auc: 0.9813 - val_loss: 0.7524 - val_auc: 0.9198 - lr: 1.0000e-05\nEpoch 28/35\n188/188 - 254s - loss: 0.3429 - auc: 0.9795 - val_loss: 0.7544 - val_auc: 0.9194 - lr: 1.0000e-05\nEpoch 29/35\n188/188 - 254s - loss: 0.3377 - auc: 0.9803 - val_loss: 0.7546 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 30/35\n188/188 - 255s - loss: 0.3363 - auc: 0.9812 - val_loss: 0.7548 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 31/35\n188/188 - 254s - loss: 0.3407 - auc: 0.9805 - val_loss: 0.7554 - val_auc: 0.9195 - lr: 1.0000e-06\nEpoch 32/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9806 - val_loss: 0.7565 - val_auc: 0.9186 - lr: 1.0000e-06\nEpoch 33/35\n188/188 - 254s - loss: 0.3392 - auc: 0.9812 - val_loss: 0.7569 - val_auc: 0.9187 - lr: 1.0000e-06\nEpoch 34/35\n188/188 - 255s - loss: 0.3388 - auc: 0.9791 - val_loss: 0.7553 - val_auc: 0.9194 - lr: 1.0000e-06\nEpoch 35/35\n188/188 - 254s - loss: 0.3417 - auc: 0.9796 - val_loss: 0.7566 - val_auc: 0.9205 - lr: 1.0000e-06\n```",
    "1194572": "Meaning: lower loss != lower auc. But there's no direct way to optimize AUC since it's not differentiable."
  },
  "source": "meta"
}