{
  "id": 265617,
  "title": "When to use val_loss vs val_auc with early stopping? ",
  "url": "/competitions/g2net-gravitational-wave-detection/discussion/265617",
  "author_name": "Fractal Feelings",
  "post_date": "2021-08-16T09:11:17.158000",
  "votes": 3,
  "comment_count": 8,
  "views": 0,
  "content": "<p>When using early stopping in NN are there any solid reasons to use either using val_loss or the metric (i.e., val_AUC in this case)?</p>",
  "messages": [
    {
      "id": 1474794,
      "postDate": "2021-08-16T09:11:17.160Z",
      "content": "<p>When using early stopping in NN are there any solid reasons to use either using val_loss or the metric (i.e., val_AUC in this case)?</p>",
      "rawMarkdown": "When using early stopping in NN are there any solid reasons to use either using val_loss or the metric (i.e., val_AUC in this case)?\n",
      "votes": 3
    },
    {
      "id": 1477121,
      "postDate": "2021-08-17T11:05:04.887Z",
      "content": "<p>I don't use early stopping.  Instead I tune so that last epoch is the best.</p>",
      "rawMarkdown": "I don't use early stopping.  Instead I tune so that last epoch is the best.",
      "votes": 1,
      "replies": [
        {
          "id": 1478464,
          "postDate": "2021-08-18T02:20:29.770Z",
          "content": "<p>Now that's an interesting approach. I assume you do this by setting a large # of epoch, then monitor the val_loss or val_auc to find the best epoch?</p>",
          "rawMarkdown": "Now that's an interesting approach. I assume you do this by setting a large # of epoch, then monitor the val_loss or val_auc to find the best epoch?"
        },
        {
          "id": 1479167,
          "postDate": "2021-08-18T10:19:33.983Z",
          "content": "<p>I fix number of epochs and do not use ES.  Tuning then drives you to values where last epoch is the best on average.  </p>",
          "rawMarkdown": "I fix number of epochs and do not use ES.  Tuning then drives you to values where last epoch is the best on average.  ",
          "votes": 2
        },
        {
          "id": 1479450,
          "postDate": "2021-08-18T13:24:13.097Z",
          "content": "<p>that is good point for best model. early stopping is  for limit resources and eager for an acceptable model? </p>",
          "rawMarkdown": "that is good point for best model. early stopping is  for limit resources and eager for an acceptable model? "
        },
        {
          "id": 1479580,
          "postDate": "2021-08-18T14:28:35.887Z",
          "content": "<p>If you are limited in compute and can only afford N epochs, then it looks better to tune so that Nth epoch is best, rather than run for more epochs and hope that early stopping will kick in before N+1.  But that's just my opinion, feel free to do otherwise.</p>",
          "rawMarkdown": "If you are limited in compute and can only afford N epochs, then it looks better to tune so that Nth epoch is best, rather than run for more epochs and hope that early stopping will kick in before N+1.  But that's just my opinion, feel free to do otherwise.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1561177,
      "postDate": "2021-10-27T12:14:55.017Z",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
    },
    {
      "id": 1482437,
      "postDate": "2021-08-20T05:06:55.947Z",
      "content": "<p>I also don't think it's necessary to stop early. I usually finish it with 3-4 epochs</p>",
      "rawMarkdown": "I also don't think it's necessary to stop early. I usually finish it with 3-4 epochs"
    },
    {
      "id": 1475844,
      "postDate": "2021-08-16T21:19:49.103Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1477121,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2021-08-17T11:05:04.887000",
      "content": "<p>I don't use early stopping.  Instead I tune so that last epoch is the best.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1478464,
          "author_name": "Fractal Feelings",
          "author_url": "",
          "post_date": "2021-08-18T02:20:29.770000",
          "content": "<p>Now that's an interesting approach. I assume you do this by setting a large # of epoch, then monitor the val_loss or val_auc to find the best epoch?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1479167,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-08-18T10:19:33.983000",
          "content": "<p>I fix number of epochs and do not use ES.  Tuning then drives you to values where last epoch is the best on average.  </p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1479450,
          "author_name": "dragon zhang",
          "author_url": "",
          "post_date": "2021-08-18T13:24:13.097000",
          "content": "<p>that is good point for best model. early stopping is  for limit resources and eager for an acceptable model? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1479580,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2021-08-18T14:28:35.887000",
          "content": "<p>If you are limited in compute and can only afford N epochs, then it looks better to tune so that Nth epoch is best, rather than run for more epochs and hope that early stopping will kick in before N+1.  But that's just my opinion, feel free to do otherwise.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1561177,
      "author_name": "ChristopherZerafa",
      "author_url": "",
      "post_date": "2021-10-27T12:14:55.017000",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1482437,
      "author_name": "zhangeng",
      "author_url": "",
      "post_date": "2021-08-20T05:06:55.947000",
      "content": "<p>I also don't think it's necessary to stop early. I usually finish it with 3-4 epochs</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1475844,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-08-16T21:19:49.103000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1474794": "When using early stopping in NN are there any solid reasons to use either using val_loss or the metric (i.e., val_AUC in this case)?\n",
    "1477121": "I don't use early stopping.  Instead I tune so that last epoch is the best.",
    "1561177": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
    "1482437": "I also don't think it's necessary to stop early. I usually finish it with 3-4 epochs",
    "1475844": ""
  }
}