{
  "id": 205572,
  "title": "Which metric to use in training to save models?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/205572",
  "author_name": "",
  "post_date": "2020-12-20T17:44:26.108369300Z",
  "votes": 5,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>I wonder that which metric do you track to save your best models? Loss, accuracy or maybe f1… Or do you save the model at every epoch and use some kind of validation scheme to choose which models to use in inference?</p>\n<p>I currently use loss and accuracy but have some doubts since I still couldn't understand which one performs better? </p>\n<p>Thanks. </p>",
  "messages": [
    {
      "id": "1120264",
      "postDate": "12/20/2020 17:44:26",
      "content": "<p>Hi everyone,</p>\n<p>I wonder that which metric do you track to save your best models? Loss, accuracy or maybe f1… Or do you save the model at every epoch and use some kind of validation scheme to choose which models to use in inference?</p>\n<p>I currently use loss and accuracy but have some doubts since I still couldn't understand which one performs better? </p>\n<p>Thanks. </p>",
      "rawMarkdown": "Hi everyone,\n\nI wonder that which metric do you track to save your best models? Loss, accuracy or maybe f1... Or do you save the model at every epoch and use some kind of validation scheme to choose which models to use in inference?\n\nI currently use loss and accuracy but have some doubts since I still couldn't understand which one performs better? \n\nThanks.",
      "votes": null
    },
    {
      "id": "1120925",
      "postDate": "12/21/2020 07:57:37",
      "content": "<p>Rankings are given by taking categorization accuracy. So I think keeping track of categorization accuracy is good idea.</p>",
      "rawMarkdown": "Rankings are given by taking categorization accuracy. So I think keeping track of categorization accuracy is good idea.",
      "votes": null
    },
    {
      "id": "1121322",
      "postDate": "12/21/2020 14:52:30",
      "content": "<p>Use the same metric mentioned in the competition i.e. <a href=\"https://developers.google.com/machine-learning/crash-course/classification/accuracy\" target=\"_blank\">categorization accuracy</a> to resonate with the competitions leaderboard…</p>",
      "rawMarkdown": "Use the same metric mentioned in the competition i.e. [categorization accuracy](https://developers.google.com/machine-learning/crash-course/classification/accuracy) to resonate with the competitions leaderboard...",
      "votes": null
    },
    {
      "id": "1124328",
      "postDate": "12/23/2020 20:27:26",
      "content": "<p>I actually do but as I said I'm just confused.. I currently keep the best models regards to loss as well. I will try to understand with some submissions.</p>",
      "rawMarkdown": "I actually do but as I said I'm just confused.. I currently keep the best models regards to loss as well. I will try to understand with some submissions.",
      "votes": null
    },
    {
      "id": "1124330",
      "postDate": "12/23/2020 20:28:55",
      "content": "<p>As I said, I currently use accuracy but I'm just confused.. I currently keep the best models regards to loss as well. I will do some experiments with this setup and compare the results.</p>",
      "rawMarkdown": "As I said, I currently use accuracy but I'm just confused.. I currently keep the best models regards to loss as well. I will do some experiments with this setup and compare the results.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1120925,
      "author_name": "shanmukh05",
      "author_url": "",
      "post_date": "12/21/2020 07:57:37",
      "content": "<p>Rankings are given by taking categorization accuracy. So I think keeping track of categorization accuracy is good idea.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1124330,
          "author_name": "snnclsr",
          "author_url": "",
          "post_date": "12/23/2020 20:28:55",
          "content": "<p>As I said, I currently use accuracy but I'm just confused.. I currently keep the best models regards to loss as well. I will do some experiments with this setup and compare the results.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1121322,
      "author_name": "atharvaingle",
      "author_url": "",
      "post_date": "12/21/2020 14:52:30",
      "content": "<p>Use the same metric mentioned in the competition i.e. <a href=\"https://developers.google.com/machine-learning/crash-course/classification/accuracy\" target=\"_blank\">categorization accuracy</a> to resonate with the competitions leaderboard…</p>",
      "votes": null,
      "replies": [
        {
          "id": 1124328,
          "author_name": "snnclsr",
          "author_url": "",
          "post_date": "12/23/2020 20:27:26",
          "content": "<p>I actually do but as I said I'm just confused.. I currently keep the best models regards to loss as well. I will try to understand with some submissions.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1120264": "Hi everyone,\n\nI wonder that which metric do you track to save your best models? Loss, accuracy or maybe f1... Or do you save the model at every epoch and use some kind of validation scheme to choose which models to use in inference?\n\nI currently use loss and accuracy but have some doubts since I still couldn't understand which one performs better? \n\nThanks.",
    "1120925": "Rankings are given by taking categorization accuracy. So I think keeping track of categorization accuracy is good idea.",
    "1121322": "Use the same metric mentioned in the competition i.e. [categorization accuracy](https://developers.google.com/machine-learning/crash-course/classification/accuracy) to resonate with the competitions leaderboard...",
    "1124328": "I actually do but as I said I'm just confused.. I currently keep the best models regards to loss as well. I will try to understand with some submissions.",
    "1124330": "As I said, I currently use accuracy but I'm just confused.. I currently keep the best models regards to loss as well. I will do some experiments with this setup and compare the results."
  },
  "source": "meta"
}