{
  "id": 217193,
  "title": "How do you estimate your models?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/217193",
  "author_name": "",
  "post_date": "2021-02-05T17:58:24.612474100Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>There are two ways to estimate models: according validation loss or according validation metrics. I used the first one during searching for the best optimizer and the rest of the time I've been using the second one. But recently I noticed that the correlation is better if models are estimated according validation loss. Which one do you use to get better correlation? </p>",
  "messages": [
    {
      "id": "1187786",
      "postDate": "02/05/2021 17:58:24",
      "content": "<p>There are two ways to estimate models: according validation loss or according validation metrics. I used the first one during searching for the best optimizer and the rest of the time I've been using the second one. But recently I noticed that the correlation is better if models are estimated according validation loss. Which one do you use to get better correlation? </p>",
      "rawMarkdown": "There are two ways to estimate models: according validation loss or according validation metrics. I used the first one during searching for the best optimizer and the rest of the time I've been using the second one. But recently I noticed that the correlation is better if models are estimated according validation loss. Which one do you use to get better correlation?",
      "votes": null
    },
    {
      "id": "1187918",
      "postDate": "02/05/2021 18:51:35",
      "content": "<p>Most time I use validation metrics.<br>\nAre the <code>correlation is better</code> you mention means offline validation and online leaderboard? It seems that the Public LB is easier than the Local validation, And the best auc is appearing after best loss with certain overfit in my setup. So I suggest this <code>correlation</code> is specific to this dataset and not the general situation.<br>\nI suggest that our aim should be provide models with higher performance and generalization ability rather than that correlation only.</p>",
      "rawMarkdown": "Most time I use validation metrics.\nAre the `correlation is better` you mention means offline validation and online leaderboard? It seems that the Public LB is easier than the Local validation, And the best auc is appearing after best loss with certain overfit in my setup. So I suggest this `correlation` is specific to this dataset and not the general situation.\nI suggest that our aim should be provide models with higher performance and generalization ability rather than that correlation only.",
      "votes": null
    },
    {
      "id": "1187956",
      "postDate": "02/05/2021 19:32:04",
      "content": "<p>Yeah, I mean this. I agree that the real local results is better than just a correlation - anyway \"Trust your CV\". It's a kind of more generic question.</p>",
      "rawMarkdown": "Yeah, I mean this. I agree that the real local results is better than just a correlation - anyway \"Trust your CV\". It's a kind of more generic question.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1187918,
      "author_name": "steamedsheep",
      "author_url": "",
      "post_date": "02/05/2021 18:51:35",
      "content": "<p>Most time I use validation metrics.<br>\nAre the <code>correlation is better</code> you mention means offline validation and online leaderboard? It seems that the Public LB is easier than the Local validation, And the best auc is appearing after best loss with certain overfit in my setup. So I suggest this <code>correlation</code> is specific to this dataset and not the general situation.<br>\nI suggest that our aim should be provide models with higher performance and generalization ability rather than that correlation only.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1187956,
          "author_name": "vadimtimakin",
          "author_url": "",
          "post_date": "02/05/2021 19:32:04",
          "content": "<p>Yeah, I mean this. I agree that the real local results is better than just a correlation - anyway \"Trust your CV\". It's a kind of more generic question.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1187786": "There are two ways to estimate models: according validation loss or according validation metrics. I used the first one during searching for the best optimizer and the rest of the time I've been using the second one. But recently I noticed that the correlation is better if models are estimated according validation loss. Which one do you use to get better correlation?",
    "1187918": "Most time I use validation metrics.\nAre the `correlation is better` you mention means offline validation and online leaderboard? It seems that the Public LB is easier than the Local validation, And the best auc is appearing after best loss with certain overfit in my setup. So I suggest this `correlation` is specific to this dataset and not the general situation.\nI suggest that our aim should be provide models with higher performance and generalization ability rather than that correlation only.",
    "1187956": "Yeah, I mean this. I agree that the real local results is better than just a correlation - anyway \"Trust your CV\". It's a kind of more generic question."
  },
  "source": "meta"
}