{
  "id": 230099,
  "title": "Strange, best model, cv and lb?",
  "url": "/competitions/plant-pathology-2021-fgvc8/discussion/230099",
  "author_name": "",
  "post_date": "2021-04-02T00:20:49.369734600Z",
  "votes": 8,
  "comment_count": 6,
  "views": 0,
  "content": "<p>se_resnext50_32x4d no TTA cv:80% lb: 72.8% single fold<br>\nsame model as last line: TTA:3 cv:80% lb:70.2% single fold<br>\nefficientnet b0 no TTA cv&gt;94% lb:52% single fold<br>\nse_resnext50_32x4d no TTA cv:93% lb: 60.0% single fold.</p>\n<p>It's strange that better CV doesn't mean better lb. I'm lost. Quite a large discrepancy between the two scores.<br>\nIf it's not a reasonable reason, I may have to give up. <br>\nDo you have any suggestions？</p>",
  "messages": [
    {
      "id": "1260229",
      "postDate": "04/02/2021 00:20:49",
      "content": "<p>se_resnext50_32x4d no TTA cv:80% lb: 72.8% single fold<br>\nsame model as last line: TTA:3 cv:80% lb:70.2% single fold<br>\nefficientnet b0 no TTA cv&gt;94% lb:52% single fold<br>\nse_resnext50_32x4d no TTA cv:93% lb: 60.0% single fold.</p>\n<p>It's strange that better CV doesn't mean better lb. I'm lost. Quite a large discrepancy between the two scores.<br>\nIf it's not a reasonable reason, I may have to give up. <br>\nDo you have any suggestions？</p>",
      "rawMarkdown": "se_resnext50_32x4d no TTA cv:80% lb: 72.8% single fold\nsame model as last line: TTA:3 cv:80% lb:70.2% single fold\nefficientnet b0 no TTA cv>94% lb:52% single fold\nse_resnext50_32x4d no TTA cv:93% lb: 60.0% single fold.\n\nIt's strange that better CV doesn't mean better lb. I'm lost. Quite a large discrepancy between the two scores.\nIf it's not a reasonable reason, I may have to give up. \nDo you have any suggestions？",
      "votes": null
    },
    {
      "id": "1260564",
      "postDate": "04/02/2021 08:22:37",
      "content": "<p>Same situation here. Very large gap between CV and LB, CV does not correlate with LB.</p>",
      "rawMarkdown": "Same situation here. Very large gap between CV and LB, CV does not correlate with LB.",
      "votes": null
    },
    {
      "id": "1260618",
      "postDate": "04/02/2021 09:12:13",
      "content": "<p>I wonder if there exists different conbination of the 6 base classes from the 12 classes in the Pub test dataset.</p>",
      "rawMarkdown": "I wonder if there exists different conbination of the 6 base classes from the 12 classes in the Pub test dataset.",
      "votes": null
    },
    {
      "id": "1262113",
      "postDate": "04/03/2021 19:50:07",
      "content": "<p>I also observe the same thing.</p>",
      "rawMarkdown": "I also observe the same thing.",
      "votes": null
    },
    {
      "id": "1263243",
      "postDate": "04/05/2021 08:32:01",
      "content": "<p>Well, I have tried with Multi label classification with only 6 labels. But I am also facing similar issue.</p>",
      "rawMarkdown": "Well, I have tried with Multi label classification with only 6 labels. But I am also facing similar issue.",
      "votes": null
    },
    {
      "id": "1264264",
      "postDate": "04/06/2021 03:22:34",
      "content": "<p>Maybe there is a inconsistency between Training and Testing data. I got the same result as you did.</p>",
      "rawMarkdown": "Maybe there is a inconsistency between Training and Testing data. I got the same result as you did.",
      "votes": null
    },
    {
      "id": "1265557",
      "postDate": "04/07/2021 02:46:25",
      "content": "<p>Wow, a 13% increase in local CV score results in a 13% drop on the LB score…</p>\n<p>I have been wondering if the score takes into account how accurate each label category is relative to the other labels and calculates score with somesort of \"error punishment.\"?? Maybe?</p>",
      "rawMarkdown": "Wow, a 13% increase in local CV score results in a 13% drop on the LB score...\n\nI have been wondering if the score takes into account how accurate each label category is relative to the other labels and calculates score with somesort of \"error punishment.\"?? Maybe?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1260564,
      "author_name": "opanichev",
      "author_url": "",
      "post_date": "04/02/2021 08:22:37",
      "content": "<p>Same situation here. Very large gap between CV and LB, CV does not correlate with LB.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1260618,
      "author_name": "eynaij",
      "author_url": "",
      "post_date": "04/02/2021 09:12:13",
      "content": "<p>I wonder if there exists different conbination of the 6 base classes from the 12 classes in the Pub test dataset.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1263243,
          "author_name": "shanmukh05",
          "author_url": "",
          "post_date": "04/05/2021 08:32:01",
          "content": "<p>Well, I have tried with Multi label classification with only 6 labels. But I am also facing similar issue.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1262113,
      "author_name": "atamazian",
      "author_url": "",
      "post_date": "04/03/2021 19:50:07",
      "content": "<p>I also observe the same thing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1264264,
      "author_name": "crissallan",
      "author_url": "",
      "post_date": "04/06/2021 03:22:34",
      "content": "<p>Maybe there is a inconsistency between Training and Testing data. I got the same result as you did.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1265557,
      "author_name": "brendanartley",
      "author_url": "",
      "post_date": "04/07/2021 02:46:25",
      "content": "<p>Wow, a 13% increase in local CV score results in a 13% drop on the LB score…</p>\n<p>I have been wondering if the score takes into account how accurate each label category is relative to the other labels and calculates score with somesort of \"error punishment.\"?? Maybe?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1260229": "se_resnext50_32x4d no TTA cv:80% lb: 72.8% single fold\nsame model as last line: TTA:3 cv:80% lb:70.2% single fold\nefficientnet b0 no TTA cv>94% lb:52% single fold\nse_resnext50_32x4d no TTA cv:93% lb: 60.0% single fold.\n\nIt's strange that better CV doesn't mean better lb. I'm lost. Quite a large discrepancy between the two scores.\nIf it's not a reasonable reason, I may have to give up. \nDo you have any suggestions？",
    "1260564": "Same situation here. Very large gap between CV and LB, CV does not correlate with LB.",
    "1260618": "I wonder if there exists different conbination of the 6 base classes from the 12 classes in the Pub test dataset.",
    "1262113": "I also observe the same thing.",
    "1263243": "Well, I have tried with Multi label classification with only 6 labels. But I am also facing similar issue.",
    "1264264": "Maybe there is a inconsistency between Training and Testing data. I got the same result as you did.",
    "1265557": "Wow, a 13% increase in local CV score results in a 13% drop on the LB score...\n\nI have been wondering if the score takes into account how accurate each label category is relative to the other labels and calculates score with somesort of \"error punishment.\"?? Maybe?"
  },
  "source": "meta"
}