{
  "id": 216902,
  "title": "Wrong labels but better CV",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/216902",
  "author_name": "",
  "post_date": "2021-02-04T13:43:24.821899Z",
  "votes": 22,
  "comment_count": 27,
  "views": 0,
  "content": "<p>This is a weird competition. I made some mistakes in my code, which changed about 2.7% labels to class 0, then trained the model on the incorrect label. After that I did the inference on the correct train dataset but the CV still improved 0.001. This is very strange.</p>",
  "messages": [
    {
      "id": "1185924",
      "postDate": "02/04/2021 13:43:24",
      "content": "<p>This is a weird competition. I made some mistakes in my code, which changed about 2.7% labels to class 0, then trained the model on the incorrect label. After that I did the inference on the correct train dataset but the CV still improved 0.001. This is very strange.</p>",
      "rawMarkdown": "This is a weird competition. I made some mistakes in my code, which changed about 2.7% labels to class 0, then trained the model on the incorrect label. After that I did the inference on the correct train dataset but the CV still improved 0.001. This is very strange.",
      "votes": null
    },
    {
      "id": "1186029",
      "postDate": "02/04/2021 14:48:35",
      "content": "<p>Wrong image size but better LB. I met a similar thing, forgot to change the inference size but improved.</p>",
      "rawMarkdown": "Wrong image size but better LB. I met a similar thing, forgot to change the inference size but improved.",
      "votes": null
    },
    {
      "id": "1186140",
      "postDate": "02/04/2021 15:51:15",
      "content": "<p>Very strange!</p>",
      "rawMarkdown": "Very strange!",
      "votes": null
    },
    {
      "id": "1186147",
      "postDate": "02/04/2021 15:55:38",
      "content": "<p>Aren't changes by maybe 0.001 not just likely to be  coincidence, noise, random stuff? Especially regarding that there are already many mislabeled images, adding some more won't make a big difference?</p>",
      "rawMarkdown": "Aren't changes by maybe 0.001 not just likely to be  coincidence, noise, random stuff? Especially regarding that there are already many mislabeled images, adding some more won't make a big difference?",
      "votes": null
    },
    {
      "id": "1186158",
      "postDate": "02/04/2021 16:06:43",
      "content": "<p>I agree, but if I corrected the error to what I intended to do, the CV dropped further and the gap increased to 0.003.</p>",
      "rawMarkdown": "I agree, but if I corrected the error to what I intended to do, the CV dropped further and the gap increased to 0.003.",
      "votes": null
    },
    {
      "id": "1186164",
      "postDate": "02/04/2021 16:09:40",
      "content": "<p>so maybe this means that the thing you're intending to do won't affect your CV in a positive way? that's probably true for many things people try out :)</p>",
      "rawMarkdown": "so maybe this means that the thing you're intending to do won't affect your CV in a positive way? that's probably true for many things people try out :)",
      "votes": null
    },
    {
      "id": "1186393",
      "postDate": "02/04/2021 19:47:42",
      "content": "<p>3 0.900lb blend -&gt; 0.898lb😰</p>",
      "rawMarkdown": "3 0.900lb blend -> 0.898lb😰",
      "votes": null
    },
    {
      "id": "1186415",
      "postDate": "02/04/2021 20:11:33",
      "content": "<blockquote>\n  <p>3 0.900lb blend -&gt; 0.898lb😰</p>\n</blockquote>\n<p>?</p>",
      "rawMarkdown": "> 3 0.900lb blend -> 0.898lb😰\n\n?",
      "votes": null
    },
    {
      "id": "1186445",
      "postDate": "02/04/2021 20:30:36",
      "content": "<p>I think he means he has 3 results which have 0.900LB, but ensemble them got 0.898</p>",
      "rawMarkdown": "I think he means he has 3 results which have 0.900LB, but ensemble them got 0.898",
      "votes": null
    },
    {
      "id": "1186455",
      "postDate": "02/04/2021 20:38:05",
      "content": "<p>ah ok. Yes what i can say is, that i also experience strange stuff going on, like CV 0.900 -&gt; LB 0.895 and CV 0.895 -&gt; LB 0.901 and these blends that don't make sense</p>",
      "rawMarkdown": "ah ok. Yes what i can say is, that i also experience strange stuff going on, like CV 0.900 -> LB 0.895 and CV 0.895 -> LB 0.901 and these blends that don't make sense",
      "votes": null
    },
    {
      "id": "1186584",
      "postDate": "02/04/2021 22:59:47",
      "content": "<p>I have the same experience and very confused, don’t know what should I trust, CV or LB or neither.</p>",
      "rawMarkdown": "I have the same experience and very confused, don’t know what should I trust, CV or LB or neither.",
      "votes": null
    },
    {
      "id": "1186598",
      "postDate": "02/04/2021 23:23:15",
      "content": "<p>Same here.. I had 5 fold ensemble models got 0.904cv but only 0.895LB..also lower cv got better LB.<br>\nI think I'm going to keep the same strategy as past competitions, select 2 submissions with 1 best CV and 1 best LB.</p>",
      "rawMarkdown": "Same here.. I had 5 fold ensemble models got 0.904cv but only 0.895LB..also lower cv got better LB.\nI think I'm going to keep the same strategy as past competitions, select 2 submissions with 1 best CV and 1 best LB.",
      "votes": null
    },
    {
      "id": "1186678",
      "postDate": "02/05/2021 01:41:55",
      "content": "<p>Maybe caused by noise. The CV vs LB is very strange here.</p>",
      "rawMarkdown": "Maybe caused by noise. The CV vs LB is very strange here.",
      "votes": null
    },
    {
      "id": "1186681",
      "postDate": "02/05/2021 01:44:51",
      "content": "<p>0.001 public LB stand for merely 4 images, It is not that stable.</p>",
      "rawMarkdown": "0.001 public LB stand for merely 4 images, It is not that stable.",
      "votes": null
    },
    {
      "id": "1186782",
      "postDate": "02/05/2021 03:21:51",
      "content": "<p>If your cv is high, the model bias maybe small, but big variance, because the data exist noise, and test also exist noise, i think all we can do is early stop, so it also explain why some low cv gives better lb. I think in this compete, we can't believe CV and LB👀</p>",
      "rawMarkdown": "If your cv is high, the model bias maybe small, but big variance, because the data exist noise, and test also exist noise, i think all we can do is early stop, so it also explain why some low cv gives better lb. I think in this compete, we can't believe CV and LB👀",
      "votes": null
    },
    {
      "id": "1186786",
      "postDate": "02/05/2021 03:27:57",
      "content": "<p>I think shake will coming🙄🙄</p>",
      "rawMarkdown": "I think shake will coming🙄🙄",
      "votes": null
    },
    {
      "id": "1187177",
      "postDate": "02/05/2021 09:01:20",
      "content": "<p>Similar things for me. I run inference with tempered softmax activation function using t=2.0 (same used during training) and got LB score 0.892, while using t=1.0 (different from the one used at training time) my LB is 0.900 (+0.008)</p>",
      "rawMarkdown": "Similar things for me. I run inference with tempered softmax activation function using t=2.0 (same used during training) and got LB score 0.892, while using t=1.0 (different from the one used at training time) my LB is 0.900 (+0.008)",
      "votes": null
    },
    {
      "id": "1187187",
      "postDate": "02/05/2021 09:16:51",
      "content": "<p>I more and more get the feeling, that it's a big portion of luck. Everyday i read of people here saying this works better for them whereas others claim the opposite works better. Except from the obvious stuff like ensembling, tta and including effnets in a way</p>",
      "rawMarkdown": "I more and more get the feeling, that it's a big portion of luck. Everyday i read of people here saying this works better for them whereas others claim the opposite works better. Except from the obvious stuff like ensembling, tta and including effnets in a way",
      "votes": null
    },
    {
      "id": "1187194",
      "postDate": "02/05/2021 09:26:21",
      "content": "<p>The images in lb only counts 29%*15000 , about 5k images. 0.001 in lb only 5 images.<br>\nIs it right?</p>",
      "rawMarkdown": "The images in lb only counts 29%*15000 , about 5k images. 0.001 in lb only 5 images.\nIs it right?",
      "votes": null
    },
    {
      "id": "1187202",
      "postDate": "02/05/2021 09:32:55",
      "content": "<p>I got similar weird improvement (setting by mistake a parameter to a bad value) in my local CV too. My point here is that I have better results with hyperparameters set to wrong values. Usually I expect worse performance being the model trained with different settings, not better (even of \"only\" 0.003/0.005).</p>",
      "rawMarkdown": "I got similar weird improvement (setting by mistake a parameter to a bad value) in my local CV too. My point here is that I have better results with hyperparameters set to wrong values. Usually I expect worse performance being the model trained with different settings, not better (even of \"only\" 0.003/0.005).",
      "votes": null
    },
    {
      "id": "1187257",
      "postDate": "02/05/2021 10:15:45",
      "content": "<p><a href=\"https://www.kaggle.com/zekunn\" target=\"_blank\">@zekunn</a> yes i guess thats right. so following this logic it's actually sad knowing that whats separate you from the first place is maybe 20 images of roots of people or smartphones in the dataset being classified wrong :D</p>",
      "rawMarkdown": "zekunn yes i guess thats right. so following this logic it's actually sad knowing that whats separate you from the first place is maybe 20 images of roots of people or smartphones in the dataset being classified wrong :D",
      "votes": null
    },
    {
      "id": "1187387",
      "postDate": "02/05/2021 11:53:14",
      "content": "<p><a href=\"https://www.kaggle.com/lazcoder\" target=\"_blank\">@lazcoder</a> 0.008 is a big jump. I have same feeling, this competition will be lottery!</p>",
      "rawMarkdown": "lazcoder 0.008 is a big jump. I have same feeling, this competition will be lottery!",
      "votes": null
    },
    {
      "id": "1187462",
      "postDate": "02/05/2021 13:09:44",
      "content": "<p>In my case,</p>\n<p>model1: CV 0.8912  LB 0.900<br>\nmodel2: CV 0.8942 LB 0.894<br>\nmodel3: CV 0.8951 LB 0.900</p>\n<p>ensemble of 5 models: CV 0.9023 LB 0.905<br>\nensemble of 4 models: CV 0.9013 LB 0.906<br>\nensemble of 5 models(weighted averaging): CV 0.903 LB 0.904</p>\n<p>I feel like cv and lb don't correlate well.<br>\nThe distribution of the data may be different between the test data and the training data.<br>\nMaybe the percentage of noisy data is also different.<br>\nHowever, the data for the public LB is only 4,500, so the difference may be insignificant.<br>\nI feel that most of the participants are probably clustered in a very small range, and a big shakeup might happen.</p>",
      "rawMarkdown": "In my case,\n\nmodel1: CV 0.8912  LB 0.900\nmodel2: CV 0.8942 LB 0.894\nmodel3: CV 0.8951 LB 0.900\n\nensemble of 5 models: CV 0.9023 LB 0.905\nensemble of 4 models: CV 0.9013 LB 0.906\nensemble of 5 models(weighted averaging): CV 0.903 LB 0.904\n\nI feel like cv and lb don't correlate well.\nThe distribution of the data may be different between the test data and the training data.\nMaybe the percentage of noisy data is also different.\nHowever, the data for the public LB is only 4,500, so the difference may be insignificant.\nI feel that most of the participants are probably clustered in a very small range, and a big shakeup might happen.",
      "votes": null
    },
    {
      "id": "1188163",
      "postDate": "02/06/2021 02:48:22",
      "content": "<p>Same experience, better CV but lower LB, lower CV but higher LB. </p>",
      "rawMarkdown": "Same experience, better CV but lower LB, lower CV but higher LB.",
      "votes": null
    },
    {
      "id": "1189804",
      "postDate": "02/07/2021 08:40:01",
      "content": "<p>Huge shake will coming 😑</p>",
      "rawMarkdown": "Huge shake will coming 😑",
      "votes": null
    },
    {
      "id": "1191481",
      "postDate": "02/08/2021 13:36:51",
      "content": "<p>Well maybe the validation data is biased toward the label you mistook, that way the result has improved but without any model improvement ?</p>",
      "rawMarkdown": "Well maybe the validation data is biased toward the label you mistook, that way the result has improved but without any model improvement ?",
      "votes": null
    },
    {
      "id": "1192275",
      "postDate": "02/09/2021 04:24:14",
      "content": "<p>how submit my notebook?</p>",
      "rawMarkdown": "how submit my notebook?",
      "votes": null
    },
    {
      "id": "1197078",
      "postDate": "02/11/2021 23:18:50",
      "content": "<p>I met the same thing, trained with 512 and submit with 384, the LB score increased from 0.901 to 0.902, but when we use it for ensemble ,the LB score decreased.</p>",
      "rawMarkdown": "I met the same thing, trained with 512 and submit with 384, the LB score increased from 0.901 to 0.902, but when we use it for ensemble ,the LB score decreased.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1186029,
      "author_name": "zlanan",
      "author_url": "",
      "post_date": "02/04/2021 14:48:35",
      "content": "<p>Wrong image size but better LB. I met a similar thing, forgot to change the inference size but improved.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1186140,
          "author_name": "lftuwujie",
          "author_url": "",
          "post_date": "02/04/2021 15:51:15",
          "content": "<p>Very strange!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1197078,
          "author_name": "librauee",
          "author_url": "",
          "post_date": "02/11/2021 23:18:50",
          "content": "<p>I met the same thing, trained with 512 and submit with 384, the LB score increased from 0.901 to 0.902, but when we use it for ensemble ,the LB score decreased.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1186147,
      "author_name": "alexanderriedel",
      "author_url": "",
      "post_date": "02/04/2021 15:55:38",
      "content": "<p>Aren't changes by maybe 0.001 not just likely to be  coincidence, noise, random stuff? Especially regarding that there are already many mislabeled images, adding some more won't make a big difference?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1186158,
          "author_name": "lftuwujie",
          "author_url": "",
          "post_date": "02/04/2021 16:06:43",
          "content": "<p>I agree, but if I corrected the error to what I intended to do, the CV dropped further and the gap increased to 0.003.</p>",
          "votes": null,
          "replies": [
            {
              "id": 1186164,
              "author_name": "alexanderriedel",
              "author_url": "",
              "post_date": "02/04/2021 16:09:40",
              "content": "<p>so maybe this means that the thing you're intending to do won't affect your CV in a positive way? that's probably true for many things people try out :)</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 1186393,
      "author_name": "underwearfitting",
      "author_url": "",
      "post_date": "02/04/2021 19:47:42",
      "content": "<p>3 0.900lb blend -&gt; 0.898lb😰</p>",
      "votes": null,
      "replies": [
        {
          "id": 1186415,
          "author_name": "alexanderriedel",
          "author_url": "",
          "post_date": "02/04/2021 20:11:33",
          "content": "<blockquote>\n  <p>3 0.900lb blend -&gt; 0.898lb😰</p>\n</blockquote>\n<p>?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186445,
          "author_name": "lftuwujie",
          "author_url": "",
          "post_date": "02/04/2021 20:30:36",
          "content": "<p>I think he means he has 3 results which have 0.900LB, but ensemble them got 0.898</p>",
          "votes": null,
          "replies": [
            {
              "id": 1186455,
              "author_name": "alexanderriedel",
              "author_url": "",
              "post_date": "02/04/2021 20:38:05",
              "content": "<p>ah ok. Yes what i can say is, that i also experience strange stuff going on, like CV 0.900 -&gt; LB 0.895 and CV 0.895 -&gt; LB 0.901 and these blends that don't make sense</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 1186584,
          "author_name": "lftuwujie",
          "author_url": "",
          "post_date": "02/04/2021 22:59:47",
          "content": "<p>I have the same experience and very confused, don’t know what should I trust, CV or LB or neither.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186598,
          "author_name": "xiejialun",
          "author_url": "",
          "post_date": "02/04/2021 23:23:15",
          "content": "<p>Same here.. I had 5 fold ensemble models got 0.904cv but only 0.895LB..also lower cv got better LB.<br>\nI think I'm going to keep the same strategy as past competitions, select 2 submissions with 1 best CV and 1 best LB.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186678,
          "author_name": "steamedsheep",
          "author_url": "",
          "post_date": "02/05/2021 01:41:55",
          "content": "<p>Maybe caused by noise. The CV vs LB is very strange here.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186681,
          "author_name": "steamedsheep",
          "author_url": "",
          "post_date": "02/05/2021 01:44:51",
          "content": "<p>0.001 public LB stand for merely 4 images, It is not that stable.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186782,
          "author_name": "cswwp347724",
          "author_url": "",
          "post_date": "02/05/2021 03:21:51",
          "content": "<p>If your cv is high, the model bias maybe small, but big variance, because the data exist noise, and test also exist noise, i think all we can do is early stop, so it also explain why some low cv gives better lb. I think in this compete, we can't believe CV and LB👀</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1187462,
          "author_name": "yosukeyama",
          "author_url": "",
          "post_date": "02/05/2021 13:09:44",
          "content": "<p>In my case,</p>\n<p>model1: CV 0.8912  LB 0.900<br>\nmodel2: CV 0.8942 LB 0.894<br>\nmodel3: CV 0.8951 LB 0.900</p>\n<p>ensemble of 5 models: CV 0.9023 LB 0.905<br>\nensemble of 4 models: CV 0.9013 LB 0.906<br>\nensemble of 5 models(weighted averaging): CV 0.903 LB 0.904</p>\n<p>I feel like cv and lb don't correlate well.<br>\nThe distribution of the data may be different between the test data and the training data.<br>\nMaybe the percentage of noisy data is also different.<br>\nHowever, the data for the public LB is only 4,500, so the difference may be insignificant.<br>\nI feel that most of the participants are probably clustered in a very small range, and a big shakeup might happen.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1188163,
          "author_name": "fuxungao",
          "author_url": "",
          "post_date": "02/06/2021 02:48:22",
          "content": "<p>Same experience, better CV but lower LB, lower CV but higher LB. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1186786,
      "author_name": "cswwp347724",
      "author_url": "",
      "post_date": "02/05/2021 03:27:57",
      "content": "<p>I think shake will coming🙄🙄</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1187177,
      "author_name": "lazcoder",
      "author_url": "",
      "post_date": "02/05/2021 09:01:20",
      "content": "<p>Similar things for me. I run inference with tempered softmax activation function using t=2.0 (same used during training) and got LB score 0.892, while using t=1.0 (different from the one used at training time) my LB is 0.900 (+0.008)</p>",
      "votes": null,
      "replies": [
        {
          "id": 1187187,
          "author_name": "alexanderriedel",
          "author_url": "",
          "post_date": "02/05/2021 09:16:51",
          "content": "<p>I more and more get the feeling, that it's a big portion of luck. Everyday i read of people here saying this works better for them whereas others claim the opposite works better. Except from the obvious stuff like ensembling, tta and including effnets in a way</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1187194,
          "author_name": "zekunn",
          "author_url": "",
          "post_date": "02/05/2021 09:26:21",
          "content": "<p>The images in lb only counts 29%*15000 , about 5k images. 0.001 in lb only 5 images.<br>\nIs it right?</p>",
          "votes": null,
          "replies": [
            {
              "id": 1187257,
              "author_name": "alexanderriedel",
              "author_url": "",
              "post_date": "02/05/2021 10:15:45",
              "content": "<p><a href=\"https://www.kaggle.com/zekunn\" target=\"_blank\">@zekunn</a> yes i guess thats right. so following this logic it's actually sad knowing that whats separate you from the first place is maybe 20 images of roots of people or smartphones in the dataset being classified wrong :D</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 1187202,
          "author_name": "lazcoder",
          "author_url": "",
          "post_date": "02/05/2021 09:32:55",
          "content": "<p>I got similar weird improvement (setting by mistake a parameter to a bad value) in my local CV too. My point here is that I have better results with hyperparameters set to wrong values. Usually I expect worse performance being the model trained with different settings, not better (even of \"only\" 0.003/0.005).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1187387,
          "author_name": "lftuwujie",
          "author_url": "",
          "post_date": "02/05/2021 11:53:14",
          "content": "<p><a href=\"https://www.kaggle.com/lazcoder\" target=\"_blank\">@lazcoder</a> 0.008 is a big jump. I have same feeling, this competition will be lottery!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1189804,
      "author_name": "raininbox",
      "author_url": "",
      "post_date": "02/07/2021 08:40:01",
      "content": "<p>Huge shake will coming 😑</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1191481,
      "author_name": "omarmohamed22",
      "author_url": "",
      "post_date": "02/08/2021 13:36:51",
      "content": "<p>Well maybe the validation data is biased toward the label you mistook, that way the result has improved but without any model improvement ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1192275,
      "author_name": "girishrane2409",
      "author_url": "",
      "post_date": "02/09/2021 04:24:14",
      "content": "<p>how submit my notebook?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1185924": "This is a weird competition. I made some mistakes in my code, which changed about 2.7% labels to class 0, then trained the model on the incorrect label. After that I did the inference on the correct train dataset but the CV still improved 0.001. This is very strange.",
    "1186029": "Wrong image size but better LB. I met a similar thing, forgot to change the inference size but improved.",
    "1186140": "Very strange!",
    "1186147": "Aren't changes by maybe 0.001 not just likely to be  coincidence, noise, random stuff? Especially regarding that there are already many mislabeled images, adding some more won't make a big difference?",
    "1186158": "I agree, but if I corrected the error to what I intended to do, the CV dropped further and the gap increased to 0.003.",
    "1186164": "so maybe this means that the thing you're intending to do won't affect your CV in a positive way? that's probably true for many things people try out :)",
    "1186393": "3 0.900lb blend -> 0.898lb😰",
    "1186415": "> 3 0.900lb blend -> 0.898lb😰\n\n?",
    "1186445": "I think he means he has 3 results which have 0.900LB, but ensemble them got 0.898",
    "1186455": "ah ok. Yes what i can say is, that i also experience strange stuff going on, like CV 0.900 -> LB 0.895 and CV 0.895 -> LB 0.901 and these blends that don't make sense",
    "1186584": "I have the same experience and very confused, don’t know what should I trust, CV or LB or neither.",
    "1186598": "Same here.. I had 5 fold ensemble models got 0.904cv but only 0.895LB..also lower cv got better LB.\nI think I'm going to keep the same strategy as past competitions, select 2 submissions with 1 best CV and 1 best LB.",
    "1186678": "Maybe caused by noise. The CV vs LB is very strange here.",
    "1186681": "0.001 public LB stand for merely 4 images, It is not that stable.",
    "1186782": "If your cv is high, the model bias maybe small, but big variance, because the data exist noise, and test also exist noise, i think all we can do is early stop, so it also explain why some low cv gives better lb. I think in this compete, we can't believe CV and LB👀",
    "1186786": "I think shake will coming🙄🙄",
    "1187177": "Similar things for me. I run inference with tempered softmax activation function using t=2.0 (same used during training) and got LB score 0.892, while using t=1.0 (different from the one used at training time) my LB is 0.900 (+0.008)",
    "1187187": "I more and more get the feeling, that it's a big portion of luck. Everyday i read of people here saying this works better for them whereas others claim the opposite works better. Except from the obvious stuff like ensembling, tta and including effnets in a way",
    "1187194": "The images in lb only counts 29%*15000 , about 5k images. 0.001 in lb only 5 images.\nIs it right?",
    "1187202": "I got similar weird improvement (setting by mistake a parameter to a bad value) in my local CV too. My point here is that I have better results with hyperparameters set to wrong values. Usually I expect worse performance being the model trained with different settings, not better (even of \"only\" 0.003/0.005).",
    "1187257": "zekunn yes i guess thats right. so following this logic it's actually sad knowing that whats separate you from the first place is maybe 20 images of roots of people or smartphones in the dataset being classified wrong :D",
    "1187387": "lazcoder 0.008 is a big jump. I have same feeling, this competition will be lottery!",
    "1187462": "In my case,\n\nmodel1: CV 0.8912  LB 0.900\nmodel2: CV 0.8942 LB 0.894\nmodel3: CV 0.8951 LB 0.900\n\nensemble of 5 models: CV 0.9023 LB 0.905\nensemble of 4 models: CV 0.9013 LB 0.906\nensemble of 5 models(weighted averaging): CV 0.903 LB 0.904\n\nI feel like cv and lb don't correlate well.\nThe distribution of the data may be different between the test data and the training data.\nMaybe the percentage of noisy data is also different.\nHowever, the data for the public LB is only 4,500, so the difference may be insignificant.\nI feel that most of the participants are probably clustered in a very small range, and a big shakeup might happen.",
    "1188163": "Same experience, better CV but lower LB, lower CV but higher LB.",
    "1189804": "Huge shake will coming 😑",
    "1191481": "Well maybe the validation data is biased toward the label you mistook, that way the result has improved but without any model improvement ?",
    "1192275": "how submit my notebook?",
    "1197078": "I met the same thing, trained with 512 and submit with 384, the LB score increased from 0.901 to 0.902, but when we use it for ensemble ,the LB score decreased."
  },
  "source": "meta"
}