{
  "id": 175394,
  "title": "CV is all you need!!! 1044 Public to 75 Private (0.953 CV)",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/175394",
  "author_name": "",
  "post_date": "2020-08-18T04:42:06.422320600Z",
  "votes": 17,
  "comment_count": 5,
  "views": 0,
  "content": "<p>It was really scary till end being at 1044 in public lb, but, we decided to stick to optimising CV. Glad things worked out in the end!! I see a perfect correlation for my CV to private lb, thanks, to triple stratified tf-records made by Chris!! Also thanks to all my teammate to pull it off. Our 0.953 CV submission scored only 0.9499 on public but did wonders on private!!<br>\nWill post solution soon.</p>",
  "messages": [
    {
      "id": "974889",
      "postDate": "08/18/2020 04:42:06",
      "content": "<p>It was really scary till end being at 1044 in public lb, but, we decided to stick to optimising CV. Glad things worked out in the end!! I see a perfect correlation for my CV to private lb, thanks, to triple stratified tf-records made by Chris!! Also thanks to all my teammate to pull it off. Our 0.953 CV submission scored only 0.9499 on public but did wonders on private!!<br>\nWill post solution soon.</p>",
      "rawMarkdown": "It was really scary till end being at 1044 in public lb, but, we decided to stick to optimising CV. Glad things worked out in the end!! I see a perfect correlation for my CV to private lb, thanks, to triple stratified tf-records made by Chris!! Also thanks to all my teammate to pull it off. Our 0.953 CV submission scored only 0.9499 on public but did wonders on private!!\nWill post solution soon.",
      "votes": null
    },
    {
      "id": "974936",
      "postDate": "08/18/2020 05:03:42",
      "content": "<p>Super! Many congratulations! Looking forward to your post detailing your solution. The fact that you were able to make the right selection shows your are a true winner!</p>",
      "rawMarkdown": "Super! Many congratulations! Looking forward to your post detailing your solution. The fact that you were able to make the right selection shows your are a true winner!",
      "votes": null
    },
    {
      "id": "974938",
      "postDate": "08/18/2020 05:05:24",
      "content": "<p>Awesome. Congratulations KS and team. I too was scared before private LB was revealed but trusting CV really works!</p>",
      "rawMarkdown": "Awesome. Congratulations KS and team. I too was scared before private LB was revealed but trusting CV really works!",
      "votes": null
    },
    {
      "id": "975166",
      "postDate": "08/18/2020 07:21:01",
      "content": "<p>I completely agree. Despite the fact that I fell from the top 7% to the top 12% and lost my medal, my best score was shown by a sub selected via CV. At the same time public blending submissions which gave me 0.9660+ score have shown bad results at private LB, usually approximately 0.92. I know that a lot of people got a good score with public ensembles. However, CV is above all. Of course, I'm disappointed about losing my medal but I'm not disappointed about choosing the wrong submission. And many people regret it.</p>",
      "rawMarkdown": "I completely agree. Despite the fact that I fell from the top 7% to the top 12% and lost my medal, my best score was shown by a sub selected via CV. At the same time public blending submissions which gave me 0.9660+ score have shown bad results at private LB, usually approximately 0.92. I know that a lot of people got a good score with public ensembles. However, CV is above all. Of course, I'm disappointed about losing my medal but I'm not disappointed about choosing the wrong submission. And many people regret it.",
      "votes": null
    },
    {
      "id": "975509",
      "postDate": "08/18/2020 10:25:00",
      "content": "<p>That's the correct attitude. If we keep doing correct things, things will fall in place. I suffered a huge shakeup on DS bowl trusting public lb, after that, I always try to find correct CV technique and rely on my CV score!! Here, thanks to chris, 1st part was already done!!</p>",
      "rawMarkdown": "That's the correct attitude. If we keep doing correct things, things will fall in place. I suffered a huge shakeup on DS bowl trusting public lb, after that, I always try to find correct CV technique and rely on my CV score!! Here, thanks to chris, 1st part was already done!!",
      "votes": null
    },
    {
      "id": "979701",
      "postDate": "08/21/2020 04:21:48",
      "content": "<p>Briefly discussing Approach</p>\n<ol>\n<li>Multi-input Model (Image + Tabular Data) with Snapshot Ensemble and  Psuedolabelling</li>\n<li>Trained models for both focal and cross entropy loss for different configurations (Image Sizes, Efficient Net, Dropout)</li>\n<li>Dropout was not used during inference</li>\n<li>Upsampling didn't work for us</li>\n<li>Weighted blending Ensemble of top 40 models- Weights were optimised using bayesian optimisation to maximise auc on OOF predictions</li>\n<li>Most important, we were team of 5. Most of us were working on our own codes, optimising things differently. So, when we ensembled our models, std between predictions were high and we got significant boost on OOF auc. </li>\n</ol>",
      "rawMarkdown": "Briefly discussing Approach\n1. Multi-input Model (Image + Tabular Data) with Snapshot Ensemble and  Psuedolabelling\n2. Trained models for both focal and cross entropy loss for different configurations (Image Sizes, Efficient Net, Dropout)\n3. Dropout was not used during inference\n4. Upsampling didn't work for us\n5. Weighted blending Ensemble of top 40 models- Weights were optimised using bayesian optimisation to maximise auc on OOF predictions\n6. Most important, we were team of 5. Most of us were working on our own codes, optimising things differently. So, when we ensembled our models, std between predictions were high and we got significant boost on OOF auc.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 974936,
      "author_name": "shikha130vv",
      "author_url": "",
      "post_date": "08/18/2020 05:03:42",
      "content": "<p>Super! Many congratulations! Looking forward to your post detailing your solution. The fact that you were able to make the right selection shows your are a true winner!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 974938,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "08/18/2020 05:05:24",
      "content": "<p>Awesome. Congratulations KS and team. I too was scared before private LB was revealed but trusting CV really works!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 975166,
      "author_name": "vadimtimakin",
      "author_url": "",
      "post_date": "08/18/2020 07:21:01",
      "content": "<p>I completely agree. Despite the fact that I fell from the top 7% to the top 12% and lost my medal, my best score was shown by a sub selected via CV. At the same time public blending submissions which gave me 0.9660+ score have shown bad results at private LB, usually approximately 0.92. I know that a lot of people got a good score with public ensembles. However, CV is above all. Of course, I'm disappointed about losing my medal but I'm not disappointed about choosing the wrong submission. And many people regret it.</p>",
      "votes": null,
      "replies": [
        {
          "id": 975509,
          "author_name": "ks2019",
          "author_url": "",
          "post_date": "08/18/2020 10:25:00",
          "content": "<p>That's the correct attitude. If we keep doing correct things, things will fall in place. I suffered a huge shakeup on DS bowl trusting public lb, after that, I always try to find correct CV technique and rely on my CV score!! Here, thanks to chris, 1st part was already done!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 979701,
      "author_name": "ks2019",
      "author_url": "",
      "post_date": "08/21/2020 04:21:48",
      "content": "<p>Briefly discussing Approach</p>\n<ol>\n<li>Multi-input Model (Image + Tabular Data) with Snapshot Ensemble and  Psuedolabelling</li>\n<li>Trained models for both focal and cross entropy loss for different configurations (Image Sizes, Efficient Net, Dropout)</li>\n<li>Dropout was not used during inference</li>\n<li>Upsampling didn't work for us</li>\n<li>Weighted blending Ensemble of top 40 models- Weights were optimised using bayesian optimisation to maximise auc on OOF predictions</li>\n<li>Most important, we were team of 5. Most of us were working on our own codes, optimising things differently. So, when we ensembled our models, std between predictions were high and we got significant boost on OOF auc. </li>\n</ol>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "974889": "It was really scary till end being at 1044 in public lb, but, we decided to stick to optimising CV. Glad things worked out in the end!! I see a perfect correlation for my CV to private lb, thanks, to triple stratified tf-records made by Chris!! Also thanks to all my teammate to pull it off. Our 0.953 CV submission scored only 0.9499 on public but did wonders on private!!\nWill post solution soon.",
    "974936": "Super! Many congratulations! Looking forward to your post detailing your solution. The fact that you were able to make the right selection shows your are a true winner!",
    "974938": "Awesome. Congratulations KS and team. I too was scared before private LB was revealed but trusting CV really works!",
    "975166": "I completely agree. Despite the fact that I fell from the top 7% to the top 12% and lost my medal, my best score was shown by a sub selected via CV. At the same time public blending submissions which gave me 0.9660+ score have shown bad results at private LB, usually approximately 0.92. I know that a lot of people got a good score with public ensembles. However, CV is above all. Of course, I'm disappointed about losing my medal but I'm not disappointed about choosing the wrong submission. And many people regret it.",
    "975509": "That's the correct attitude. If we keep doing correct things, things will fall in place. I suffered a huge shakeup on DS bowl trusting public lb, after that, I always try to find correct CV technique and rely on my CV score!! Here, thanks to chris, 1st part was already done!!",
    "979701": "Briefly discussing Approach\n1. Multi-input Model (Image + Tabular Data) with Snapshot Ensemble and  Psuedolabelling\n2. Trained models for both focal and cross entropy loss for different configurations (Image Sizes, Efficient Net, Dropout)\n3. Dropout was not used during inference\n4. Upsampling didn't work for us\n5. Weighted blending Ensemble of top 40 models- Weights were optimised using bayesian optimisation to maximise auc on OOF predictions\n6. Most important, we were team of 5. Most of us were working on our own codes, optimising things differently. So, when we ensembled our models, std between predictions were high and we got significant boost on OOF auc."
  },
  "source": "meta"
}