{
  "id": 220624,
  "title": "#77 LB - some low-tech ideas",
  "url": "/competitions/cassava-leaf-disease-classification/writeups/dave-e-77-lb-some-low-tech-ideas",
  "author_name": "",
  "post_date": "2021-02-20T04:24:51.623Z",
  "votes": 6,
  "comment_count": 4,
  "views": 0,
  "content": "<p><strong>Disclaimer:</strong> I don't have background knowledge in image classification etc. - just learnt what I could from Kaggle, other competition posts &amp; notebooks over last few months. I'm just sharing this as it's possible there are some ideas that others did not try.</p>\n<p>Really enjoyed learning from this comp though by the end of 3 months I think I was happy to end and spend less time tracking results of model runs for a while regardless of where I landed up on LB :-) Thanks to everyone who posted very insightful notebooks and discussion posts.</p>\n<p>Landed up simply following anything which improved both CV and LB, given the problems in correlating the two. In retrospect, should probably have had a little more faith in CV (as many people on forum were saying!).</p>\n<p><strong>Things I did do:</strong></p>\n<p>Dropped some of the likely mislabeled or at least potentially misleading healthy samples (around 150) based on model predictions. Dropping a lot of other samples seemed to make public LB drop and I felt I could not be confident it was helpful.</p>\n<p>Pretrained on 2019 images (used features to create 50 labels based on clusters). This was probably mostly practical as I felt like the model then could be trained faster from that point. I don't think I saw any specific edge from including 2019 data in training directly, but it may have been that my experiments were not sufficiently thorough. It seemed to me that adversarial analysis of 2019 vs 2020 extracted features showed some significant differences in the average image (I didn't go into depth on this). Rightly or wrongly, this also made me wonder whether the image test set in this comp might have some modest differences compared to the train - which made me lean towards things which improved both CV and LB.</p>\n<h2>Individual Classifiers</h2>\n<p>Individual classifiers used in blend had CVs in range of 0.905 (EnetB4, VIT) down to around 0.89. The CV numbers are including some TTA (different crops).</p>\n<p>Trained with some variation in settings:<br>\nPretrain (2019) - 3 epochs with lowish / slightly increasing LR. More than this did not really produce any benefit I could see.</p>\n<p>Training Loss = bitempered ('t1' : 0.6,  't2' : 1.4) or TaylorCrossEntropyLoss</p>\n<p>Augmentations: CutMix p=0.5 (most of the models), CoarseDropout, Cutout (some models), other standard Albumenations (flip/rotate/hue/brightness)</p>\n<p>Learning rate: for main training (after pre training) started around 1e-04 and declined over around 8-15 epochs. Think I found that mostly some final epochs with a low LR could squeeze out a little more validation accuracy.</p>\n<p>Validation accuracy - not sure if this is the best approach, but moved from random crop to centre crop during the comp to have more stable validation trend. Tried to look at validation loss as well as accuracy and to keep training accuracy below validation.</p>\n<h2>What made a difference:</h2>\n<p>From Public 0.91 LB (best efficientnet 5-fold with TTA) to 0.96 was:</p>\n<p><strong>Blending</strong><br>\nAnalysis of image crops - realised from various iterations that just random TTA might not be the best approach.</p>\n<p>Therefore tried weighting of Classifier + Crop with Optuna analysis of CV predictions. This was imperfect because I didn't actually have 5 models of each type, so I did a bit of rounding where the Optuna outputs looked a bit strange, and dropped models/crops with very low weighting to control total run time.</p>\n<p><strong>Classifiers:</strong></p>\n<p>5 x tf_efficientnet_b4_ns, 512*512 image size</p>\n<ul>\n<li>1 x full image weight 0.082631</li>\n<li>1 x centre crop weight 0.102575</li>\n<li>4 x tta random resized crop weight 0.042924</li>\n</ul>\n<p>4 x tf_efficientnet_b3_ns, 384*384 image size</p>\n<ul>\n<li>1 x full image weight 0.028432</li>\n</ul>\n<p>3 x vit_base_patch16_384, 384*384 image size</p>\n<ul>\n<li>1 x full image weight 0.037909</li>\n<li>1 x centre crop (512*512 resized) weight 0.126364</li>\n<li>5 x tta random resized crop weight 0.063182</li>\n</ul>\n<p>2 x resnext50_32x4d, 384*384 image size</p>\n<ul>\n<li>1 x full image weight 0.1</li>\n<li>1 x centre crop (512*512 resized) weight 0.1</li>\n<li>5 x tta random resized crop weight 0.078689</li>\n</ul>\n<p>2 x vit_small_patch16_224, 224*224image size</p>\n<ul>\n<li>1 x full image weight 0.075</li>\n</ul>\n<p>All TTA were limited to scale around 0.37 - 0.85 (in albumenations RandomResizedCrop) based on some limited CV experiments.</p>\n<p>I'm not really sure that I understood all the weights that Optuna allocated, in particular the weighting given to vit_small_patch16_224, but the LB increased roughly in line with what Optuna calculated for CV improvement and end leaderboard change was relatively little as far as I can see compared to the general shakeup. Changing a few things in my final submissions yielded no improvements though I'm sure many could be found with time. </p>\n<p>Am not sharing notebooks as a) they are not particularly organised and b) there is nothing special I think in my training / use of optuna (plenty of other optuna notebooks out there!).</p>\n<p>Anyway, I look forward to learning from the top solutions :-)</p>",
  "messages": [
    {
      "id": "1209671",
      "postDate": "02/19/2021 02:18:36",
      "content": "<p><strong>Disclaimer:</strong> I don't have background knowledge in image classification etc. - just learnt what I could from Kaggle, other competition posts &amp; notebooks over last few months. I'm just sharing this as it's possible there are some ideas that others did not try.</p>\n<p>Really enjoyed learning from this comp though by the end of 3 months I think I was happy to end and spend less time tracking results of model runs for a while regardless of where I landed up on LB :-) Thanks to everyone who posted very insightful notebooks and discussion posts.</p>\n<p>Landed up simply following anything which improved both CV and LB, given the problems in correlating the two. In retrospect, should probably have had a little more faith in CV (as many people on forum were saying!).</p>\n<p><strong>Things I did do:</strong></p>\n<p>Dropped some of the likely mislabeled or at least potentially misleading healthy samples (around 150) based on model predictions. Dropping a lot of other samples seemed to make public LB drop and I felt I could not be confident it was helpful.</p>\n<p>Pretrained on 2019 images (used features to create 50 labels based on clusters). This was probably mostly practical as I felt like the model then could be trained faster from that point. I don't think I saw any specific edge from including 2019 data in training directly, but it may have been that my experiments were not sufficiently thorough. It seemed to me that adversarial analysis of 2019 vs 2020 extracted features showed some significant differences in the average image (I didn't go into depth on this). Rightly or wrongly, this also made me wonder whether the image test set in this comp might have some modest differences compared to the train - which made me lean towards things which improved both CV and LB.</p>\n<h2>Individual Classifiers</h2>\n<p>Individual classifiers used in blend had CVs in range of 0.905 (EnetB4, VIT) down to around 0.89. The CV numbers are including some TTA (different crops).</p>\n<p>Trained with some variation in settings:<br>\nPretrain (2019) - 3 epochs with lowish / slightly increasing LR. More than this did not really produce any benefit I could see.</p>\n<p>Training Loss = bitempered ('t1' : 0.6,  't2' : 1.4) or TaylorCrossEntropyLoss</p>\n<p>Augmentations: CutMix p=0.5 (most of the models), CoarseDropout, Cutout (some models), other standard Albumenations (flip/rotate/hue/brightness)</p>\n<p>Learning rate: for main training (after pre training) started around 1e-04 and declined over around 8-15 epochs. Think I found that mostly some final epochs with a low LR could squeeze out a little more validation accuracy.</p>\n<p>Validation accuracy - not sure if this is the best approach, but moved from random crop to centre crop during the comp to have more stable validation trend. Tried to look at validation loss as well as accuracy and to keep training accuracy below validation.</p>\n<h2>What made a difference:</h2>\n<p>From Public 0.91 LB (best efficientnet 5-fold with TTA) to 0.96 was:</p>\n<p><strong>Blending</strong><br>\nAnalysis of image crops - realised from various iterations that just random TTA might not be the best approach.</p>\n<p>Therefore tried weighting of Classifier + Crop with Optuna analysis of CV predictions. This was imperfect because I didn't actually have 5 models of each type, so I did a bit of rounding where the Optuna outputs looked a bit strange, and dropped models/crops with very low weighting to control total run time.</p>\n<p><strong>Classifiers:</strong></p>\n<p>5 x tf_efficientnet_b4_ns, 512*512 image size</p>\n<ul>\n<li>1 x full image weight 0.082631</li>\n<li>1 x centre crop weight 0.102575</li>\n<li>4 x tta random resized crop weight 0.042924</li>\n</ul>\n<p>4 x tf_efficientnet_b3_ns, 384*384 image size</p>\n<ul>\n<li>1 x full image weight 0.028432</li>\n</ul>\n<p>3 x vit_base_patch16_384, 384*384 image size</p>\n<ul>\n<li>1 x full image weight 0.037909</li>\n<li>1 x centre crop (512*512 resized) weight 0.126364</li>\n<li>5 x tta random resized crop weight 0.063182</li>\n</ul>\n<p>2 x resnext50_32x4d, 384*384 image size</p>\n<ul>\n<li>1 x full image weight 0.1</li>\n<li>1 x centre crop (512*512 resized) weight 0.1</li>\n<li>5 x tta random resized crop weight 0.078689</li>\n</ul>\n<p>2 x vit_small_patch16_224, 224*224image size</p>\n<ul>\n<li>1 x full image weight 0.075</li>\n</ul>\n<p>All TTA were limited to scale around 0.37 - 0.85 (in albumenations RandomResizedCrop) based on some limited CV experiments.</p>\n<p>I'm not really sure that I understood all the weights that Optuna allocated, in particular the weighting given to vit_small_patch16_224, but the LB increased roughly in line with what Optuna calculated for CV improvement and end leaderboard change was relatively little as far as I can see compared to the general shakeup. Changing a few things in my final submissions yielded no improvements though I'm sure many could be found with time. </p>\n<p>Am not sharing notebooks as a) they are not particularly organised and b) there is nothing special I think in my training / use of optuna (plenty of other optuna notebooks out there!).</p>\n<p>Anyway, I look forward to learning from the top solutions :-)</p>",
      "rawMarkdown": "**Disclaimer:** I don't have background knowledge in image classification etc. - just learnt what I could from Kaggle, other competition posts & notebooks over last few months. I'm just sharing this as it's possible there are some ideas that others did not try.\n\nReally enjoyed learning from this comp though by the end of 3 months I think I was happy to end and spend less time tracking results of model runs for a while regardless of where I landed up on LB :-) Thanks to everyone who posted very insightful notebooks and discussion posts.\n\nLanded up simply following anything which improved both CV and LB, given the problems in correlating the two. In retrospect, should probably have had a little more faith in CV (as many people on forum were saying!).\n\n**Things I did do:**\n\nDropped some of the likely mislabeled or at least potentially misleading healthy samples (around 150) based on model predictions. Dropping a lot of other samples seemed to make public LB drop and I felt I could not be confident it was helpful.\n\nPretrained on 2019 images (used features to create 50 labels based on clusters). This was probably mostly practical as I felt like the model then could be trained faster from that point. I don't think I saw any specific edge from including 2019 data in training directly, but it may have been that my experiments were not sufficiently thorough. It seemed to me that adversarial analysis of 2019 vs 2020 extracted features showed some significant differences in the average image (I didn't go into depth on this). Rightly or wrongly, this also made me wonder whether the image test set in this comp might have some modest differences compared to the train - which made me lean towards things which improved both CV and LB.\n\n## Individual Classifiers\n\nIndividual classifiers used in blend had CVs in range of 0.905 (EnetB4, VIT) down to around 0.89. The CV numbers are including some TTA (different crops).\n\nTrained with some variation in settings:\nPretrain (2019) - 3 epochs with lowish / slightly increasing LR. More than this did not really produce any benefit I could see.\n\nTraining Loss = bitempered ('t1' : 0.6,  't2' : 1.4) or TaylorCrossEntropyLoss\n\nAugmentations: CutMix p=0.5 (most of the models), CoarseDropout, Cutout (some models), other standard Albumenations (flip/rotate/hue/brightness)\n\nLearning rate: for main training (after pre training) started around 1e-04 and declined over around 8-15 epochs. Think I found that mostly some final epochs with a low LR could squeeze out a little more validation accuracy.\n\nValidation accuracy - not sure if this is the best approach, but moved from random crop to centre crop during the comp to have more stable validation trend. Tried to look at validation loss as well as accuracy and to keep training accuracy below validation.\n\n## What made a difference:\n\nFrom Public 0.91 LB (best efficientnet 5-fold with TTA) to 0.96 was:\n\n**Blending**\nAnalysis of image crops - realised from various iterations that just random TTA might not be the best approach.\n\nTherefore tried weighting of Classifier + Crop with Optuna analysis of CV predictions. This was imperfect because I didn't actually have 5 models of each type, so I did a bit of rounding where the Optuna outputs looked a bit strange, and dropped models/crops with very low weighting to control total run time.\n\n**Classifiers:**\n\n5 x tf_efficientnet_b4_ns, 512*512 image size\n- 1 x full image weight 0.082631\n- 1 x centre crop weight 0.102575\n- 4 x tta random resized crop weight 0.042924\n\n4 x tf_efficientnet_b3_ns, 384*384 image size\n- 1 x full image weight 0.028432\n\n3 x vit_base_patch16_384, 384*384 image size\n- 1 x full image weight 0.037909\n- 1 x centre crop (512*512 resized) weight 0.126364\n- 5 x tta random resized crop weight 0.063182\n\n2 x resnext50_32x4d, 384*384 image size\n- 1 x full image weight 0.1\n- 1 x centre crop (512*512 resized) weight 0.1\n- 5 x tta random resized crop weight 0.078689\n\n2 x vit_small_patch16_224, 224*224image size\n- 1 x full image weight 0.075\n\nAll TTA were limited to scale around 0.37 - 0.85 (in albumenations RandomResizedCrop) based on some limited CV experiments.\n\nI'm not really sure that I understood all the weights that Optuna allocated, in particular the weighting given to vit_small_patch16_224, but the LB increased roughly in line with what Optuna calculated for CV improvement and end leaderboard change was relatively little as far as I can see compared to the general shakeup. Changing a few things in my final submissions yielded no improvements though I'm sure many could be found with time. \n\nAm not sharing notebooks as a) they are not particularly organised and b) there is nothing special I think in my training / use of optuna (plenty of other optuna notebooks out there!).\n\nAnyway, I look forward to learning from the top solutions :-)",
      "votes": null
    },
    {
      "id": "1209778",
      "postDate": "02/19/2021 03:28:58",
      "content": "<p>Thanks for sharing your solution! :) <a href=\"https://www.kaggle.com/davidedwards1\" target=\"_blank\">@davidedwards1</a> </p>",
      "rawMarkdown": "Thanks for sharing your solution! :) @davidedwards1",
      "votes": null
    },
    {
      "id": "1209805",
      "postDate": "02/19/2021 03:47:36",
      "content": "<p>Thanks! I read and upvoted your notebook during the competition :-)</p>",
      "rawMarkdown": "Thanks! I read and upvoted your notebook during the competition :-)",
      "votes": null
    },
    {
      "id": "1209851",
      "postDate": "02/19/2021 04:28:53",
      "content": "<p>Thanks for sharing. So optuna seems to have really helped you. </p>",
      "rawMarkdown": "Thanks for sharing. So optuna seems to have really helped you.",
      "votes": null
    },
    {
      "id": "1209963",
      "postDate": "02/19/2021 06:07:21",
      "content": "<p>It does seem like it.</p>\n<p>Just the <strong>vit_base_patch16_384 / EnetB4 / Resnext blend</strong> (no weighting, just unweighted full image+center crop+tta) scored:<br>\nPriv/Public 0.898 0.905</p>\n<p><strong>Adding crop/tta weightings</strong> to the above 3:<br>\nPriv/Public 0.899 0.903</p>\n<p>Adding in the <strong>EnetB3 and vit_small_patch16_224</strong> (even though they had poor accuracy by themselves) and their weightings gave<br>\nPriv/Public 0.900 0.906</p>\n<p>So the last step is probably borderline (made more of a difference on public LB, did a little better overall but hard to see exactly with 3 decimal places!) but as far as I can see, optuna came up with some decent weightings, unless it was just good luck.</p>\n<p>I am actually fortunate that I ran the 3rd of the above submissions, because otherwise I would possibly have gone with the higher Public LB score / lower private as at least one of the 2 selected subs, even though Optuna said CV would be higher with weightings.</p>\n<p>Edit: should also add, Optuna analysis provided this improvement (assuming it's not just good luck) with probably non-ideal data, as I only had the full 5-fold ensemble for efficientnetb4. Some images only had OOF predictions from 2-3 models.</p>",
      "rawMarkdown": "It does seem like it.\n\nJust the **vit_base_patch16_384 / EnetB4 / Resnext blend** (no weighting, just unweighted full image+center crop+tta) scored:\nPriv/Public 0.898 0.905\n\n**Adding crop/tta weightings** to the above 3:\nPriv/Public 0.899 0.903\n\nAdding in the **EnetB3 and vit_small_patch16_224** (even though they had poor accuracy by themselves) and their weightings gave\nPriv/Public 0.900 0.906\n\nSo the last step is probably borderline (made more of a difference on public LB, did a little better overall but hard to see exactly with 3 decimal places!) but as far as I can see, optuna came up with some decent weightings, unless it was just good luck.\n\nI am actually fortunate that I ran the 3rd of the above submissions, because otherwise I would possibly have gone with the higher Public LB score / lower private as at least one of the 2 selected subs, even though Optuna said CV would be higher with weightings.\n\nEdit: should also add, Optuna analysis provided this improvement (assuming it's not just good luck) with probably non-ideal data, as I only had the full 5-fold ensemble for efficientnetb4. Some images only had OOF predictions from 2-3 models.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1209778,
      "author_name": "piantic",
      "author_url": "",
      "post_date": "02/19/2021 03:28:58",
      "content": "<p>Thanks for sharing your solution! :) <a href=\"https://www.kaggle.com/davidedwards1\" target=\"_blank\">@davidedwards1</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 1209805,
          "author_name": "davidedwards1",
          "author_url": "",
          "post_date": "02/19/2021 03:47:36",
          "content": "<p>Thanks! I read and upvoted your notebook during the competition :-)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1209851,
      "author_name": "krisho007",
      "author_url": "",
      "post_date": "02/19/2021 04:28:53",
      "content": "<p>Thanks for sharing. So optuna seems to have really helped you. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1209963,
          "author_name": "davidedwards1",
          "author_url": "",
          "post_date": "02/19/2021 06:07:21",
          "content": "<p>It does seem like it.</p>\n<p>Just the <strong>vit_base_patch16_384 / EnetB4 / Resnext blend</strong> (no weighting, just unweighted full image+center crop+tta) scored:<br>\nPriv/Public 0.898 0.905</p>\n<p><strong>Adding crop/tta weightings</strong> to the above 3:<br>\nPriv/Public 0.899 0.903</p>\n<p>Adding in the <strong>EnetB3 and vit_small_patch16_224</strong> (even though they had poor accuracy by themselves) and their weightings gave<br>\nPriv/Public 0.900 0.906</p>\n<p>So the last step is probably borderline (made more of a difference on public LB, did a little better overall but hard to see exactly with 3 decimal places!) but as far as I can see, optuna came up with some decent weightings, unless it was just good luck.</p>\n<p>I am actually fortunate that I ran the 3rd of the above submissions, because otherwise I would possibly have gone with the higher Public LB score / lower private as at least one of the 2 selected subs, even though Optuna said CV would be higher with weightings.</p>\n<p>Edit: should also add, Optuna analysis provided this improvement (assuming it's not just good luck) with probably non-ideal data, as I only had the full 5-fold ensemble for efficientnetb4. Some images only had OOF predictions from 2-3 models.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1209671": "**Disclaimer:** I don't have background knowledge in image classification etc. - just learnt what I could from Kaggle, other competition posts & notebooks over last few months. I'm just sharing this as it's possible there are some ideas that others did not try.\n\nReally enjoyed learning from this comp though by the end of 3 months I think I was happy to end and spend less time tracking results of model runs for a while regardless of where I landed up on LB :-) Thanks to everyone who posted very insightful notebooks and discussion posts.\n\nLanded up simply following anything which improved both CV and LB, given the problems in correlating the two. In retrospect, should probably have had a little more faith in CV (as many people on forum were saying!).\n\n**Things I did do:**\n\nDropped some of the likely mislabeled or at least potentially misleading healthy samples (around 150) based on model predictions. Dropping a lot of other samples seemed to make public LB drop and I felt I could not be confident it was helpful.\n\nPretrained on 2019 images (used features to create 50 labels based on clusters). This was probably mostly practical as I felt like the model then could be trained faster from that point. I don't think I saw any specific edge from including 2019 data in training directly, but it may have been that my experiments were not sufficiently thorough. It seemed to me that adversarial analysis of 2019 vs 2020 extracted features showed some significant differences in the average image (I didn't go into depth on this). Rightly or wrongly, this also made me wonder whether the image test set in this comp might have some modest differences compared to the train - which made me lean towards things which improved both CV and LB.\n\n## Individual Classifiers\n\nIndividual classifiers used in blend had CVs in range of 0.905 (EnetB4, VIT) down to around 0.89. The CV numbers are including some TTA (different crops).\n\nTrained with some variation in settings:\nPretrain (2019) - 3 epochs with lowish / slightly increasing LR. More than this did not really produce any benefit I could see.\n\nTraining Loss = bitempered ('t1' : 0.6,  't2' : 1.4) or TaylorCrossEntropyLoss\n\nAugmentations: CutMix p=0.5 (most of the models), CoarseDropout, Cutout (some models), other standard Albumenations (flip/rotate/hue/brightness)\n\nLearning rate: for main training (after pre training) started around 1e-04 and declined over around 8-15 epochs. Think I found that mostly some final epochs with a low LR could squeeze out a little more validation accuracy.\n\nValidation accuracy - not sure if this is the best approach, but moved from random crop to centre crop during the comp to have more stable validation trend. Tried to look at validation loss as well as accuracy and to keep training accuracy below validation.\n\n## What made a difference:\n\nFrom Public 0.91 LB (best efficientnet 5-fold with TTA) to 0.96 was:\n\n**Blending**\nAnalysis of image crops - realised from various iterations that just random TTA might not be the best approach.\n\nTherefore tried weighting of Classifier + Crop with Optuna analysis of CV predictions. This was imperfect because I didn't actually have 5 models of each type, so I did a bit of rounding where the Optuna outputs looked a bit strange, and dropped models/crops with very low weighting to control total run time.\n\n**Classifiers:**\n\n5 x tf_efficientnet_b4_ns, 512*512 image size\n- 1 x full image weight 0.082631\n- 1 x centre crop weight 0.102575\n- 4 x tta random resized crop weight 0.042924\n\n4 x tf_efficientnet_b3_ns, 384*384 image size\n- 1 x full image weight 0.028432\n\n3 x vit_base_patch16_384, 384*384 image size\n- 1 x full image weight 0.037909\n- 1 x centre crop (512*512 resized) weight 0.126364\n- 5 x tta random resized crop weight 0.063182\n\n2 x resnext50_32x4d, 384*384 image size\n- 1 x full image weight 0.1\n- 1 x centre crop (512*512 resized) weight 0.1\n- 5 x tta random resized crop weight 0.078689\n\n2 x vit_small_patch16_224, 224*224image size\n- 1 x full image weight 0.075\n\nAll TTA were limited to scale around 0.37 - 0.85 (in albumenations RandomResizedCrop) based on some limited CV experiments.\n\nI'm not really sure that I understood all the weights that Optuna allocated, in particular the weighting given to vit_small_patch16_224, but the LB increased roughly in line with what Optuna calculated for CV improvement and end leaderboard change was relatively little as far as I can see compared to the general shakeup. Changing a few things in my final submissions yielded no improvements though I'm sure many could be found with time. \n\nAm not sharing notebooks as a) they are not particularly organised and b) there is nothing special I think in my training / use of optuna (plenty of other optuna notebooks out there!).\n\nAnyway, I look forward to learning from the top solutions :-)",
    "1209778": "Thanks for sharing your solution! :) @davidedwards1",
    "1209805": "Thanks! I read and upvoted your notebook during the competition :-)",
    "1209851": "Thanks for sharing. So optuna seems to have really helped you.",
    "1209963": "It does seem like it.\n\nJust the **vit_base_patch16_384 / EnetB4 / Resnext blend** (no weighting, just unweighted full image+center crop+tta) scored:\nPriv/Public 0.898 0.905\n\n**Adding crop/tta weightings** to the above 3:\nPriv/Public 0.899 0.903\n\nAdding in the **EnetB3 and vit_small_patch16_224** (even though they had poor accuracy by themselves) and their weightings gave\nPriv/Public 0.900 0.906\n\nSo the last step is probably borderline (made more of a difference on public LB, did a little better overall but hard to see exactly with 3 decimal places!) but as far as I can see, optuna came up with some decent weightings, unless it was just good luck.\n\nI am actually fortunate that I ran the 3rd of the above submissions, because otherwise I would possibly have gone with the higher Public LB score / lower private as at least one of the 2 selected subs, even though Optuna said CV would be higher with weightings.\n\nEdit: should also add, Optuna analysis provided this improvement (assuming it's not just good luck) with probably non-ideal data, as I only had the full 5-fold ensemble for efficientnetb4. Some images only had OOF predictions from 2-3 models."
  },
  "source": "meta"
}