{
  "id": 220714,
  "title": "public 87th -> private 668th very bad solution for remembering sad memory...",
  "url": "/competitions/cassava-leaf-disease-classification/writeups/dress-rosa-public-87th-private-668th-very-bad-solu",
  "author_name": "",
  "post_date": "2021-02-19T09:44:44.443Z",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Firstly, congratulation to all competitors!</p>\n<p>This is my regret for large shake down.</p>\n<p>I used 3 type CNN.</p>\n<ol>\n<li>SEResNext50 32x_4d   public : 0.898 -&gt; private : 0.895</li>\n<li>EfficientNetB4              public : 0.899 -&gt; private : 0.890 </li>\n<li>ViT_base_384               public : 0.899 -&gt; private : 0.890</li>\n</ol>\n<p>In final, I weights each model with 1 : 1.2 : 1.2 for ensemble.</p>\n<ul>\n<li>Public LB : 0.905</li>\n<li>Private LB : 0.896</li>\n</ul>\n<p>I regret to overfit to public LB…</p>\n<p>I thought this competition is my first silver, but result is reflecting my current ability.</p>\n<p>And, I think the cause of weak model is below</p>\n<ul>\n<li>using cutmix</li>\n<li>don't use tempered loss</li>\n<li>fixed random seed for cv through all model</li>\n</ul>\n<p>I doubt seed ensemble can help me a little, but the bird has flown :(</p>\n<p>The ensemble SEResNeXt50_32x4d + SEResNet152d_320</p>\n<ul>\n<li>Public LB : 0.900</li>\n<li>Pirvate LB : 0.899</li>\n</ul>\n<p>The ensemble 1SEResNext50 32x_4d + EfficientNetB4  + ViT_base_384  (public:0.895, private:0.891) </p>\n<ul>\n<li>Public LB : 0.902</li>\n<li>Pirvate LB : 0.899</li>\n</ul>\n<p>These model can get medal zone private score, but I don't know why bad score ViT is essential for good score. This competition definiely needs for early stopping for not perfect fitting training data?</p>\n<p>Please tell me how do you think cutmix , tempered loss, seed and so on and your preventing shake method.</p>",
  "messages": [
    {
      "id": "1210209",
      "postDate": "02/19/2021 09:07:36",
      "content": "<p>Firstly, congratulation to all competitors!</p>\n<p>This is my regret for large shake down.</p>\n<p>I used 3 type CNN.</p>\n<ol>\n<li>SEResNext50 32x_4d   public : 0.898 -&gt; private : 0.895</li>\n<li>EfficientNetB4              public : 0.899 -&gt; private : 0.890 </li>\n<li>ViT_base_384               public : 0.899 -&gt; private : 0.890</li>\n</ol>\n<p>In final, I weights each model with 1 : 1.2 : 1.2 for ensemble.</p>\n<ul>\n<li>Public LB : 0.905</li>\n<li>Private LB : 0.896</li>\n</ul>\n<p>I regret to overfit to public LB…</p>\n<p>I thought this competition is my first silver, but result is reflecting my current ability.</p>\n<p>And, I think the cause of weak model is below</p>\n<ul>\n<li>using cutmix</li>\n<li>don't use tempered loss</li>\n<li>fixed random seed for cv through all model</li>\n</ul>\n<p>I doubt seed ensemble can help me a little, but the bird has flown :(</p>\n<p>The ensemble SEResNeXt50_32x4d + SEResNet152d_320</p>\n<ul>\n<li>Public LB : 0.900</li>\n<li>Pirvate LB : 0.899</li>\n</ul>\n<p>The ensemble 1SEResNext50 32x_4d + EfficientNetB4  + ViT_base_384  (public:0.895, private:0.891) </p>\n<ul>\n<li>Public LB : 0.902</li>\n<li>Pirvate LB : 0.899</li>\n</ul>\n<p>These model can get medal zone private score, but I don't know why bad score ViT is essential for good score. This competition definiely needs for early stopping for not perfect fitting training data?</p>\n<p>Please tell me how do you think cutmix , tempered loss, seed and so on and your preventing shake method.</p>",
      "rawMarkdown": "Firstly, congratulation to all competitors!\n\nThis is my regret for large shake down.\n\nI used 3 type CNN.\n\n1. SEResNext50 32x_4d   public : 0.898 -> private : 0.895\n2. EfficientNetB4              public : 0.899 -> private : 0.890 \n3. ViT_base_384               public : 0.899 -> private : 0.890\n\nIn final, I weights each model with 1 : 1.2 : 1.2 for ensemble.\n- Public LB : 0.905\n- Private LB : 0.896\n\nI regret to overfit to public LB...\n\nI thought this competition is my first silver, but result is reflecting my current ability.\n\nAnd, I think the cause of weak model is below\n- using cutmix\n- don't use tempered loss\n- fixed random seed for cv through all model\n\nI doubt seed ensemble can help me a little, but the bird has flown :(\n\n\nThe ensemble SEResNeXt50_32x4d + SEResNet152d_320\n- Public LB : 0.900\n- Pirvate LB : 0.899\n\nThe ensemble 1SEResNext50 32x_4d + EfficientNetB4  + ViT_base_384  (public:0.895, private:0.891) \n- Public LB : 0.902\n- Pirvate LB : 0.899\n\nThese model can get medal zone private score, but I don't know why bad score ViT is essential for good score. This competition definiely needs for early stopping for not perfect fitting training data?\n\nPlease tell me how do you think cutmix , tempered loss, seed and so on and your preventing shake method.",
      "votes": null
    },
    {
      "id": "1210246",
      "postDate": "02/19/2021 09:39:31",
      "content": "<p>I doubt that \"fixed random seed for cv through all model\" is a problem. You want the same seed so that you can determine your blending weights properly via CV (rather than based on the LB). And, yes, predicting with a model per CV fold can be rather helpful (it's essentially a form of bagging). For ensembling, variety in models is important (in fact checking the ensemble weights via the CV should normally show that rather clearly). Differences in model performance of 0.89x to 0.90x on the public LB are rather unreliable (it's a small number of images with a very low information metric - i.e. accuracy, the signal from the public LB would be much stronger, if one had e.g. log-loss), so I would not even say that ViT was all that much worse.</p>",
      "rawMarkdown": "I doubt that \"fixed random seed for cv through all model\" is a problem. You want the same seed so that you can determine your blending weights properly via CV (rather than based on the LB). And, yes, predicting with a model per CV fold can be rather helpful (it's essentially a form of bagging). For ensembling, variety in models is important (in fact checking the ensemble weights via the CV should normally show that rather clearly). Differences in model performance of 0.89x to 0.90x on the public LB are rather unreliable (it's a small number of images with a very low information metric - i.e. accuracy, the signal from the public LB would be much stronger, if one had e.g. log-loss), so I would not even say that ViT was all that much worse.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1210246,
      "author_name": "bjoernholzhauer",
      "author_url": "",
      "post_date": "02/19/2021 09:39:31",
      "content": "<p>I doubt that \"fixed random seed for cv through all model\" is a problem. You want the same seed so that you can determine your blending weights properly via CV (rather than based on the LB). And, yes, predicting with a model per CV fold can be rather helpful (it's essentially a form of bagging). For ensembling, variety in models is important (in fact checking the ensemble weights via the CV should normally show that rather clearly). Differences in model performance of 0.89x to 0.90x on the public LB are rather unreliable (it's a small number of images with a very low information metric - i.e. accuracy, the signal from the public LB would be much stronger, if one had e.g. log-loss), so I would not even say that ViT was all that much worse.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1210209": "Firstly, congratulation to all competitors!\n\nThis is my regret for large shake down.\n\nI used 3 type CNN.\n\n1. SEResNext50 32x_4d   public : 0.898 -> private : 0.895\n2. EfficientNetB4              public : 0.899 -> private : 0.890 \n3. ViT_base_384               public : 0.899 -> private : 0.890\n\nIn final, I weights each model with 1 : 1.2 : 1.2 for ensemble.\n- Public LB : 0.905\n- Private LB : 0.896\n\nI regret to overfit to public LB...\n\nI thought this competition is my first silver, but result is reflecting my current ability.\n\nAnd, I think the cause of weak model is below\n- using cutmix\n- don't use tempered loss\n- fixed random seed for cv through all model\n\nI doubt seed ensemble can help me a little, but the bird has flown :(\n\n\nThe ensemble SEResNeXt50_32x4d + SEResNet152d_320\n- Public LB : 0.900\n- Pirvate LB : 0.899\n\nThe ensemble 1SEResNext50 32x_4d + EfficientNetB4  + ViT_base_384  (public:0.895, private:0.891) \n- Public LB : 0.902\n- Pirvate LB : 0.899\n\nThese model can get medal zone private score, but I don't know why bad score ViT is essential for good score. This competition definiely needs for early stopping for not perfect fitting training data?\n\nPlease tell me how do you think cutmix , tempered loss, seed and so on and your preventing shake method.",
    "1210246": "I doubt that \"fixed random seed for cv through all model\" is a problem. You want the same seed so that you can determine your blending weights properly via CV (rather than based on the LB). And, yes, predicting with a model per CV fold can be rather helpful (it's essentially a form of bagging). For ensembling, variety in models is important (in fact checking the ensemble weights via the CV should normally show that rather clearly). Differences in model performance of 0.89x to 0.90x on the public LB are rather unreliable (it's a small number of images with a very low information metric - i.e. accuracy, the signal from the public LB would be much stronger, if one had e.g. log-loss), so I would not even say that ViT was all that much worse."
  },
  "source": "meta"
}