{
  "id": 210918,
  "title": "Which loss function is better?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/210918",
  "author_name": "",
  "post_date": "2021-01-13T01:18:06.587075100Z",
  "votes": 7,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I tried some loss functions which contain discussed one.</p>\n<ul>\n<li>Softmax Cross Entropy</li>\n<li>Smoothing Label</li>\n<li>Focal Cosine Loss</li>\n<li>Tempered Loss</li>\n</ul>\n<p>but public LB is not improved.</p>\n<p>Please let me your loss strategy, if you can…</p>\n<p>I noticed below from my experiments.</p>\n<ul>\n<li>Focal Cosine Loss : local CV score(train and val) is improved, but LB is not.</li>\n<li>Tempered Loss : local CV accuracy is not improved, but LB is not.</li>\n<li>Smoothing Label : local CV accuracy is not improved, but LB is not.  However, local CV and public is nearest.</li>\n</ul>\n<p>And, I noticed Vision Transformer may be good at public LB, because of its local CV is worse than other model toward public LB score.</p>\n<p>ViT predict to only 1 image we can access for test is<br>\narray([[0.01223303, 0.01697356, 0.65529186, 0.04579847, 0.26970305]])<br>\n( this may be worse predict).<br>\nI think this ViT is not trained well, but public LB is not worse.</p>\n<p>For this reason, I think public LB data contains some noise, so confused random prediction accidently contribute to score.</p>",
  "messages": [
    {
      "id": "1150930",
      "postDate": "01/13/2021 01:18:06",
      "content": "<p>I tried some loss functions which contain discussed one.</p>\n<ul>\n<li>Softmax Cross Entropy</li>\n<li>Smoothing Label</li>\n<li>Focal Cosine Loss</li>\n<li>Tempered Loss</li>\n</ul>\n<p>but public LB is not improved.</p>\n<p>Please let me your loss strategy, if you can…</p>\n<p>I noticed below from my experiments.</p>\n<ul>\n<li>Focal Cosine Loss : local CV score(train and val) is improved, but LB is not.</li>\n<li>Tempered Loss : local CV accuracy is not improved, but LB is not.</li>\n<li>Smoothing Label : local CV accuracy is not improved, but LB is not.  However, local CV and public is nearest.</li>\n</ul>\n<p>And, I noticed Vision Transformer may be good at public LB, because of its local CV is worse than other model toward public LB score.</p>\n<p>ViT predict to only 1 image we can access for test is<br>\narray([[0.01223303, 0.01697356, 0.65529186, 0.04579847, 0.26970305]])<br>\n( this may be worse predict).<br>\nI think this ViT is not trained well, but public LB is not worse.</p>\n<p>For this reason, I think public LB data contains some noise, so confused random prediction accidently contribute to score.</p>",
      "rawMarkdown": "I tried some loss functions which contain discussed one.\n\n- Softmax Cross Entropy\n- Smoothing Label\n- Focal Cosine Loss\n- Tempered Loss\n\nbut public LB is not improved.\n\nPlease let me your loss strategy, if you can...\n\nI noticed below from my experiments.\n\n- Focal Cosine Loss : local CV score(train and val) is improved, but LB is not.\n- Tempered Loss : local CV accuracy is not improved, but LB is not.\n- Smoothing Label : local CV accuracy is not improved, but LB is not.  However, local CV and public is nearest.\n\nAnd, I noticed Vision Transformer may be good at public LB, because of its local CV is worse than other model toward public LB score.\n\nViT predict to only 1 image we can access for test is\narray([[0.01223303, 0.01697356, 0.65529186, 0.04579847, 0.26970305]])\n( this may be worse predict).\nI think this ViT is not trained well, but public LB is not worse.\n\nFor this reason, I think public LB data contains some noise, so confused random prediction accidently contribute to score.",
      "votes": null
    },
    {
      "id": "1151099",
      "postDate": "01/13/2021 06:24:28",
      "content": "<p>Tempered loss has a little effect</p>\n<p>A proper value is effective</p>\n<p>Softmax cross entropy is equivalent to tempered loss, but because tempered loss does not do many experiments, it is not easy to compare</p>\n<p>Focal cosine loss has never been tested</p>",
      "rawMarkdown": "Tempered loss has a little effect\n\nA proper value is effective\n\nSoftmax cross entropy is equivalent to tempered loss, but because tempered loss does not do many experiments, it is not easy to compare\n\nFocal cosine loss has never been tested",
      "votes": null
    },
    {
      "id": "1151135",
      "postDate": "01/13/2021 06:46:57",
      "content": "<p>Symmetric Cross Entropy Loss：CV and LB are not improved.</p>",
      "rawMarkdown": "Symmetric Cross Entropy Loss：CV and LB are not improved.",
      "votes": null
    },
    {
      "id": "1151142",
      "postDate": "01/13/2021 06:53:33",
      "content": "<p>You are right. My understanding is that this loss function can improve the score when it is applied to the image with approximate symmetry. But in this project, there are almost no symmetrical images, and the background is noisy, so it is not applicable?</p>",
      "rawMarkdown": "You are right. My understanding is that this loss function can improve the score when it is applied to the image with approximate symmetry. But in this project, there are almost no symmetrical images, and the background is noisy, so it is not applicable?",
      "votes": null
    },
    {
      "id": "1151830",
      "postDate": "01/13/2021 15:39:54",
      "content": "<p>Thank you for your reply.<br>\nLoss may not be critical idea for improving scorre…</p>",
      "rawMarkdown": "Thank you for your reply.\nLoss may not be critical idea for improving scorre...",
      "votes": null
    },
    {
      "id": "1151854",
      "postDate": "01/13/2021 16:00:43",
      "content": "<p>I think symetric noise means that some images are labeled incorectly between 2 classes. ex: an image can be labeled healthy when its unhealthy and vice-versa</p>",
      "rawMarkdown": "I think symetric noise means that some images are labeled incorectly between 2 classes. ex: an image can be labeled healthy when its unhealthy and vice-versa",
      "votes": null
    },
    {
      "id": "1152202",
      "postDate": "01/13/2021 23:25:09",
      "content": "<p>I think tempered do not work.<br>\nWhat's your t1, t2?</p>",
      "rawMarkdown": "I think tempered do not work.\nWhat's your t1, t2?",
      "votes": null
    },
    {
      "id": "1165605",
      "postDate": "01/23/2021 05:42:00",
      "content": "<p>Soryy, my train setting is not best, so i seek to other parameters…</p>",
      "rawMarkdown": "Soryy, my train setting is not best, so i seek to other parameters...",
      "votes": null
    },
    {
      "id": "1165612",
      "postDate": "01/23/2021 05:59:32",
      "content": "<p>Sorry, I just noticed your reply<br>\nt1=0.2 t2=1.0</p>",
      "rawMarkdown": "Sorry, I just noticed your reply\nt1=0.2 t2=1.0",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1151099,
      "author_name": "zhangeng",
      "author_url": "",
      "post_date": "01/13/2021 06:24:28",
      "content": "<p>Tempered loss has a little effect</p>\n<p>A proper value is effective</p>\n<p>Softmax cross entropy is equivalent to tempered loss, but because tempered loss does not do many experiments, it is not easy to compare</p>\n<p>Focal cosine loss has never been tested</p>",
      "votes": null,
      "replies": [
        {
          "id": 1152202,
          "author_name": "yoshito",
          "author_url": "",
          "post_date": "01/13/2021 23:25:09",
          "content": "<p>I think tempered do not work.<br>\nWhat's your t1, t2?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1165612,
          "author_name": "zhangeng",
          "author_url": "",
          "post_date": "01/23/2021 05:59:32",
          "content": "<p>Sorry, I just noticed your reply<br>\nt1=0.2 t2=1.0</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1151135,
      "author_name": "denglin",
      "author_url": "",
      "post_date": "01/13/2021 06:46:57",
      "content": "<p>Symmetric Cross Entropy Loss：CV and LB are not improved.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1151142,
          "author_name": "zhangeng",
          "author_url": "",
          "post_date": "01/13/2021 06:53:33",
          "content": "<p>You are right. My understanding is that this loss function can improve the score when it is applied to the image with approximate symmetry. But in this project, there are almost no symmetrical images, and the background is noisy, so it is not applicable?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1151854,
          "author_name": "yannmajewski",
          "author_url": "",
          "post_date": "01/13/2021 16:00:43",
          "content": "<p>I think symetric noise means that some images are labeled incorectly between 2 classes. ex: an image can be labeled healthy when its unhealthy and vice-versa</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1151830,
      "author_name": "yoshito",
      "author_url": "",
      "post_date": "01/13/2021 15:39:54",
      "content": "<p>Thank you for your reply.<br>\nLoss may not be critical idea for improving scorre…</p>",
      "votes": null,
      "replies": [
        {
          "id": 1165605,
          "author_name": "yoshito",
          "author_url": "",
          "post_date": "01/23/2021 05:42:00",
          "content": "<p>Soryy, my train setting is not best, so i seek to other parameters…</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1150930": "I tried some loss functions which contain discussed one.\n\n- Softmax Cross Entropy\n- Smoothing Label\n- Focal Cosine Loss\n- Tempered Loss\n\nbut public LB is not improved.\n\nPlease let me your loss strategy, if you can...\n\nI noticed below from my experiments.\n\n- Focal Cosine Loss : local CV score(train and val) is improved, but LB is not.\n- Tempered Loss : local CV accuracy is not improved, but LB is not.\n- Smoothing Label : local CV accuracy is not improved, but LB is not.  However, local CV and public is nearest.\n\nAnd, I noticed Vision Transformer may be good at public LB, because of its local CV is worse than other model toward public LB score.\n\nViT predict to only 1 image we can access for test is\narray([[0.01223303, 0.01697356, 0.65529186, 0.04579847, 0.26970305]])\n( this may be worse predict).\nI think this ViT is not trained well, but public LB is not worse.\n\nFor this reason, I think public LB data contains some noise, so confused random prediction accidently contribute to score.",
    "1151099": "Tempered loss has a little effect\n\nA proper value is effective\n\nSoftmax cross entropy is equivalent to tempered loss, but because tempered loss does not do many experiments, it is not easy to compare\n\nFocal cosine loss has never been tested",
    "1151135": "Symmetric Cross Entropy Loss：CV and LB are not improved.",
    "1151142": "You are right. My understanding is that this loss function can improve the score when it is applied to the image with approximate symmetry. But in this project, there are almost no symmetrical images, and the background is noisy, so it is not applicable?",
    "1151830": "Thank you for your reply.\nLoss may not be critical idea for improving scorre...",
    "1151854": "I think symetric noise means that some images are labeled incorectly between 2 classes. ex: an image can be labeled healthy when its unhealthy and vice-versa",
    "1152202": "I think tempered do not work.\nWhat's your t1, t2?",
    "1165605": "Soryy, my train setting is not best, so i seek to other parameters...",
    "1165612": "Sorry, I just noticed your reply\nt1=0.2 t2=1.0"
  },
  "source": "meta"
}