{
  "id": 99421,
  "title": "Huge difference in accuracy with different loss functions",
  "url": "/competitions/recursion-cellular-image-classification/discussion/99421",
  "author_name": "",
  "post_date": "2019-07-11T08:09:58.464954900Z",
  "votes": 2,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Has anyone noticed huge differences in terms of accuracy when using different loss functions (I'm using PyTorch)?</p>",
  "messages": [
    {
      "id": "572652",
      "postDate": "07/11/2019 08:09:58",
      "content": "<p>Has anyone noticed huge differences in terms of accuracy when using different loss functions (I'm using PyTorch)?</p>",
      "rawMarkdown": "Has anyone noticed huge differences in terms of accuracy when using different loss functions (I'm using PyTorch)?",
      "votes": null
    },
    {
      "id": "572693",
      "postDate": "07/11/2019 09:20:21",
      "content": "<p>Which loss functions do you mean?</p>",
      "rawMarkdown": "Which loss functions do you mean?",
      "votes": null
    },
    {
      "id": "572697",
      "postDate": "07/11/2019 09:25:34",
      "content": "<p>Simple cross entropy and BCEWithLogitsLoss. With the former I cannot go above 0.09 while with the latter I can go to ~0.3 with a very simple CNN. Same model for both losses.</p>",
      "rawMarkdown": "Simple cross entropy and BCEWithLogitsLoss. With the former I cannot go above 0.09 while with the latter I can go to ~0.3 with a very simple CNN. Same model for both losses.",
      "votes": null
    },
    {
      "id": "572757",
      "postDate": "07/11/2019 11:22:24",
      "content": "<p>Interesting, I never tried BCE here. 0.3 is quite high, is this a public LB score?</p>",
      "rawMarkdown": "Interesting, I never tried BCE here. 0.3 is quite high, is this a public LB score?",
      "votes": null
    },
    {
      "id": "572760",
      "postDate": "07/11/2019 11:28:07",
      "content": "<p>Thats quite surprising, maybe you have some kind of bug in your CE, or maybe BCE is just much more suitable for this task, I will test BCE to find out.</p>",
      "rawMarkdown": "Thats quite surprising, maybe you have some kind of bug in your CE, or maybe BCE is just much more suitable for this task, I will test BCE to find out.",
      "votes": null
    },
    {
      "id": "572769",
      "postDate": "07/11/2019 11:37:49",
      "content": "<p>I still have to make a submission since I keep observing this strange behavior.</p>\n\n<p>I do not think there's any bug, since I tested it on a PyTorch dataset and it works!</p>",
      "rawMarkdown": "I still have to make a submission since I keep observing this strange behavior.\n\nI do not think there's any bug, since I tested it on a PyTorch dataset and it works!",
      "votes": null
    },
    {
      "id": "572821",
      "postDate": "07/11/2019 12:42:24",
      "content": "<p>I observe opposite behavior, in my case CE performs much better than BCE</p>",
      "rawMarkdown": "I observe opposite behavior, in my case CE performs much better than BCE",
      "votes": null
    },
    {
      "id": "572825",
      "postDate": "07/11/2019 12:48:53",
      "content": "<p>Well, thanks a lot. I really don't know what's wrong then!</p>",
      "rawMarkdown": "Well, thanks a lot. I really don't know what's wrong then!",
      "votes": null
    },
    {
      "id": "572843",
      "postDate": "07/11/2019 13:09:16",
      "content": "<p>Be sure to use correct СЕ implementation, some accept unnormalized scores (logits) as input, and others accept probabilities (softmax/sigmoid)</p>",
      "rawMarkdown": "Be sure to use correct СЕ implementation, some accept unnormalized scores (logits) as input, and others accept probabilities (softmax/sigmoid)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 572693,
      "author_name": "lopuhin",
      "author_url": "",
      "post_date": "07/11/2019 09:20:21",
      "content": "<p>Which loss functions do you mean?</p>",
      "votes": null,
      "replies": [
        {
          "id": 572697,
          "author_name": "lorenzofabbri92",
          "author_url": "",
          "post_date": "07/11/2019 09:25:34",
          "content": "<p>Simple cross entropy and BCEWithLogitsLoss. With the former I cannot go above 0.09 while with the latter I can go to ~0.3 with a very simple CNN. Same model for both losses.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 572757,
          "author_name": "lopuhin",
          "author_url": "",
          "post_date": "07/11/2019 11:22:24",
          "content": "<p>Interesting, I never tried BCE here. 0.3 is quite high, is this a public LB score?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 572760,
          "author_name": "vshmyhlo",
          "author_url": "",
          "post_date": "07/11/2019 11:28:07",
          "content": "<p>Thats quite surprising, maybe you have some kind of bug in your CE, or maybe BCE is just much more suitable for this task, I will test BCE to find out.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 572769,
          "author_name": "lorenzofabbri92",
          "author_url": "",
          "post_date": "07/11/2019 11:37:49",
          "content": "<p>I still have to make a submission since I keep observing this strange behavior.</p>\n\n<p>I do not think there's any bug, since I tested it on a PyTorch dataset and it works!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 572821,
          "author_name": "vshmyhlo",
          "author_url": "",
          "post_date": "07/11/2019 12:42:24",
          "content": "<p>I observe opposite behavior, in my case CE performs much better than BCE</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 572825,
          "author_name": "lorenzofabbri92",
          "author_url": "",
          "post_date": "07/11/2019 12:48:53",
          "content": "<p>Well, thanks a lot. I really don't know what's wrong then!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 572843,
          "author_name": "vshmyhlo",
          "author_url": "",
          "post_date": "07/11/2019 13:09:16",
          "content": "<p>Be sure to use correct СЕ implementation, some accept unnormalized scores (logits) as input, and others accept probabilities (softmax/sigmoid)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "572652": "Has anyone noticed huge differences in terms of accuracy when using different loss functions (I'm using PyTorch)?",
    "572693": "Which loss functions do you mean?",
    "572697": "Simple cross entropy and BCEWithLogitsLoss. With the former I cannot go above 0.09 while with the latter I can go to ~0.3 with a very simple CNN. Same model for both losses.",
    "572757": "Interesting, I never tried BCE here. 0.3 is quite high, is this a public LB score?",
    "572760": "Thats quite surprising, maybe you have some kind of bug in your CE, or maybe BCE is just much more suitable for this task, I will test BCE to find out.",
    "572769": "I still have to make a submission since I keep observing this strange behavior.\n\nI do not think there's any bug, since I tested it on a PyTorch dataset and it works!",
    "572821": "I observe opposite behavior, in my case CE performs much better than BCE",
    "572825": "Well, thanks a lot. I really don't know what's wrong then!",
    "572843": "Be sure to use correct СЕ implementation, some accept unnormalized scores (logits) as input, and others accept probabilities (softmax/sigmoid)"
  },
  "source": "meta"
}