{
  "id": 313644,
  "title": "Focal loss tensorflow + TPU",
  "url": "/competitions/happy-whale-and-dolphin/discussion/313644",
  "author_name": "",
  "post_date": "2022-03-18T06:43:21.232635800Z",
  "votes": 2,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Anyone try Focal for TF + TPU?, I tried but the loss decreased very slow and then become nan. Is it a problem of TPU?</p>\n<p>The focal loss function I taken from <a href=\"https://github.com/artemmavrin/focal-loss\" target=\"_blank\">https://github.com/artemmavrin/focal-loss</a></p>",
  "messages": [
    {
      "id": "1727662",
      "postDate": "03/18/2022 06:43:21",
      "content": "<p>Anyone try Focal for TF + TPU?, I tried but the loss decreased very slow and then become nan. Is it a problem of TPU?</p>\n<p>The focal loss function I taken from <a href=\"https://github.com/artemmavrin/focal-loss\" target=\"_blank\">https://github.com/artemmavrin/focal-loss</a></p>",
      "rawMarkdown": "Anyone try Focal for TF + TPU?, I tried but the loss decreased very slow and then become nan. Is it a problem of TPU?\n\nThe focal loss function I taken from https://github.com/artemmavrin/focal-loss",
      "votes": null
    },
    {
      "id": "1727703",
      "postDate": "03/18/2022 07:31:10",
      "content": "<p>the loss itself doesn't matter that much? </p>\n<p>because I tuned  same model parameters, with smaller val loss and better val acc, however, the LB almost same. </p>",
      "rawMarkdown": "the loss itself doesn't matter that much? \n\nbecause I tuned  same model parameters, with smaller val loss and better val acc, however, the LB almost same.",
      "votes": null
    },
    {
      "id": "1727712",
      "postDate": "03/18/2022 07:39:03",
      "content": "<p>The val acc is not always correlated with CV/LB but large gap between loss/val acc will be a problem</p>",
      "rawMarkdown": "The val acc is not always correlated with CV/LB but large gap between loss/val acc will be a problem",
      "votes": null
    },
    {
      "id": "1728470",
      "postDate": "03/18/2022 23:53:39",
      "content": "<p>I tried using the focal loss from this library but I kept getting tpu compilation errors. Did you encounter this error and solve it?</p>",
      "rawMarkdown": "I tried using the focal loss from this library but I kept getting tpu compilation errors. Did you encounter this error and solve it?",
      "votes": null
    },
    {
      "id": "1728800",
      "postDate": "03/19/2022 09:50:23",
      "content": "<p>I did not see any error on colab</p>",
      "rawMarkdown": "I did not see any error on colab",
      "votes": null
    },
    {
      "id": "1728925",
      "postDate": "03/19/2022 12:54:29",
      "content": "<p>Hm. I have been using kaggle TPU for my experiments so maybe the different TPU versions might mess something up?</p>",
      "rawMarkdown": "Hm. I have been using kaggle TPU for my experiments so maybe the different TPU versions might mess something up?",
      "votes": null
    },
    {
      "id": "1729449",
      "postDate": "03/20/2022 05:13:04",
      "content": "<p>Is Focal loss better than others?</p>",
      "rawMarkdown": "Is Focal loss better than others?",
      "votes": null
    },
    {
      "id": "1729534",
      "postDate": "03/20/2022 07:32:17",
      "content": "<p>I did not submit yet, the loss is become nan after few epochs</p>",
      "rawMarkdown": "I did not submit yet, the loss is become nan after few epochs",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1727703,
      "author_name": "dragonzhang",
      "author_url": "",
      "post_date": "03/18/2022 07:31:10",
      "content": "<p>the loss itself doesn't matter that much? </p>\n<p>because I tuned  same model parameters, with smaller val loss and better val acc, however, the LB almost same. </p>",
      "votes": null,
      "replies": [
        {
          "id": 1727712,
          "author_name": "ptran1203",
          "author_url": "",
          "post_date": "03/18/2022 07:39:03",
          "content": "<p>The val acc is not always correlated with CV/LB but large gap between loss/val acc will be a problem</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1729449,
          "author_name": "dragonzhang",
          "author_url": "",
          "post_date": "03/20/2022 05:13:04",
          "content": "<p>Is Focal loss better than others?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1729534,
          "author_name": "ptran1203",
          "author_url": "",
          "post_date": "03/20/2022 07:32:17",
          "content": "<p>I did not submit yet, the loss is become nan after few epochs</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1728470,
      "author_name": "vexxingbanana",
      "author_url": "",
      "post_date": "03/18/2022 23:53:39",
      "content": "<p>I tried using the focal loss from this library but I kept getting tpu compilation errors. Did you encounter this error and solve it?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1728800,
          "author_name": "ptran1203",
          "author_url": "",
          "post_date": "03/19/2022 09:50:23",
          "content": "<p>I did not see any error on colab</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1728925,
          "author_name": "vexxingbanana",
          "author_url": "",
          "post_date": "03/19/2022 12:54:29",
          "content": "<p>Hm. I have been using kaggle TPU for my experiments so maybe the different TPU versions might mess something up?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1727662": "Anyone try Focal for TF + TPU?, I tried but the loss decreased very slow and then become nan. Is it a problem of TPU?\n\nThe focal loss function I taken from https://github.com/artemmavrin/focal-loss",
    "1727703": "the loss itself doesn't matter that much? \n\nbecause I tuned  same model parameters, with smaller val loss and better val acc, however, the LB almost same.",
    "1727712": "The val acc is not always correlated with CV/LB but large gap between loss/val acc will be a problem",
    "1728470": "I tried using the focal loss from this library but I kept getting tpu compilation errors. Did you encounter this error and solve it?",
    "1728800": "I did not see any error on colab",
    "1728925": "Hm. I have been using kaggle TPU for my experiments so maybe the different TPU versions might mess something up?",
    "1729449": "Is Focal loss better than others?",
    "1729534": "I did not submit yet, the loss is become nan after few epochs"
  },
  "source": "meta"
}