{
  "id": 144641,
  "title": "How to use label smoothing on tpu, I failed, but gpu can run",
  "url": "/competitions/flower-classification-with-tpus/discussion/144641",
  "author_name": "",
  "post_date": "2020-04-20T01:16:09.041489600Z",
  "votes": 1,
  "comment_count": 6,
  "views": 0,
  "content": "<p>How to use label smoothing on tpu, I failed, but gpu can run</p>",
  "messages": [
    {
      "id": "813754",
      "postDate": "04/20/2020 01:16:09",
      "content": "<p>How to use label smoothing on tpu, I failed, but gpu can run</p>",
      "rawMarkdown": "How to use label smoothing on tpu, I failed, but gpu can run",
      "votes": null
    },
    {
      "id": "813777",
      "postDate": "04/20/2020 01:49:20",
      "content": "<p>If you are using tensoflow/keras framework, I think label smoothing can be applied by simply change the parameters in loss function.</p>\n\n<p><code>tf.keras.losses.categorical_crossentropy(\n    y_true, y_pred, from_logits=False, label_smoothing=0\n)</code></p>\n\n<p><a href=\"https://www.tensorflow.org/api_docs/python/tf/keras/losses/categorical_crossentropy\">tf.keras.losses.categorical_crossentropy</a></p>",
      "rawMarkdown": "If you are using tensoflow/keras framework, I think label smoothing can be applied by simply change the parameters in loss function.\n\n`tf.keras.losses.categorical_crossentropy(\n    y_true, y_pred, from_logits=False, label_smoothing=0\n)`\n\n[tf.keras.losses.categorical_crossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/categorical_crossentropy)",
      "votes": null
    },
    {
      "id": "813788",
      "postDate": "04/20/2020 02:11:45",
      "content": "<p>I changed one-hot and also changed this, but an error will be reported. Can I achieve smoothness just by changing loss?</p>",
      "rawMarkdown": "I changed one-hot and also changed this, but an error will be reported. Can I achieve smoothness just by changing loss?",
      "votes": null
    },
    {
      "id": "813789",
      "postDate": "04/20/2020 02:16:34",
      "content": "<p>The error message:\nInvalidArgumentError: Unable to find a context_id matching the specified one (6504725637968138488). Perhaps the worker was restarted, or the context was GC'd?</p>",
      "rawMarkdown": "The error message:\nInvalidArgumentError: Unable to find a context_id matching the specified one (6504725637968138488). Perhaps the worker was restarted, or the context was GC'd?",
      "votes": null
    },
    {
      "id": "813794",
      "postDate": "04/20/2020 02:26:49",
      "content": "<p><code>def CCE_labelsmoothing(y_true, y_pred):</code>\n<code>return tf.keras.losses.categorical_crossentropy(y_true=y_true, y_pred=y_pred, from_logits=False, label_smoothing=0.1)</code></p>\n\n<p><code>model.compile(optimizer='adam', loss=CCE_labelsmoothing, metrics=['accuracy']</code>\nThis works for me, maybe you can try it.</p>",
      "rawMarkdown": "`def CCE_labelsmoothing(y_true, y_pred):`\n`return tf.keras.losses.categorical_crossentropy(y_true=y_true, y_pred=y_pred, from_logits=False, label_smoothing=0.1)`\n\n`model.compile(optimizer='adam', loss=CCE_labelsmoothing, metrics=['accuracy']`\nThis works for me, maybe you can try it.",
      "votes": null
    },
    {
      "id": "813832",
      "postDate": "04/20/2020 04:08:53",
      "content": "<p>thanks you</p>",
      "rawMarkdown": "thanks you",
      "votes": null
    },
    {
      "id": "814744",
      "postDate": "04/20/2020 23:40:04",
      "content": "<p>In Keras, <code>model.fit</code> supports a <code>class_weight</code> parameter. You can provide either a list or a dictionary as value. Unfortunately on TPU, only the list of class weights works. This is fixed in the upcoming TF 2.2 release.</p>",
      "rawMarkdown": "In Keras, `model.fit` supports a `class_weight` parameter. You can provide either a list or a dictionary as value. Unfortunately on TPU, only the list of class weights works. This is fixed in the upcoming TF 2.2 release.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 813777,
      "author_name": "xiejialun",
      "author_url": "",
      "post_date": "04/20/2020 01:49:20",
      "content": "<p>If you are using tensoflow/keras framework, I think label smoothing can be applied by simply change the parameters in loss function.</p>\n\n<p><code>tf.keras.losses.categorical_crossentropy(\n    y_true, y_pred, from_logits=False, label_smoothing=0\n)</code></p>\n\n<p><a href=\"https://www.tensorflow.org/api_docs/python/tf/keras/losses/categorical_crossentropy\">tf.keras.losses.categorical_crossentropy</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 813788,
          "author_name": "catchlove",
          "author_url": "",
          "post_date": "04/20/2020 02:11:45",
          "content": "<p>I changed one-hot and also changed this, but an error will be reported. Can I achieve smoothness just by changing loss?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 813794,
          "author_name": "xiejialun",
          "author_url": "",
          "post_date": "04/20/2020 02:26:49",
          "content": "<p><code>def CCE_labelsmoothing(y_true, y_pred):</code>\n<code>return tf.keras.losses.categorical_crossentropy(y_true=y_true, y_pred=y_pred, from_logits=False, label_smoothing=0.1)</code></p>\n\n<p><code>model.compile(optimizer='adam', loss=CCE_labelsmoothing, metrics=['accuracy']</code>\nThis works for me, maybe you can try it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 813832,
          "author_name": "catchlove",
          "author_url": "",
          "post_date": "04/20/2020 04:08:53",
          "content": "<p>thanks you</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 814744,
          "author_name": "mgorner",
          "author_url": "",
          "post_date": "04/20/2020 23:40:04",
          "content": "<p>In Keras, <code>model.fit</code> supports a <code>class_weight</code> parameter. You can provide either a list or a dictionary as value. Unfortunately on TPU, only the list of class weights works. This is fixed in the upcoming TF 2.2 release.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 813789,
      "author_name": "catchlove",
      "author_url": "",
      "post_date": "04/20/2020 02:16:34",
      "content": "<p>The error message:\nInvalidArgumentError: Unable to find a context_id matching the specified one (6504725637968138488). Perhaps the worker was restarted, or the context was GC'd?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "813754": "How to use label smoothing on tpu, I failed, but gpu can run",
    "813777": "If you are using tensoflow/keras framework, I think label smoothing can be applied by simply change the parameters in loss function.\n\n`tf.keras.losses.categorical_crossentropy(\n    y_true, y_pred, from_logits=False, label_smoothing=0\n)`\n\n[tf.keras.losses.categorical_crossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/categorical_crossentropy)",
    "813788": "I changed one-hot and also changed this, but an error will be reported. Can I achieve smoothness just by changing loss?",
    "813789": "The error message:\nInvalidArgumentError: Unable to find a context_id matching the specified one (6504725637968138488). Perhaps the worker was restarted, or the context was GC'd?",
    "813794": "`def CCE_labelsmoothing(y_true, y_pred):`\n`return tf.keras.losses.categorical_crossentropy(y_true=y_true, y_pred=y_pred, from_logits=False, label_smoothing=0.1)`\n\n`model.compile(optimizer='adam', loss=CCE_labelsmoothing, metrics=['accuracy']`\nThis works for me, maybe you can try it.",
    "813832": "thanks you",
    "814744": "In Keras, `model.fit` supports a `class_weight` parameter. You can provide either a list or a dictionary as value. Unfortunately on TPU, only the list of class weights works. This is fixed in the upcoming TF 2.2 release."
  },
  "source": "meta"
}