{
  "id": 203183,
  "title": "How to implement \"Gradient Accumulation\" with tensorflow/keras",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/203183",
  "author_name": "",
  "post_date": "2020-12-14T06:08:39.919903700Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I want to know how to implement \"Gradient Accumulation\" with tensorflow/keras .</p>",
  "messages": [
    {
      "id": "1111904",
      "postDate": "12/14/2020 06:08:39",
      "content": "<p>I want to know how to implement \"Gradient Accumulation\" with tensorflow/keras .</p>",
      "rawMarkdown": "I want to know how to implement \"Gradient Accumulation\" with tensorflow/keras .",
      "votes": null
    },
    {
      "id": "1111937",
      "postDate": "12/14/2020 06:37:14",
      "content": "<p>Sadly it's not that smooth to use as PyTorch. Two way to do this </p>\n<ul>\n<li>with optimizer (in keras): <a href=\"https://stackoverflow.com/a/55281501/9215780\" target=\"_blank\">https://stackoverflow.com/a/55281501/9215780</a> or, </li>\n<li>with custom training in tf.keras: <a href=\"https://stackoverflow.com/questions/59893850/how-to-accumulate-gradients-in-tensorflow-2-0\" target=\"_blank\">https://stackoverflow.com/questions/59893850/how-to-accumulate-gradients-in-tensorflow-2-0</a></li>\n</ul>",
      "rawMarkdown": "Sadly it's not that smooth to use as PyTorch. Two way to do this \n\n- with optimizer (in keras): https://stackoverflow.com/a/55281501/9215780 or, \n- with custom training in tf.keras: https://stackoverflow.com/questions/59893850/how-to-accumulate-gradients-in-tensorflow-2-0",
      "votes": null
    },
    {
      "id": "1112220",
      "postDate": "12/14/2020 11:56:41",
      "content": "<p>Thank you for your reply !! I will try to implement \"Gradient Accumulation\"  with custom training in tf.keras .  </p>",
      "rawMarkdown": "Thank you for your reply !! I will try to implement \"Gradient Accumulation\"  with custom training in tf.keras .",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1111937,
      "author_name": "ipythonx",
      "author_url": "",
      "post_date": "12/14/2020 06:37:14",
      "content": "<p>Sadly it's not that smooth to use as PyTorch. Two way to do this </p>\n<ul>\n<li>with optimizer (in keras): <a href=\"https://stackoverflow.com/a/55281501/9215780\" target=\"_blank\">https://stackoverflow.com/a/55281501/9215780</a> or, </li>\n<li>with custom training in tf.keras: <a href=\"https://stackoverflow.com/questions/59893850/how-to-accumulate-gradients-in-tensorflow-2-0\" target=\"_blank\">https://stackoverflow.com/questions/59893850/how-to-accumulate-gradients-in-tensorflow-2-0</a></li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 1112220,
          "author_name": "nishikattt",
          "author_url": "",
          "post_date": "12/14/2020 11:56:41",
          "content": "<p>Thank you for your reply !! I will try to implement \"Gradient Accumulation\"  with custom training in tf.keras .  </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1111904": "I want to know how to implement \"Gradient Accumulation\" with tensorflow/keras .",
    "1111937": "Sadly it's not that smooth to use as PyTorch. Two way to do this \n\n- with optimizer (in keras): https://stackoverflow.com/a/55281501/9215780 or, \n- with custom training in tf.keras: https://stackoverflow.com/questions/59893850/how-to-accumulate-gradients-in-tensorflow-2-0",
    "1112220": "Thank you for your reply !! I will try to implement \"Gradient Accumulation\"  with custom training in tf.keras ."
  },
  "source": "meta"
}