{
  "id": 126973,
  "title": "How to train model with tf.GradientTape() ?",
  "url": "/competitions/bengaliai-cv19/discussion/126973",
  "author_name": "nadare",
  "post_date": "2020-01-21T13:48:29.493000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I want to train my models with tf.GradientTape() and wrote following code.\n```\n@tf.function\ndef train_step(images, labels):\n    with tf.GradientTape() as tape:\n        preds = model(images)\n        loss_root = tf.losses.categorical_crossentropy(labels[0], preds[0])\n        loss_vowel = tf.losses.categorical_crossentropy(labels[1], preds[1])\n        loss_consonant = tf.losses.categorical_crossentropy(labels[2], preds[2])\n        loss = [loss_root, loss_vowel, loss_consonant]\n    grad = tape.gradient(loss, model.trainable_variables)\n    return grad</p>\n\n<p>train_dataset = tf.data.Dataset.from_tensor_slices((X_train, (y_root_train, y_vowel_train, y_consonant_train)))\\\n                       .shuffle(buffer_size=train_size)\\\n                       .batch(batch_size)\\\n                       .prefetch(-1)</p>\n\n<p>for images, labels in (train_dataset):\n    grad = train_step(images, labels)\n    optimizer.apply_gradients(zip(grad, model.trainable_variables))\n```</p>\n\n<p>but it doesn`t seems to work. So what should I do?</p>",
  "messages": [
    {
      "id": 724803,
      "postDate": "2020-01-21T13:48:29.493Z",
      "content": "<p>I want to train my models with tf.GradientTape() and wrote following code.\n```\n@tf.function\ndef train_step(images, labels):\n    with tf.GradientTape() as tape:\n        preds = model(images)\n        loss_root = tf.losses.categorical_crossentropy(labels[0], preds[0])\n        loss_vowel = tf.losses.categorical_crossentropy(labels[1], preds[1])\n        loss_consonant = tf.losses.categorical_crossentropy(labels[2], preds[2])\n        loss = [loss_root, loss_vowel, loss_consonant]\n    grad = tape.gradient(loss, model.trainable_variables)\n    return grad</p>\n\n<p>train_dataset = tf.data.Dataset.from_tensor_slices((X_train, (y_root_train, y_vowel_train, y_consonant_train)))\\\n                       .shuffle(buffer_size=train_size)\\\n                       .batch(batch_size)\\\n                       .prefetch(-1)</p>\n\n<p>for images, labels in (train_dataset):\n    grad = train_step(images, labels)\n    optimizer.apply_gradients(zip(grad, model.trainable_variables))\n```</p>\n\n<p>but it doesn`t seems to work. So what should I do?</p>",
      "rawMarkdown": "I want to train my models with tf.GradientTape() and wrote following code.\n```\n@tf.function\ndef train_step(images, labels):\n    with tf.GradientTape() as tape:\n        preds = model(images)\n        loss_root = tf.losses.categorical_crossentropy(labels[0], preds[0])\n        loss_vowel = tf.losses.categorical_crossentropy(labels[1], preds[1])\n        loss_consonant = tf.losses.categorical_crossentropy(labels[2], preds[2])\n        loss = [loss_root, loss_vowel, loss_consonant]\n    grad = tape.gradient(loss, model.trainable_variables)\n    return grad\n\ntrain_dataset = tf.data.Dataset.from_tensor_slices((X_train, (y_root_train, y_vowel_train, y_consonant_train)))\\\n                       .shuffle(buffer_size=train_size)\\\n                       .batch(batch_size)\\\n                       .prefetch(-1)\n\nfor images, labels in (train_dataset):\n    grad = train_step(images, labels)\n    optimizer.apply_gradients(zip(grad, model.trainable_variables))\n```\n\nbut it doesn`t seems to work. So what should I do?",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "724803": "I want to train my models with tf.GradientTape() and wrote following code.\n```\n@tf.function\ndef train_step(images, labels):\n    with tf.GradientTape() as tape:\n        preds = model(images)\n        loss_root = tf.losses.categorical_crossentropy(labels[0], preds[0])\n        loss_vowel = tf.losses.categorical_crossentropy(labels[1], preds[1])\n        loss_consonant = tf.losses.categorical_crossentropy(labels[2], preds[2])\n        loss = [loss_root, loss_vowel, loss_consonant]\n    grad = tape.gradient(loss, model.trainable_variables)\n    return grad\n\ntrain_dataset = tf.data.Dataset.from_tensor_slices((X_train, (y_root_train, y_vowel_train, y_consonant_train)))\\\n                       .shuffle(buffer_size=train_size)\\\n                       .batch(batch_size)\\\n                       .prefetch(-1)\n\nfor images, labels in (train_dataset):\n    grad = train_step(images, labels)\n    optimizer.apply_gradients(zip(grad, model.trainable_variables))\n```\n\nbut it doesn`t seems to work. So what should I do?"
  }
}