{
  "id": 210505,
  "title": "TPU UnimplementedError",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/210505",
  "author_name": "jim frank",
  "post_date": "2021-01-11T03:04:12.025000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I'm a beginner of tpu programming. <br>\nI try to train a shufflenet model in tpu with cat-vs-dog dataset. The label of picture is numerical  not one-hot.The loss is <code>tf.keras.losses.BinaryCrossentropy</code>.But  i have met a strange problem which I have pasted  below. MeanWhile I have used <code>drop_remainder=True</code>,but it didn't work.</p>\n<pre><code>UnimplementedError: {{function_node __inference_train_function_433786}} Compilation failure: Asked to propagate a dynamic dimension from hlo convolution.6864@{}@2 to hlo %all-reduce.6869 = f32[3,3,&lt;=3,24]{3,2,1,0} all-reduce(f32[3,3,&lt;=3,24]{3,2,1,0} %convolution.6864), replica_groups={{0,1,2,3,4,5,6,7}}, to_apply=%sum.6865, metadata={op_type=\"CrossReplicaSum\" op_name=\"CrossReplicaSum_2\"}, which is not implemented.\n    TPU compilation failed\n     [[{{node tpu_compile_succeeded_assert/_3737259795873226432/_5}}]]\n</code></pre>",
  "messages": [
    {
      "id": 1148259,
      "postDate": "2021-01-11T03:04:12.027Z",
      "content": "<p>I'm a beginner of tpu programming. <br>\nI try to train a shufflenet model in tpu with cat-vs-dog dataset. The label of picture is numerical  not one-hot.The loss is <code>tf.keras.losses.BinaryCrossentropy</code>.But  i have met a strange problem which I have pasted  below. MeanWhile I have used <code>drop_remainder=True</code>,but it didn't work.</p>\n<pre><code>UnimplementedError: {{function_node __inference_train_function_433786}} Compilation failure: Asked to propagate a dynamic dimension from hlo convolution.6864@{}@2 to hlo %all-reduce.6869 = f32[3,3,&lt;=3,24]{3,2,1,0} all-reduce(f32[3,3,&lt;=3,24]{3,2,1,0} %convolution.6864), replica_groups={{0,1,2,3,4,5,6,7}}, to_apply=%sum.6865, metadata={op_type=\"CrossReplicaSum\" op_name=\"CrossReplicaSum_2\"}, which is not implemented.\n    TPU compilation failed\n     [[{{node tpu_compile_succeeded_assert/_3737259795873226432/_5}}]]\n</code></pre>",
      "rawMarkdown": "I'm a beginner of tpu programming. \nI try to train a shufflenet model in tpu with cat-vs-dog dataset. The label of picture is numerical  not one-hot.The loss is `tf.keras.losses.BinaryCrossentropy`.But  i have met a strange problem which I have pasted  below. MeanWhile I have used `drop_remainder=True`,but it didn't work.\n\n\n\n```\nUnimplementedError: {{function_node __inference_train_function_433786}} Compilation failure: Asked to propagate a dynamic dimension from hlo convolution.6864@{}@2 to hlo %all-reduce.6869 = f32[3,3,<=3,24]{3,2,1,0} all-reduce(f32[3,3,<=3,24]{3,2,1,0} %convolution.6864), replica_groups={{0,1,2,3,4,5,6,7}}, to_apply=%sum.6865, metadata={op_type=\"CrossReplicaSum\" op_name=\"CrossReplicaSum_2\"}, which is not implemented.\n\tTPU compilation failed\n\t [[{{node tpu_compile_succeeded_assert/_3737259795873226432/_5}}]]\n```"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1148259": "I'm a beginner of tpu programming. \nI try to train a shufflenet model in tpu with cat-vs-dog dataset. The label of picture is numerical  not one-hot.The loss is `tf.keras.losses.BinaryCrossentropy`.But  i have met a strange problem which I have pasted  below. MeanWhile I have used `drop_remainder=True`,but it didn't work.\n\n\n\n```\nUnimplementedError: {{function_node __inference_train_function_433786}} Compilation failure: Asked to propagate a dynamic dimension from hlo convolution.6864@{}@2 to hlo %all-reduce.6869 = f32[3,3,<=3,24]{3,2,1,0} all-reduce(f32[3,3,<=3,24]{3,2,1,0} %convolution.6864), replica_groups={{0,1,2,3,4,5,6,7}}, to_apply=%sum.6865, metadata={op_type=\"CrossReplicaSum\" op_name=\"CrossReplicaSum_2\"}, which is not implemented.\n\tTPU compilation failed\n\t [[{{node tpu_compile_succeeded_assert/_3737259795873226432/_5}}]]\n```"
  }
}