{ "id": 209093, "title": "Getting TypeError: when using label smoothing in Categorical Cross entropy", "url": "/competitions/cassava-leaf-disease-classification/discussion/209093", "author_name": "", "post_date": "2021-01-06T08:49:09.429716900Z", "votes": null, "comment_count": 4, "views": 0, "content": "

I am using label smoothing with Categorical Cross entropy. When I turn off label smoothing the model works fine but when I turn it The model gives the following error
\nTypeError: Input 'y' of 'Mul' Op has type float32 that does not match type float16 of argument 'x'.\n

\n

I am not able to understand what is the problem with my code?. Here is My code
\ncode
\nI have tried setting the dtype to float16 in Image Data Generator. But that also does not work. Can somebody Help me?

", "messages": [ { "id": "1140763", "postDate": "01/06/2021 08:49:09", "content": "

I am using label smoothing with Categorical Cross entropy. When I turn off label smoothing the model works fine but when I turn it The model gives the following error
\nTypeError: Input 'y' of 'Mul' Op has type float32 that does not match type float16 of argument 'x'.\n

\n

I am not able to understand what is the problem with my code?. Here is My code
\ncode
\nI have tried setting the dtype to float16 in Image Data Generator. But that also does not work. Can somebody Help me?

", "rawMarkdown": "I am using label smoothing with Categorical Cross entropy. When I turn off label smoothing the model works fine but when I turn it The model gives the following error\n` TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type float16 of argument 'x'.\n`\n\nI am not able to understand what is the problem with my code?. Here is My code\n[code](https://drive.google.com/file/d/1lEVPMOtWOWesDmpRDgTJQPTFOQ6UTI2o/view?usp=sharing)\nI have tried setting the dtype to float16 in Image Data Generator. But that also does not work. Can somebody Help me?", "votes": null }, { "id": "1159138", "postDate": "01/19/2021 04:25:22", "content": "

I know its a bit late, but try this.
\nreplace:
\nDense(5, activation='softmax')
\nwith:
\nDense(5, activation='softmax',dtype='float32')

\n

This solved the problem for me, especially if you are using the mixed precision policy. (I think you need to make appropriate changes to all dense layers when we use mixed precision,although I dont't see the policy implemented in your code, just an import)

", "rawMarkdown": "I know its a bit late, but try this.\nreplace:\n`Dense(5, activation='softmax')`\nwith:\n`Dense(5, activation='softmax',dtype='float32')`\n\nThis solved the problem for me, especially if you are using the mixed precision policy. (I think you need to make appropriate changes to all dense layers when we use mixed precision,although I dont't see the policy implemented in your code, just an import)", "votes": null }, { "id": "1159485", "postDate": "01/19/2021 09:55:42", "content": "

Thanks man, That worked for me

", "rawMarkdown": "Thanks man, That worked for me", "votes": null }, { "id": "1159495", "postDate": "01/19/2021 10:02:11", "content": "

I think you only need to implement this on the last layer. And i Do have Mixed precision implemented. But thanks for the fix

", "rawMarkdown": "I think you only need to implement this on the last layer. And i Do have Mixed precision implemented. But thanks for the fix", "votes": null }, { "id": "1159602", "postDate": "01/19/2021 11:15:43", "content": "

Happy I could help.

", "rawMarkdown": "Happy I could help.", "votes": null } ], "comments": [ { "id": 1159138, "author_name": "mohneesh7", "author_url": "", "post_date": "01/19/2021 04:25:22", "content": "

I know its a bit late, but try this.
\nreplace:
\nDense(5, activation='softmax')
\nwith:
\nDense(5, activation='softmax',dtype='float32')

\n

This solved the problem for me, especially if you are using the mixed precision policy. (I think you need to make appropriate changes to all dense layers when we use mixed precision,although I dont't see the policy implemented in your code, just an import)

", "votes": null, "replies": [ { "id": 1159485, "author_name": "mithilsalunkhe", "author_url": "", "post_date": "01/19/2021 09:55:42", "content": "

Thanks man, That worked for me

", "votes": null, "replies": [] }, { "id": 1159495, "author_name": "mithilsalunkhe", "author_url": "", "post_date": "01/19/2021 10:02:11", "content": "

I think you only need to implement this on the last layer. And i Do have Mixed precision implemented. But thanks for the fix

", "votes": null, "replies": [] }, { "id": 1159602, "author_name": "mohneesh7", "author_url": "", "post_date": "01/19/2021 11:15:43", "content": "

Happy I could help.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1140763": "I am using label smoothing with Categorical Cross entropy. When I turn off label smoothing the model works fine but when I turn it The model gives the following error\n` TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type float16 of argument 'x'.\n`\n\nI am not able to understand what is the problem with my code?. Here is My code\n[code](https://drive.google.com/file/d/1lEVPMOtWOWesDmpRDgTJQPTFOQ6UTI2o/view?usp=sharing)\nI have tried setting the dtype to float16 in Image Data Generator. But that also does not work. Can somebody Help me?", "1159138": "I know its a bit late, but try this.\nreplace:\n`Dense(5, activation='softmax')`\nwith:\n`Dense(5, activation='softmax',dtype='float32')`\n\nThis solved the problem for me, especially if you are using the mixed precision policy. (I think you need to make appropriate changes to all dense layers when we use mixed precision,although I dont't see the policy implemented in your code, just an import)", "1159485": "Thanks man, That worked for me", "1159495": "I think you only need to implement this on the last layer. And i Do have Mixed precision implemented. But thanks for the fix", "1159602": "Happy I could help." }, "source": "meta" }