I am having what seems like connection issues when attempting to re-run my old notebboks on TPU , for example version 10 ran and the retry in version 11 did not run https://www.kaggle.com/felipebihaiek/fork-of-baseline-implementation-resnet34?scriptVersionId=53473064. This has been happening for the last 3 days.
\nThe message thrown is the following:
\nfailed to connect to all addresses\nAdditional GRPC error information from remote target /job:localhost/replica:0/task:0/device:CPU:0:
Thanks in advance for any help.
", "messages": [ { "id": "1185310", "postDate": "02/04/2021 05:36:42", "content": "I am having what seems like connection issues when attempting to re-run my old notebboks on TPU , for example version 10 ran and the retry in version 11 did not run https://www.kaggle.com/felipebihaiek/fork-of-baseline-implementation-resnet34?scriptVersionId=53473064. This has been happening for the last 3 days.
\nThe message thrown is the following:
\nfailed to connect to all addresses\nAdditional GRPC error information from remote target /job:localhost/replica:0/task:0/device:CPU:0:
Thanks in advance for any help.
", "rawMarkdown": "I am having what seems like connection issues when attempting to re-run my old notebboks on TPU , for example version 10 ran and the retry in version 11 did not run https://www.kaggle.com/felipebihaiek/fork-of-baseline-implementation-resnet34?scriptVersionId=53473064. This has been happening for the last 3 days.\n\nThe message thrown is the following:\n\n`failed to connect to all addresses\nAdditional GRPC error information from remote target /job:localhost/replica:0/task:0/device:CPU:0:`\n\nThanks in advance for any help.", "votes": null }, { "id": "1185878", "postDate": "02/04/2021 12:55:01", "content": "+1 to this issue
", "rawMarkdown": "1 to this issue", "votes": null }, { "id": "1186515", "postDate": "02/04/2021 21:34:26", "content": "I was told to add the complaint here https://www.kaggle.com/product-feedback/217011 if you are having the same issue please upvote or comment so that we get visibillity
", "rawMarkdown": "I was told to add the complaint here https://www.kaggle.com/product-feedback/217011 if you are having the same issue please upvote or comment so that we get visibillity", "votes": null }, { "id": "1186646", "postDate": "02/05/2021 01:01:45", "content": "I looked into this and it looks like tfa.image.cutout is not compatible with TPUs. I wonder if it has ever been or if it is a tensorflow-addons version issue.
", "rawMarkdown": "I looked into this and it looks like tfa.image.cutout is not compatible with TPUs. I wonder if it has ever been or if it is a tensorflow-addons version issue.", "votes": null }, { "id": "1186673", "postDate": "02/05/2021 01:28:18", "content": "Hi thank you. If it was not compatible then it was not throwing error or warning as you can see in version 10. BTW the kernel is a fork of a very upvoted kernel https://www.kaggle.com/prvnkmr/baseline-implementation-resnet34 which originally ran in TPU and used the cut_out. I will try a past version of TFA in any case.
", "rawMarkdown": "Hi thank you. If it was not compatible then it was not throwing error or warning as you can see in version 10. BTW the kernel is a fork of a very upvoted kernel https://www.kaggle.com/prvnkmr/baseline-implementation-resnet34 which originally ran in TPU and used the cut_out. I will try a past version of TFA in any case.", "votes": null }, { "id": "1186680", "postDate": "02/05/2021 01:43:28", "content": "The problem is we can only use the environment that has tensorflow 2.4 on the kernels ( ythe toggle for past image is disabled) . For tensorflo2.4 there is only one available version of TFA.
", "rawMarkdown": "The problem is we can only use the environment that has tensorflow 2.4 on the kernels ( ythe toggle for past image is disabled) . For tensorflo2.4 there is only one available version of TFA.", "votes": null }, { "id": "1186848", "postDate": "02/05/2021 04:48:17", "content": "There is a workaround available now. See comments by Kaggle support staff in :
\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/216408
other than specaugment you can get everything else to run just by commenting @tf.function decorator
", "rawMarkdown": "There is a workaround available now. See comments by Kaggle support staff in :\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/216408\n\nother than specaugment you can get everything else to run just by commenting @tf.function decorator", "votes": null } ], "comments": [ { "id": 1185878, "author_name": "germmie", "author_url": "", "post_date": "02/04/2021 12:55:01", "content": "+1 to this issue
", "votes": null, "replies": [] }, { "id": 1186515, "author_name": "felipebihaiek", "author_url": "", "post_date": "02/04/2021 21:34:26", "content": "I was told to add the complaint here https://www.kaggle.com/product-feedback/217011 if you are having the same issue please upvote or comment so that we get visibillity
", "votes": null, "replies": [] }, { "id": 1186646, "author_name": "mgorner", "author_url": "", "post_date": "02/05/2021 01:01:45", "content": "I looked into this and it looks like tfa.image.cutout is not compatible with TPUs. I wonder if it has ever been or if it is a tensorflow-addons version issue.
", "votes": null, "replies": [ { "id": 1186673, "author_name": "felipebihaiek", "author_url": "", "post_date": "02/05/2021 01:28:18", "content": "Hi thank you. If it was not compatible then it was not throwing error or warning as you can see in version 10. BTW the kernel is a fork of a very upvoted kernel https://www.kaggle.com/prvnkmr/baseline-implementation-resnet34 which originally ran in TPU and used the cut_out. I will try a past version of TFA in any case.
", "votes": null, "replies": [] }, { "id": 1186680, "author_name": "felipebihaiek", "author_url": "", "post_date": "02/05/2021 01:43:28", "content": "The problem is we can only use the environment that has tensorflow 2.4 on the kernels ( ythe toggle for past image is disabled) . For tensorflo2.4 there is only one available version of TFA.
", "votes": null, "replies": [] } ] }, { "id": 1186848, "author_name": "allohvk", "author_url": "", "post_date": "02/05/2021 04:48:17", "content": "There is a workaround available now. See comments by Kaggle support staff in :
\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/216408
other than specaugment you can get everything else to run just by commenting @tf.function decorator
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1185310": "I am having what seems like connection issues when attempting to re-run my old notebboks on TPU , for example version 10 ran and the retry in version 11 did not run https://www.kaggle.com/felipebihaiek/fork-of-baseline-implementation-resnet34?scriptVersionId=53473064. This has been happening for the last 3 days.\n\nThe message thrown is the following:\n\n`failed to connect to all addresses\nAdditional GRPC error information from remote target /job:localhost/replica:0/task:0/device:CPU:0:`\n\nThanks in advance for any help.", "1185878": "1 to this issue", "1186515": "I was told to add the complaint here https://www.kaggle.com/product-feedback/217011 if you are having the same issue please upvote or comment so that we get visibillity", "1186646": "I looked into this and it looks like tfa.image.cutout is not compatible with TPUs. I wonder if it has ever been or if it is a tensorflow-addons version issue.", "1186673": "Hi thank you. If it was not compatible then it was not throwing error or warning as you can see in version 10. BTW the kernel is a fork of a very upvoted kernel https://www.kaggle.com/prvnkmr/baseline-implementation-resnet34 which originally ran in TPU and used the cut_out. I will try a past version of TFA in any case.", "1186680": "The problem is we can only use the environment that has tensorflow 2.4 on the kernels ( ythe toggle for past image is disabled) . For tensorflo2.4 there is only one available version of TFA.", "1186848": "There is a workaround available now. See comments by Kaggle support staff in :\nhttps://www.kaggle.com/c/rfcx-species-audio-detection/discussion/216408\n\nother than specaugment you can get everything else to run just by commenting @tf.function decorator" }, "source": "meta" }