{ "id": 198272, "title": "Something wrong with organizer's tfrecords files. (closed)", "url": "/competitions/cassava-leaf-disease-classification/discussion/198272", "author_name": "", "post_date": "2020-11-20T14:03:45.629444600Z", "votes": 16, "comment_count": 22, "views": 0, "content": "

I think there are something wrong with official tfrecords files. By using tfrecords, CV stops at 0.61xxx. But by using jpeg file with the same model, I can easy get 0.83xx. @juliaelliott Could you check the official tfrecords. Thanks.
\nFor detail:
\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402
\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook

\n

The leaves in the red circle look like healthy (4)?
\n \"\"
\n\"\"

\n

Update (2020/11/26): The organizer has fixed the tfrecords.

", "messages": [ { "id": "1084909", "postDate": "11/20/2020 14:03:45", "content": "

I think there are something wrong with official tfrecords files. By using tfrecords, CV stops at 0.61xxx. But by using jpeg file with the same model, I can easy get 0.83xx. @juliaelliott Could you check the official tfrecords. Thanks.
\nFor detail:
\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402
\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook

\n

The leaves in the red circle look like healthy (4)?
\n \"\"
\n\"\"

\n

Update (2020/11/26): The organizer has fixed the tfrecords.

", "rawMarkdown": "I think there are something wrong with official tfrecords files. By using tfrecords, CV stops at 0.61xxx. But by using jpeg file with the same model, I can easy get 0.83xx. @juliaelliott Could you check the official tfrecords. Thanks.\nFor detail:\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook\n\nThe leaves in the red circle look like healthy (4)?\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2006644%2F2b8d371c0086ba44215c31e130df20d8%2F__results___30_0.png?generation=1605925654430111&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2006644%2F0dcb877a341a067be8ffbb61426c712b%2F__results___33_0.png?generation=1605925837212453&alt=media)\n\nUpdate (2020/11/26): The organizer has fixed the tfrecords.", "votes": null }, { "id": "1085138", "postDate": "11/20/2020 17:42:52", "content": "

@wuliaokaola -- Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. I believe other participants are also observing this here. But we encourage you to push the envelope further, as we're sure there are advancements to be made!

", "rawMarkdown": "wuliaokaola -- Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. I believe other participants are also [observing this here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198190). But we encourage you to push the envelope further, as we're sure there are advancements to be made!", "votes": null }, { "id": "1085461", "postDate": "11/20/2020 22:22:34", "content": "

Does it also apply to test data too?

", "rawMarkdown": "Does it also apply to test data too?", "votes": null }, { "id": "1085482", "postDate": "11/20/2020 22:51:54", "content": "

@awsaf49 Yes, the test images also have a tfrec equivalent in the same folder formatting as you see organized on the Data page.

", "rawMarkdown": "awsaf49 Yes, the test images also have a tfrec equivalent in the same folder formatting as you see organized on the Data page.", "votes": null }, { "id": "1085486", "postDate": "11/20/2020 23:08:06", "content": "

I'm wondering if the test tfrecord has the same error? Then should we avoid using tfrecord for competition submission?

", "rawMarkdown": "I'm wondering if the test tfrecord has the same error? Then should we avoid using tfrecord for competition submission?", "votes": null }, { "id": "1085596", "postDate": "11/21/2020 02:34:16", "content": "

@juliaelliott Thank you for your reply. But I still think there is something wrong with tfrecords, maybe the label. The images are come from official notebook. https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease
\nI don't think the leaves in red circle are healthy (4).

", "rawMarkdown": "juliaelliott Thank you for your reply. But I still think there is something wrong with tfrecords, maybe the label. The images are come from official notebook. https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease\nI don't think the leaves in red circle are healthy (4).", "votes": null }, { "id": "1085717", "postDate": "11/21/2020 06:28:26", "content": "

+1 this problem. My TFRecords model was always at 0.6x accuracy, while using ImageDataGenerator (or TF Dataset) I can easily get 0.8x.

", "rawMarkdown": "1 this problem. My TFRecords model was always at 0.6x accuracy, while using ImageDataGenerator (or TF Dataset) I can easily get 0.8x.", "votes": null }, { "id": "1086127", "postDate": "11/21/2020 12:09:42", "content": "

@juliaelliott There is no image_id in the tfrecords, I can't confirm it. But Please check these two notebooks.
\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402
\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook

", "rawMarkdown": "juliaelliott There is no image_id in the tfrecords, I can't confirm it. But Please check these two notebooks.\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook", "votes": null }, { "id": "1086325", "postDate": "11/21/2020 15:34:58", "content": "

@juliaelliott I have I think the same issue than @wuliaokaola . My submit work for the test, but as soon as the prediction is done on the hidden test set, I got an error which told me \"submission.csv not found\". I suspect there is no image_id or something goes wrong with the test tfrecord.

", "rawMarkdown": "juliaelliott I have I think the same issue than @wuliaokaola . My submit work for the test, but as soon as the prediction is done on the hidden test set, I got an error which told me \"submission.csv not found\". I suspect there is no image_id or something goes wrong with the test tfrecord.", "votes": null }, { "id": "1087563", "postDate": "11/22/2020 21:16:03", "content": "

I have also noticed this. After switching from loading images to records my score dropped significantly.

", "rawMarkdown": "I have also noticed this. After switching from loading images to records my score dropped significantly.", "votes": null }, { "id": "1087620", "postDate": "11/22/2020 23:03:42", "content": "

I think the TFRecords do not have the correct image names (as an aside, the field in TFRecord is image_name, not image_id).

\n

This notebook displays a TFRecord image and the corresponding jpeg:

\n

https://www.kaggle.com/richardepstein/check-tfrecords

\n

Organizers - please take a look.

\n

-Rich

", "rawMarkdown": "I think the TFRecords do not have the correct image names (as an aside, the field in TFRecord is image_name, not image_id).\n\nThis notebook displays a TFRecord image and the corresponding jpeg:\n\nhttps://www.kaggle.com/richardepstein/check-tfrecords\n\nOrganizers - please take a look.\n\n-Rich", "votes": null }, { "id": "1087622", "postDate": "11/22/2020 23:05:03", "content": "

Field is called \"image_name\" instead of \"image_id\".

\n

However, I think the field has the wrong data (see other reply in this thread).

\n

-Rich

", "rawMarkdown": "Field is called \"image_name\" instead of \"image_id\".\n\nHowever, I think the field has the wrong data (see other reply in this thread).\n\n-Rich", "votes": null }, { "id": "1088590", "postDate": "11/23/2020 19:16:42", "content": "

Thank you for your reports. We are investigating!

", "rawMarkdown": "Thank you for your reports. We are investigating!", "votes": null }, { "id": "1088915", "postDate": "11/24/2020 04:06:13", "content": "

So far, the tfrec format data set has not changed, so it is recommended to use JPG format data set at present. Because the original tfrec format data set has errors, it is difficult for training and prediction to reach the expected results, resulting in large errors. I doubt that the image and label can not be well one-to-one correspondence? Or?

", "rawMarkdown": "So far, the tfrec format data set has not changed, so it is recommended to use JPG format data set at present. Because the original tfrec format data set has errors, it is difficult for training and prediction to reach the expected results, resulting in large errors. I doubt that the image and label can not be well one-to-one correspondence? Or?", "votes": null }, { "id": "1088917", "postDate": "11/24/2020 04:10:08", "content": "

I created a datasets and made it public. You can use it to train your model.
\nhttps://www.kaggle.com/wuliaokaola/cldc-ds-1121-size-512

", "rawMarkdown": "I created a datasets and made it public. You can use it to train your model.\nhttps://www.kaggle.com/wuliaokaola/cldc-ds-1121-size-512", "votes": null }, { "id": "1089121", "postDate": "11/24/2020 08:07:40", "content": "

How is the work going?

", "rawMarkdown": "How is the work going?", "votes": null }, { "id": "1089224", "postDate": "11/24/2020 10:02:48", "content": "

It seems that tfrec format file is also needed in the prediction, otherwise it is not convenient to predict the score…

", "rawMarkdown": "It seems that tfrec format file is also needed in the prediction, otherwise it is not convenient to predict the score...", "votes": null }, { "id": "1089236", "postDate": "11/24/2020 10:14:57", "content": "

The problem is about label.
\nBecause the label is only in the train datasets. I think private test datasets is ok. You can use it.

", "rawMarkdown": "The problem is about label.\nBecause the label is only in the train datasets. I think private test datasets is ok. You can use it.", "votes": null }, { "id": "1089667", "postDate": "11/24/2020 17:06:11", "content": "

An update that we have pinpointed the issue and are working on a fixed version. We'll let you all know when the records in the bundle have been resolved.

", "rawMarkdown": "An update that we have pinpointed the issue and are working on a fixed version. We'll let you all know when the records in the bundle have been resolved.", "votes": null }, { "id": "1089962", "postDate": "11/25/2020 00:00:17", "content": "

Thank you.

", "rawMarkdown": "Thank you.", "votes": null }, { "id": "1090194", "postDate": "11/25/2020 06:36:06", "content": "

I have the same issue but I created my own tfrecords. Also I didn't get above 0.6 acc with imagedatagen

", "rawMarkdown": "I have the same issue but I created my own tfrecords. Also I didn't get above 0.6 acc with imagedatagen", "votes": null }, { "id": "1090975", "postDate": "11/25/2020 17:46:04", "content": "

Update: The TFRecords have been fixed and updated to the data bundle, per this post. Thanks for all of your reports!

", "rawMarkdown": "Update: The TFRecords have been fixed and updated to the data bundle, per [this post](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/199414). Thanks for all of your reports!", "votes": null }, { "id": "1091391", "postDate": "11/26/2020 01:19:08", "content": "

Thank you for your rapid response. It will help the people who use TensorFlow.

", "rawMarkdown": "Thank you for your rapid response. It will help the people who use TensorFlow.", "votes": null } ], "comments": [ { "id": 1085138, "author_name": "juliaelliott", "author_url": "", "post_date": "11/20/2020 17:42:52", "content": "

@wuliaokaola -- Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. I believe other participants are also observing this here. But we encourage you to push the envelope further, as we're sure there are advancements to be made!

", "votes": null, "replies": [ { "id": 1085461, "author_name": "awsaf49", "author_url": "", "post_date": "11/20/2020 22:22:34", "content": "

Does it also apply to test data too?

", "votes": null, "replies": [] }, { "id": 1085482, "author_name": "juliaelliott", "author_url": "", "post_date": "11/20/2020 22:51:54", "content": "

@awsaf49 Yes, the test images also have a tfrec equivalent in the same folder formatting as you see organized on the Data page.

", "votes": null, "replies": [] }, { "id": 1085486, "author_name": "awsaf49", "author_url": "", "post_date": "11/20/2020 23:08:06", "content": "

I'm wondering if the test tfrecord has the same error? Then should we avoid using tfrecord for competition submission?

", "votes": null, "replies": [] }, { "id": 1085596, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/21/2020 02:34:16", "content": "

@juliaelliott Thank you for your reply. But I still think there is something wrong with tfrecords, maybe the label. The images are come from official notebook. https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease
\nI don't think the leaves in red circle are healthy (4).

", "votes": null, "replies": [] }, { "id": 1086127, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/21/2020 12:09:42", "content": "

@juliaelliott There is no image_id in the tfrecords, I can't confirm it. But Please check these two notebooks.
\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402
\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook

", "votes": null, "replies": [ { "id": 1087622, "author_name": "richardepstein", "author_url": "", "post_date": "11/22/2020 23:05:03", "content": "

Field is called \"image_name\" instead of \"image_id\".

\n

However, I think the field has the wrong data (see other reply in this thread).

\n

-Rich

", "votes": null, "replies": [] } ] }, { "id": 1086325, "author_name": "ludovick", "author_url": "", "post_date": "11/21/2020 15:34:58", "content": "

@juliaelliott I have I think the same issue than @wuliaokaola . My submit work for the test, but as soon as the prediction is done on the hidden test set, I got an error which told me \"submission.csv not found\". I suspect there is no image_id or something goes wrong with the test tfrecord.

", "votes": null, "replies": [] } ] }, { "id": 1085717, "author_name": "aeryss", "author_url": "", "post_date": "11/21/2020 06:28:26", "content": "

+1 this problem. My TFRecords model was always at 0.6x accuracy, while using ImageDataGenerator (or TF Dataset) I can easily get 0.8x.

", "votes": null, "replies": [ { "id": 1090194, "author_name": "kingofarmy", "author_url": "", "post_date": "11/25/2020 06:36:06", "content": "

I have the same issue but I created my own tfrecords. Also I didn't get above 0.6 acc with imagedatagen

", "votes": null, "replies": [] } ] }, { "id": 1087563, "author_name": "claverru", "author_url": "", "post_date": "11/22/2020 21:16:03", "content": "

I have also noticed this. After switching from loading images to records my score dropped significantly.

", "votes": null, "replies": [ { "id": 1087620, "author_name": "richardepstein", "author_url": "", "post_date": "11/22/2020 23:03:42", "content": "

I think the TFRecords do not have the correct image names (as an aside, the field in TFRecord is image_name, not image_id).

\n

This notebook displays a TFRecord image and the corresponding jpeg:

\n

https://www.kaggle.com/richardepstein/check-tfrecords

\n

Organizers - please take a look.

\n

-Rich

", "votes": null, "replies": [] } ] }, { "id": 1088590, "author_name": "juliaelliott", "author_url": "", "post_date": "11/23/2020 19:16:42", "content": "

Thank you for your reports. We are investigating!

", "votes": null, "replies": [ { "id": 1089121, "author_name": "dynmiwang", "author_url": "", "post_date": "11/24/2020 08:07:40", "content": "

How is the work going?

", "votes": null, "replies": [] } ] }, { "id": 1088915, "author_name": "zhangeng", "author_url": "", "post_date": "11/24/2020 04:06:13", "content": "

So far, the tfrec format data set has not changed, so it is recommended to use JPG format data set at present. Because the original tfrec format data set has errors, it is difficult for training and prediction to reach the expected results, resulting in large errors. I doubt that the image and label can not be well one-to-one correspondence? Or?

", "votes": null, "replies": [ { "id": 1088917, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/24/2020 04:10:08", "content": "

I created a datasets and made it public. You can use it to train your model.
\nhttps://www.kaggle.com/wuliaokaola/cldc-ds-1121-size-512

", "votes": null, "replies": [] }, { "id": 1089224, "author_name": "zhangeng", "author_url": "", "post_date": "11/24/2020 10:02:48", "content": "

It seems that tfrec format file is also needed in the prediction, otherwise it is not convenient to predict the score…

", "votes": null, "replies": [] }, { "id": 1089236, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/24/2020 10:14:57", "content": "

The problem is about label.
\nBecause the label is only in the train datasets. I think private test datasets is ok. You can use it.

", "votes": null, "replies": [] } ] }, { "id": 1089667, "author_name": "juliaelliott", "author_url": "", "post_date": "11/24/2020 17:06:11", "content": "

An update that we have pinpointed the issue and are working on a fixed version. We'll let you all know when the records in the bundle have been resolved.

", "votes": null, "replies": [ { "id": 1089962, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/25/2020 00:00:17", "content": "

Thank you.

", "votes": null, "replies": [] } ] }, { "id": 1090975, "author_name": "juliaelliott", "author_url": "", "post_date": "11/25/2020 17:46:04", "content": "

Update: The TFRecords have been fixed and updated to the data bundle, per this post. Thanks for all of your reports!

", "votes": null, "replies": [ { "id": 1091391, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/26/2020 01:19:08", "content": "

Thank you for your rapid response. It will help the people who use TensorFlow.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1084909": "I think there are something wrong with official tfrecords files. By using tfrecords, CV stops at 0.61xxx. But by using jpeg file with the same model, I can easy get 0.83xx. @juliaelliott Could you check the official tfrecords. Thanks.\nFor detail:\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook\n\nThe leaves in the red circle look like healthy (4)?\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2006644%2F2b8d371c0086ba44215c31e130df20d8%2F__results___30_0.png?generation=1605925654430111&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2006644%2F0dcb877a341a067be8ffbb61426c712b%2F__results___33_0.png?generation=1605925837212453&alt=media)\n\nUpdate (2020/11/26): The organizer has fixed the tfrecords.", "1085138": "wuliaokaola -- Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. I believe other participants are also [observing this here](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198190). But we encourage you to push the envelope further, as we're sure there are advancements to be made!", "1085461": "Does it also apply to test data too?", "1085482": "awsaf49 Yes, the test images also have a tfrec equivalent in the same folder formatting as you see organized on the Data page.", "1085486": "I'm wondering if the test tfrecord has the same error? Then should we avoid using tfrecord for competition submission?", "1085596": "juliaelliott Thank you for your reply. But I still think there is something wrong with tfrecords, maybe the label. The images are come from official notebook. https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease\nI don't think the leaves in red circle are healthy (4).", "1085717": "1 this problem. My TFRecords model was always at 0.6x accuracy, while using ImageDataGenerator (or TF Dataset) I can easily get 0.8x.", "1086127": "juliaelliott There is no image_id in the tfrecords, I can't confirm it. But Please check these two notebooks.\nhttps://www.kaggle.com/wuliaokaola/something-wrong-with-official-tfrecords?scriptVersionId=47386402\nhttps://www.kaggle.com/wuliaokaola/getting-started-tpus-new-tfrecords/notebook", "1086325": "juliaelliott I have I think the same issue than @wuliaokaola . My submit work for the test, but as soon as the prediction is done on the hidden test set, I got an error which told me \"submission.csv not found\". I suspect there is no image_id or something goes wrong with the test tfrecord.", "1087563": "I have also noticed this. After switching from loading images to records my score dropped significantly.", "1087620": "I think the TFRecords do not have the correct image names (as an aside, the field in TFRecord is image_name, not image_id).\n\nThis notebook displays a TFRecord image and the corresponding jpeg:\n\nhttps://www.kaggle.com/richardepstein/check-tfrecords\n\nOrganizers - please take a look.\n\n-Rich", "1087622": "Field is called \"image_name\" instead of \"image_id\".\n\nHowever, I think the field has the wrong data (see other reply in this thread).\n\n-Rich", "1088590": "Thank you for your reports. We are investigating!", "1088915": "So far, the tfrec format data set has not changed, so it is recommended to use JPG format data set at present. Because the original tfrec format data set has errors, it is difficult for training and prediction to reach the expected results, resulting in large errors. I doubt that the image and label can not be well one-to-one correspondence? Or?", "1088917": "I created a datasets and made it public. You can use it to train your model.\nhttps://www.kaggle.com/wuliaokaola/cldc-ds-1121-size-512", "1089121": "How is the work going?", "1089224": "It seems that tfrec format file is also needed in the prediction, otherwise it is not convenient to predict the score...", "1089236": "The problem is about label.\nBecause the label is only in the train datasets. I think private test datasets is ok. You can use it.", "1089667": "An update that we have pinpointed the issue and are working on a fixed version. We'll let you all know when the records in the bundle have been resolved.", "1089962": "Thank you.", "1090194": "I have the same issue but I created my own tfrecords. Also I didn't get above 0.6 acc with imagedatagen", "1090975": "Update: The TFRecords have been fixed and updated to the data bundle, per [this post](https://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/199414). Thanks for all of your reports!", "1091391": "Thank you for your rapid response. It will help the people who use TensorFlow." }, "source": "meta" }