{ "id": 198190, "title": "Organizer's Baseline for TPU training", "url": "/competitions/cassava-leaf-disease-classification/discussion/198190", "author_name": "", "post_date": "2020-11-20T07:11:22.286201700Z", "votes": 5, "comment_count": 10, "views": 0, "content": "

Firstly, I really appreciate the clean pipeline provided for TPU training from Kaggle's team's notebook.

\n

However, I noticed that the val_sparse_categorical_accuracy stays at 0.6079 after the 3rd epoch, could this be a case of the optimizer hitting a local minima very early?

", "messages": [ { "id": "1084577", "postDate": "11/20/2020 07:11:22", "content": "

Firstly, I really appreciate the clean pipeline provided for TPU training from Kaggle's team's notebook.

\n

However, I noticed that the val_sparse_categorical_accuracy stays at 0.6079 after the 3rd epoch, could this be a case of the optimizer hitting a local minima very early?

", "rawMarkdown": "Firstly, I really appreciate the clean pipeline provided for TPU training from Kaggle's team's [notebook](https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease).\n\nHowever, I noticed that the `val_sparse_categorical_accuracy` stays at 0.6079 after the 3rd epoch, could this be a case of the optimizer hitting a local minima very early?", "votes": null }, { "id": "1084764", "postDate": "11/20/2020 11:10:57", "content": "

Yes, optimizer is hitting the local minima very early. One of the possible reasons could be imbalance in data ?
\nAlso, I don't understand the submission file for this competition. We only have one image id to predict for the public lb ?

", "rawMarkdown": "Yes, optimizer is hitting the local minima very early. One of the possible reasons could be imbalance in data ?\nAlso, I don't understand the submission file for this competition. We only have one image id to predict for the public lb ?", "votes": null }, { "id": "1084787", "postDate": "11/20/2020 11:38:48", "content": "

@thakurudit don’t worry about the one test image. It’s just there for you to test out your prediction. The real test set is hidden and it will be run when you inference

", "rawMarkdown": "thakurudit don’t worry about the one test image. It’s just there for you to test out your prediction. The real test set is hidden and it will be run when you inference", "votes": null }, { "id": "1085104", "postDate": "11/20/2020 17:07:06", "content": "

Not sure if it's related to local minima, tried with different optimizers and different lr/batch sizes, with different models. My model also on tpu, could it be due to tfrecords? I noticed people doing better by using jpegs, didn't try that approach but idk. Haven't got much time to test that out…

", "rawMarkdown": "Not sure if it's related to local minima, tried with different optimizers and different lr/batch sizes, with different models. My model also on tpu, could it be due to tfrecords? I noticed people doing better by using jpegs, didn't try that approach but idk. Haven't got much time to test that out...", "votes": null }, { "id": "1085139", "postDate": "11/20/2020 17:43:24", "content": "

Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. But we encourage you to push the envelope further, as we're sure there are advancements to be made!

", "rawMarkdown": "Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. But we encourage you to push the envelope further, as we're sure there are advancements to be made!", "votes": null }, { "id": "1085227", "postDate": "11/20/2020 19:04:21", "content": "

I just used custom 512x512 tfrecords and got much better results with same model. Just for your information.

", "rawMarkdown": "I just used custom 512x512 tfrecords and got much better results with same model. Just for your information.", "votes": null }, { "id": "1085536", "postDate": "11/21/2020 00:42:34", "content": "

quite strange, but I got the same results than the starter notebook even with using different architecture/data aug etc. I wonder if there is something wrong with the tfrecord. Am I the only one with this weird behavior?

", "rawMarkdown": "quite strange, but I got the same results than the starter notebook even with using different architecture/data aug etc. I wonder if there is something wrong with the tfrecord. Am I the only one with this weird behavior?", "votes": null }, { "id": "1085568", "postDate": "11/21/2020 01:32:50", "content": "

@juliaelliott Many thanks for clarifying this! Now I know the workaround for it :)

", "rawMarkdown": "juliaelliott Many thanks for clarifying this! Now I know the workaround for it :)", "votes": null }, { "id": "1085569", "postDate": "11/21/2020 01:33:44", "content": "

@datafan07 Thanks a lot for the info, btw do you create the custom tfrecords using the 512 x 512 jpegs?

", "rawMarkdown": "datafan07 Thanks a lot for the info, btw do you create the custom tfrecords using the 512 x 512 jpegs?", "votes": null }, { "id": "1085570", "postDate": "11/21/2020 01:34:08", "content": "

Your hunch seems to be correct!

", "rawMarkdown": "Your hunch seems to be correct!", "votes": null }, { "id": "1085608", "postDate": "11/21/2020 02:45:26", "content": "

I think the label is mismatched.
\nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198272#1085596

", "rawMarkdown": "I think the label is mismatched. \nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198272#1085596", "votes": null } ], "comments": [ { "id": 1084764, "author_name": "thakurudit", "author_url": "", "post_date": "11/20/2020 11:10:57", "content": "

Yes, optimizer is hitting the local minima very early. One of the possible reasons could be imbalance in data ?
\nAlso, I don't understand the submission file for this competition. We only have one image id to predict for the public lb ?

", "votes": null, "replies": [ { "id": 1084787, "author_name": "reighns", "author_url": "", "post_date": "11/20/2020 11:38:48", "content": "

@thakurudit don’t worry about the one test image. It’s just there for you to test out your prediction. The real test set is hidden and it will be run when you inference

", "votes": null, "replies": [] } ] }, { "id": 1085104, "author_name": "datafan07", "author_url": "", "post_date": "11/20/2020 17:07:06", "content": "

Not sure if it's related to local minima, tried with different optimizers and different lr/batch sizes, with different models. My model also on tpu, could it be due to tfrecords? I noticed people doing better by using jpegs, didn't try that approach but idk. Haven't got much time to test that out…

", "votes": null, "replies": [ { "id": 1085570, "author_name": "reighns", "author_url": "", "post_date": "11/21/2020 01:34:08", "content": "

Your hunch seems to be correct!

", "votes": null, "replies": [] } ] }, { "id": 1085139, "author_name": "juliaelliott", "author_url": "", "post_date": "11/20/2020 17:43:24", "content": "

Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. But we encourage you to push the envelope further, as we're sure there are advancements to be made!

", "votes": null, "replies": [ { "id": 1085568, "author_name": "reighns", "author_url": "", "post_date": "11/21/2020 01:32:50", "content": "

@juliaelliott Many thanks for clarifying this! Now I know the workaround for it :)

", "votes": null, "replies": [] } ] }, { "id": 1085227, "author_name": "datafan07", "author_url": "", "post_date": "11/20/2020 19:04:21", "content": "

I just used custom 512x512 tfrecords and got much better results with same model. Just for your information.

", "votes": null, "replies": [ { "id": 1085569, "author_name": "reighns", "author_url": "", "post_date": "11/21/2020 01:33:44", "content": "

@datafan07 Thanks a lot for the info, btw do you create the custom tfrecords using the 512 x 512 jpegs?

", "votes": null, "replies": [] } ] }, { "id": 1085536, "author_name": "ludovick", "author_url": "", "post_date": "11/21/2020 00:42:34", "content": "

quite strange, but I got the same results than the starter notebook even with using different architecture/data aug etc. I wonder if there is something wrong with the tfrecord. Am I the only one with this weird behavior?

", "votes": null, "replies": [] }, { "id": 1085608, "author_name": "wuliaokaola", "author_url": "", "post_date": "11/21/2020 02:45:26", "content": "

I think the label is mismatched.
\nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198272#1085596

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1084577": "Firstly, I really appreciate the clean pipeline provided for TPU training from Kaggle's team's [notebook](https://www.kaggle.com/jessemostipak/getting-started-tpus-cassava-leaf-disease).\n\nHowever, I noticed that the `val_sparse_categorical_accuracy` stays at 0.6079 after the 3rd epoch, could this be a case of the optimizer hitting a local minima very early?", "1084764": "Yes, optimizer is hitting the local minima very early. One of the possible reasons could be imbalance in data ?\nAlso, I don't understand the submission file for this competition. We only have one image id to predict for the public lb ?", "1084787": "thakurudit don’t worry about the one test image. It’s just there for you to test out your prediction. The real test set is hidden and it will be run when you inference", "1085104": "Not sure if it's related to local minima, tried with different optimizers and different lr/batch sizes, with different models. My model also on tpu, could it be due to tfrecords? I noticed people doing better by using jpegs, didn't try that approach but idk. Haven't got much time to test that out...", "1085139": "Yes, the TPU-enabled notebook with tfrecords provided was designed to be a starter which hits its local optima quite early but gives everyone somewhere to begin. But we encourage you to push the envelope further, as we're sure there are advancements to be made!", "1085227": "I just used custom 512x512 tfrecords and got much better results with same model. Just for your information.", "1085536": "quite strange, but I got the same results than the starter notebook even with using different architecture/data aug etc. I wonder if there is something wrong with the tfrecord. Am I the only one with this weird behavior?", "1085568": "juliaelliott Many thanks for clarifying this! Now I know the workaround for it :)", "1085569": "datafan07 Thanks a lot for the info, btw do you create the custom tfrecords using the 512 x 512 jpegs?", "1085570": "Your hunch seems to be correct!", "1085608": "I think the label is mismatched. \nhttps://www.kaggle.com/c/cassava-leaf-disease-classification/discussion/198272#1085596" }, "source": "meta" }