![\"image\"](\"https://user-images.githubusercontent.com/34497776/154219340-b4928e30-1661-499f-a8bb-793c1436c7b5.png\")
Hi to all!
\nAs I wrote in my notebook, using k-fold increases the Mean Average Precision of our model on the public dataset (https://www.kaggle.com/aikhmelnytskyy/happywhale-arcface-baseline-eff-net-kfold5-0-652).
\nFrom this public notebook https://www.kaggle.com/manojprabhaakr/effnet-b6-whale-comp, I realized that increasing IMAGE_SIZE will further improve my results.
\nHowever, both methods dramatically increase the time required to train models.
\nHow do you plan to solve these problems?
\nI tried to reduce STEPS_PER_EPOCH to teach less data in each epoch, but this trick didn't work because of the specifics of the dataset.
\nWhat other options can there be to speed up the training of models?
\nI understand that it will no longer be possible to use TPU for a large number of experiments.
\nFree colab will not help, as well as 20 hours on kaggle.
\nAnother piece of information so far I don't see a significant difference between the results showing efficientnet-b6 and efficientnet-b5.
\nTherefore, it is obviously better to use efficientnet-b5.
Hi to all!
\nAs I wrote in my notebook, using k-fold increases the Mean Average Precision of our model on the public dataset (https://www.kaggle.com/aikhmelnytskyy/happywhale-arcface-baseline-eff-net-kfold5-0-652).
\nFrom this public notebook https://www.kaggle.com/manojprabhaakr/effnet-b6-whale-comp, I realized that increasing IMAGE_SIZE will further improve my results.
\nHowever, both methods dramatically increase the time required to train models.
\nHow do you plan to solve these problems?
\nI tried to reduce STEPS_PER_EPOCH to teach less data in each epoch, but this trick didn't work because of the specifics of the dataset.
\nWhat other options can there be to speed up the training of models?
\nI understand that it will no longer be possible to use TPU for a large number of experiments.
\nFree colab will not help, as well as 20 hours on kaggle.
\nAnother piece of information so far I don't see a significant difference between the results showing efficientnet-b6 and efficientnet-b5.
\nTherefore, it is obviously better to use efficientnet-b5.
I think having a proper experimentation setup + research is important. If you see last kaggle whale comp, it was unique and hard and if I remember a team published a paper to CVPR(Divide and conquer embedding space). So my goal is to learn and do a lot of research and ofcourse experiment a lot. Plus this data is long tailed, you may take a look at few shot strategies like meta learning, proto-nets, etc. All the best for the competition !
", "rawMarkdown": "I think having a proper experimentation setup + research is important. If you see last kaggle whale comp, it was unique and hard and if I remember a team published a paper to CVPR(Divide and conquer embedding space). So my goal is to learn and do a lot of research and ofcourse experiment a lot. Plus this data is long tailed, you may take a look at few shot strategies like meta learning, proto-nets, etc. All the best for the competition !", "votes": null }, { "id": "1685203", "postDate": "02/11/2022 05:31:27", "content": "Thank you for the information from the previous competition. When I created this discussion, I realized that it was a competition of increased complexity. And the difficulty for me personally is the limited resources. It's neither good nor bad, it's just the specifics of the competition. By the way, it is funny that all such competitions are united by one fact: a relatively small number of participants:-)
", "rawMarkdown": "Thank you for the information from the previous competition. When I created this discussion, I realized that it was a competition of increased complexity. And the difficulty for me personally is the limited resources. It's neither good nor bad, it's just the specifics of the competition. By the way, it is funny that all such competitions are united by one fact: a relatively small number of participants:-)", "votes": null }, { "id": "1686016", "postDate": "02/11/2022 17:35:13", "content": "if you want to speed up the training additional hardware is must, kaggle and colab pro you can only push so much. The dataset has quite high res image which can help the model. Right now ArcFace is the standard approach but I am pretty sure there will lot of amazing approaches down the line. You look into Face-ReID architectures which is essentially task at hand. Happy Researching !
", "rawMarkdown": "if you want to speed up the training additional hardware is must, kaggle and colab pro you can only push so much. The dataset has quite high res image which can help the model. Right now ArcFace is the standard approach but I am pretty sure there will lot of amazing approaches down the line. You look into Face-ReID architectures which is essentially task at hand. Happy Researching !", "votes": null }, { "id": "1687660", "postDate": "02/13/2022 03:25:01", "content": "I believe GCP grants TPU access under their \"TFRC\" program, you should check it out to request access :)
", "rawMarkdown": "I believe GCP grants TPU access under their \"TFRC\" program, you should check it out to request access :)", "votes": null }, { "id": "1688339", "postDate": "02/13/2022 15:27:26", "content": "\n\nHow do you plan to solve these problems?
\n
Create 2 accounts on Kaggle and run 2 folds, then create 3 Google Accounts and run 3 folds on Google Colab 😄. (just kidding). These problems are usually depended on your GPU.
\nGood luck, @aikhmelnytskyy!
", "rawMarkdown": "> How do you plan to solve these problems?\n\n\n\nCreate 2 accounts on Kaggle and run 2 folds, then create 3 Google Accounts and run 3 folds on Google Colab 😄. (just kidding). These problems are usually depended on your GPU.\n\nGood luck, @aikhmelnytskyy!", "votes": null }, { "id": "1688375", "postDate": "02/13/2022 15:42:02", "content": "😄😄😄I was interested in \"legal\" decisions. But for the time being, I can see that brute force (increasing the size of images and the complexity of networks) has the best effect. Good luck
", "rawMarkdown": "😄😄😄I was interested in \"legal\" decisions. But for the time being, I can see that brute force (increasing the size of images and the complexity of networks) has the best effect. Good luck", "votes": null }, { "id": "1692775", "postDate": "02/16/2022 08:16:14", "content": "I recommend Progressive Learning. This speeds up the training time and slightly improves the accuracy.
\nThis technique is introduced in efficientnetv2.
\nIn Progressive Learning, the image size is first small and the augmentation is weak. Then, as the training progresses, the image size is increased and the augmentation is stronger.
\n
\n\nThe figure is cited from a paper.
\n
The result is below.
\n![\"image\"](\"https://user-images.githubusercontent.com/34497776/154220218-1f3d7da8-af72-4be1-b052-ca1b00ba5a67.png\")
To add, it's a war-tested technique-It was really effective in the Tensorflow competition too that ended yesterday!
\nI had created a summary here and here's the 37th Pos Writeup that mentions about it
", "rawMarkdown": "To add, it's a war-tested technique-It was really effective in the Tensorflow competition too that ended yesterday! \n\nI had created a summary [here](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/307831) and here's the [37th Pos Writeup](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/307669) that mentions about it", "votes": null }, { "id": "1696048", "postDate": "02/18/2022 14:58:16", "content": "This is very interesting. I feel that the training difficulty can be gradually increased from the target scale and environmental complexity, just like warming up
", "rawMarkdown": "This is very interesting. I feel that the training difficulty can be gradually increased from the target scale and environmental complexity, just like warming up", "votes": null } ], "comments": [ { "id": 1685167, "author_name": "atharvap329", "author_url": "", "post_date": "02/11/2022 04:40:45", "content": "I think having a proper experimentation setup + research is important. If you see last kaggle whale comp, it was unique and hard and if I remember a team published a paper to CVPR(Divide and conquer embedding space). So my goal is to learn and do a lot of research and ofcourse experiment a lot. Plus this data is long tailed, you may take a look at few shot strategies like meta learning, proto-nets, etc. All the best for the competition !
", "votes": null, "replies": [ { "id": 1685203, "author_name": "aikhmelnytskyy", "author_url": "", "post_date": "02/11/2022 05:31:27", "content": "Thank you for the information from the previous competition. When I created this discussion, I realized that it was a competition of increased complexity. And the difficulty for me personally is the limited resources. It's neither good nor bad, it's just the specifics of the competition. By the way, it is funny that all such competitions are united by one fact: a relatively small number of participants:-)
", "votes": null, "replies": [] }, { "id": 1686016, "author_name": "atharvap329", "author_url": "", "post_date": "02/11/2022 17:35:13", "content": "if you want to speed up the training additional hardware is must, kaggle and colab pro you can only push so much. The dataset has quite high res image which can help the model. Right now ArcFace is the standard approach but I am pretty sure there will lot of amazing approaches down the line. You look into Face-ReID architectures which is essentially task at hand. Happy Researching !
", "votes": null, "replies": [] } ] }, { "id": 1687660, "author_name": "init27", "author_url": "", "post_date": "02/13/2022 03:25:01", "content": "I believe GCP grants TPU access under their \"TFRC\" program, you should check it out to request access :)
", "votes": null, "replies": [] }, { "id": 1688339, "author_name": "vad13irt", "author_url": "", "post_date": "02/13/2022 15:27:26", "content": "\n\nHow do you plan to solve these problems?
\n
Create 2 accounts on Kaggle and run 2 folds, then create 3 Google Accounts and run 3 folds on Google Colab 😄. (just kidding). These problems are usually depended on your GPU.
\nGood luck, @aikhmelnytskyy!
", "votes": null, "replies": [ { "id": 1688375, "author_name": "aikhmelnytskyy", "author_url": "", "post_date": "02/13/2022 15:42:02", "content": "😄😄😄I was interested in \"legal\" decisions. But for the time being, I can see that brute force (increasing the size of images and the complexity of networks) has the best effect. Good luck
", "votes": null, "replies": [] } ] }, { "id": 1692775, "author_name": "shinmurashinmura", "author_url": "", "post_date": "02/16/2022 08:16:14", "content": "I recommend Progressive Learning. This speeds up the training time and slightly improves the accuracy.
\nThis technique is introduced in efficientnetv2.
\nIn Progressive Learning, the image size is first small and the augmentation is weak. Then, as the training progresses, the image size is increased and the augmentation is stronger.
\n
\n\nThe figure is cited from a paper.
\n
The result is below.
\n
To add, it's a war-tested technique-It was really effective in the Tensorflow competition too that ended yesterday!
\nI had created a summary here and here's the 37th Pos Writeup that mentions about it
", "votes": null, "replies": [] }, { "id": 1696048, "author_name": "biglafe", "author_url": "", "post_date": "02/18/2022 14:58:16", "content": "This is very interesting. I feel that the training difficulty can be gradually increased from the target scale and environmental complexity, just like warming up
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1684927": "Hi to all! \nAs I wrote in my notebook, using k-fold increases the Mean Average Precision of our model on the public dataset (https://www.kaggle.com/aikhmelnytskyy/happywhale-arcface-baseline-eff-net-kfold5-0-652).\nFrom this public notebook https://www.kaggle.com/manojprabhaakr/effnet-b6-whale-comp, I realized that increasing IMAGE_SIZE will further improve my results. \nHowever, both methods dramatically increase the time required to train models. \nHow do you plan to solve these problems?\nI tried to reduce STEPS_PER_EPOCH to teach less data in each epoch, but this trick didn't work because of the specifics of the dataset.\nWhat other options can there be to speed up the training of models? \nI understand that it will no longer be possible to use TPU for a large number of experiments.\nFree colab will not help, as well as 20 hours on kaggle.\nAnother piece of information so far I don't see a significant difference between the results showing efficientnet-b6 and efficientnet-b5. \nTherefore, it is obviously better to use efficientnet-b5.", "1685167": "I think having a proper experimentation setup + research is important. If you see last kaggle whale comp, it was unique and hard and if I remember a team published a paper to CVPR(Divide and conquer embedding space). So my goal is to learn and do a lot of research and ofcourse experiment a lot. Plus this data is long tailed, you may take a look at few shot strategies like meta learning, proto-nets, etc. All the best for the competition !", "1685203": "Thank you for the information from the previous competition. When I created this discussion, I realized that it was a competition of increased complexity. And the difficulty for me personally is the limited resources. It's neither good nor bad, it's just the specifics of the competition. By the way, it is funny that all such competitions are united by one fact: a relatively small number of participants:-)", "1686016": "if you want to speed up the training additional hardware is must, kaggle and colab pro you can only push so much. The dataset has quite high res image which can help the model. Right now ArcFace is the standard approach but I am pretty sure there will lot of amazing approaches down the line. You look into Face-ReID architectures which is essentially task at hand. Happy Researching !", "1687660": "I believe GCP grants TPU access under their \"TFRC\" program, you should check it out to request access :)", "1688339": "> How do you plan to solve these problems?\n\n\n\nCreate 2 accounts on Kaggle and run 2 folds, then create 3 Google Accounts and run 3 folds on Google Colab 😄. (just kidding). These problems are usually depended on your GPU.\n\nGood luck, @aikhmelnytskyy!", "1688375": "😄😄😄I was interested in \"legal\" decisions. But for the time being, I can see that brute force (increasing the size of images and the complexity of networks) has the best effect. Good luck", "1692775": "I recommend Progressive Learning. This speeds up the training time and slightly improves the accuracy.\n\nThis technique is introduced in [efficientnetv2](https://arxiv.org/abs/2104.00298).\n\nIn Progressive Learning, the image size is first **small** and the augmentation is **weak**. Then, as the training progresses, the image size is **increased** and the augmentation is **stronger**.\n\n\n> The figure is cited from a paper.\n\nThe result is below.\n\n", "1692796": "To add, it's a war-tested technique-It was really effective in the Tensorflow competition too that ended yesterday! \n\nI had created a summary [here](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/307831) and here's the [37th Pos Writeup](https://www.kaggle.com/c/tensorflow-great-barrier-reef/discussion/307669) that mentions about it", "1696048": "This is very interesting. I feel that the training difficulty can be gradually increased from the target scale and environmental complexity, just like warming up" }, "source": "meta" }