Since there are always questions about hardware in these types of competitions, I will share my most economical model: relatively good performance at an affordable price.
\nProbably the best candidate on my side is the B3 model. I used the instance g4dn.12xlarge, and the model trained for about 48 hours (10 epochs on 1.6m images). The instance cost is 3.9 USD / hour, which yields a total training cost of 187 USD. The private score of this model is 0.357, which would be 24th place on the LB.
\nInstance specification and pricing:
\nhttps://aws.amazon.com/ec2/instance-types/g4/
Since there are always questions about hardware in these types of competitions, I will share my most economical model: relatively good performance at an affordable price.
\nProbably the best candidate on my side is the B3 model. I used the instance g4dn.12xlarge, and the model trained for about 48 hours (10 epochs on 1.6m images). The instance cost is 3.9 USD / hour, which yields a total training cost of 187 USD. The private score of this model is 0.357, which would be 24th place on the LB.
\nInstance specification and pricing:
\nhttps://aws.amazon.com/ec2/instance-types/g4/
Awesome to know @narsil I was just looking at that instance earlier.
", "rawMarkdown": "Awesome to know @narsil I was just looking at that instance earlier.", "votes": null }, { "id": "1531616", "postDate": "10/02/2021 07:26:46", "content": "To be fair, it is not so much about length of training the final model, but hardware requirements to get to that final model, i.e. how many experiments did you need, run, etc.
", "rawMarkdown": "To be fair, it is not so much about length of training the final model, but hardware requirements to get to that final model, i.e. how many experiments did you need, run, etc.", "votes": null }, { "id": "1531631", "postDate": "10/02/2021 07:51:07", "content": "Well, I wrote more about training costs (rather than training time) which capture the hardware requirements. And it indeed only measures the final model, not the road to achieve it. But it is still an interesting efficiency KPI to compare between models. E.g. when evaluating models for production which would have to be retrained frequently (typical working setting I operate in on daily basis, albeit not for CV models, but for more for events and time-series) this would exactly be the measure that I would monitor
", "rawMarkdown": "Well, I wrote more about training costs (rather than training time) which capture the hardware requirements. And it indeed only measures the final model, not the road to achieve it. But it is still an interesting efficiency KPI to compare between models. E.g. when evaluating models for production which would have to be retrained frequently (typical working setting I operate in on daily basis, albeit not for CV models, but for more for events and time-series) this would exactly be the measure that I would monitor", "votes": null }, { "id": "1532238", "postDate": "10/02/2021 18:38:39", "content": "I trained my model on kaggle tpu. So It was for free. But I just used effectnet 5. With a bigger model I am sure I would have scored higher
", "rawMarkdown": "I trained my model on kaggle tpu. So It was for free. But I just used effectnet 5. With a bigger model I am sure I would have scored higher", "votes": null }, { "id": "1532439", "postDate": "10/03/2021 01:52:06", "content": "Hi @lucamtb , may I ask what's your effnetb5 validation acc? cause somehow I can't get it work.
\nbtw, thanks for your notebook, it helps me a lot.
TPU on Kaggle is free, but on standard pricing it is 8$/hour, so we could also compare the costs of models' training on TPU.
\nhttps://cloud.google.com/tpu/pricing
", "rawMarkdown": "TPU on Kaggle is free, but on standard pricing it is 8$/hour, so we could also compare the costs of models' training on TPU.\n\nhttps://cloud.google.com/tpu/pricing", "votes": null }, { "id": "2269866", "postDate": "05/22/2023 18:54:11", "content": "perfect.
\ndo you mean bazel commitee by b3?
Awesome to know @narsil I was just looking at that instance earlier.
", "votes": null, "replies": [] }, { "id": 1531616, "author_name": "philippsinger", "author_url": "", "post_date": "10/02/2021 07:26:46", "content": "To be fair, it is not so much about length of training the final model, but hardware requirements to get to that final model, i.e. how many experiments did you need, run, etc.
", "votes": null, "replies": [ { "id": 1531631, "author_name": "narsil", "author_url": "", "post_date": "10/02/2021 07:51:07", "content": "Well, I wrote more about training costs (rather than training time) which capture the hardware requirements. And it indeed only measures the final model, not the road to achieve it. But it is still an interesting efficiency KPI to compare between models. E.g. when evaluating models for production which would have to be retrained frequently (typical working setting I operate in on daily basis, albeit not for CV models, but for more for events and time-series) this would exactly be the measure that I would monitor
", "votes": null, "replies": [] } ] }, { "id": 1532238, "author_name": "lucamtb", "author_url": "", "post_date": "10/02/2021 18:38:39", "content": "I trained my model on kaggle tpu. So It was for free. But I just used effectnet 5. With a bigger model I am sure I would have scored higher
", "votes": null, "replies": [ { "id": 1532439, "author_name": "gdoong", "author_url": "", "post_date": "10/03/2021 01:52:06", "content": "Hi @lucamtb , may I ask what's your effnetb5 validation acc? cause somehow I can't get it work.
\nbtw, thanks for your notebook, it helps me a lot.
TPU on Kaggle is free, but on standard pricing it is 8$/hour, so we could also compare the costs of models' training on TPU.
\nhttps://cloud.google.com/tpu/pricing
", "votes": null, "replies": [] } ] }, { "id": 2269866, "author_name": "batoolhosseini", "author_url": "", "post_date": "05/22/2023 18:54:11", "content": "perfect.
\ndo you mean bazel commitee by b3?