I started this competitions training custom models (darknet, wideresnets, etc..) with little success, around ~0.45LB. I had limited computing power and just wanted to test this fast architectures.\nWe teamed up with David and got the extra data from the HPA v18. (99k images in total where used). We ended using just simple, pretrained Resnets.
\n\nI used mostly GCP with P4 GPUs and some P100 at the end. It was the first time I used GCP, I was mostly an AWS user, but discovered that we get $300 free credits, so I decided to try. The only thing I would say is that the P4 cards are not very fast, roughly equivalent to a GTX 1070, but you can use up to 4 of them, and they are cheap $0.24/hour. The SSD in the VM is not very fast, and was expensive ($1/day) for 200GB) for a hobbyst like me).\nThe RTX cards available at vast.ai are very fast for fp16 training, I found that the RTX2080 (not TI) was the best value for the money, almost 3x faster than the P4.\nI would love google added some value cards, like Titan's or even RTX 2080's...\n- Total personal Cost: $250 credits in GCP and $50 in vast.ai
\n\nThanks you to everyone, the forums at fastai and the Discussion here was very inspiring.
", "messages": [ { "id": 454327, "postDate": "2019-01-11T13:00:26.017Z", "content": "I started this competitions training custom models (darknet, wideresnets, etc..) with little success, around ~0.45LB. I had limited computing power and just wanted to test this fast architectures.\nWe teamed up with David and got the extra data from the HPA v18. (99k images in total where used). We ended using just simple, pretrained Resnets.
\n\nI used mostly GCP with P4 GPUs and some P100 at the end. It was the first time I used GCP, I was mostly an AWS user, but discovered that we get $300 free credits, so I decided to try. The only thing I would say is that the P4 cards are not very fast, roughly equivalent to a GTX 1070, but you can use up to 4 of them, and they are cheap $0.24/hour. The SSD in the VM is not very fast, and was expensive ($1/day) for 200GB) for a hobbyst like me).\nThe RTX cards available at vast.ai are very fast for fp16 training, I found that the RTX2080 (not TI) was the best value for the money, almost 3x faster than the P4.\nI would love google added some value cards, like Titan's or even RTX 2080's...\n- Total personal Cost: $250 credits in GCP and $50 in vast.ai
\n\nThanks you to everyone, the forums at fastai and the Discussion here was very inspiring.
", "rawMarkdown": "# Fastai V1 solution\nI started this competitions training custom models (darknet, wideresnets, etc..) with little success, around ~0.45LB. I had limited computing power and just wanted to test this fast architectures.\nWe teamed up with David and got the extra data from the HPA v18. (99k images in total where used). We ended using just simple, pretrained Resnets.\n\n- RGB worked better for us, only png images where used.\n- Oversampling was done with an Imbalanced sampler, with weights for low classes (a simple pytorch sampler that computer the distribution of classes, and assigns a value to each sample, we used 1/samples and log sample).\n- Very late in the competition (1 weeks ago) we realised that simple Resnet18, 34 and 50 worked better, and thanks to fastai one_cycle we could train them very fast. (a couple of hours).\n- Threshold optim did not worked for us.\n- I was a sad that without external data it was impossible to get good results.\n- Vast.ai is cheap and works very good, I have a bash script that install, copies and setups everything in a couple of minutes, RTX cards are very fast.\n- Mixup and fp16 was used all the time.\n- We should have chosen better models to submit, I forgot to click the better ones.\n- The model f1 score was almost identical to the LB score.\n- We used BCE, focal Loss and soft F1 loss.\n- Only single fold models, did not have time to do CV.\n- 90% train, 10% test was used.\n- fastai library changed a lot during the competition, we started with V1.06 and ended in V1.39 (using the datablock API). It was a personal choice to use V1 and not V0.7, thanks @sgugger\n- The highest Private Leaderboard model was a mix of Resnet18, Resnet34 and Resnet50 (0.509)\n\n# Hardware\nI used mostly GCP with P4 GPUs and some P100 at the end. It was the first time I used GCP, I was mostly an AWS user, but discovered that we get $300 free credits, so I decided to try. The only thing I would say is that the P4 cards are not very fast, roughly equivalent to a GTX 1070, but you can use up to 4 of them, and they are cheap $0.24/hour. The SSD in the VM is not very fast, and was expensive ($1/day) for 200GB) for a hobbyst like me).\nThe RTX cards available at vast.ai are very fast for fp16 training, I found that the RTX2080 (not TI) was the best value for the money, almost 3x faster than the P4.\nI would love google added some value cards, like Titan's or even RTX 2080's...\n- Total personal Cost: $250 credits in GCP and $50 in vast.ai\n\n\nThanks you to everyone, the forums at fastai and the Discussion here was very inspiring.\n", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "454327": "# Fastai V1 solution\nI started this competitions training custom models (darknet, wideresnets, etc..) with little success, around ~0.45LB. I had limited computing power and just wanted to test this fast architectures.\nWe teamed up with David and got the extra data from the HPA v18. (99k images in total where used). We ended using just simple, pretrained Resnets.\n\n- RGB worked better for us, only png images where used.\n- Oversampling was done with an Imbalanced sampler, with weights for low classes (a simple pytorch sampler that computer the distribution of classes, and assigns a value to each sample, we used 1/samples and log sample).\n- Very late in the competition (1 weeks ago) we realised that simple Resnet18, 34 and 50 worked better, and thanks to fastai one_cycle we could train them very fast. (a couple of hours).\n- Threshold optim did not worked for us.\n- I was a sad that without external data it was impossible to get good results.\n- Vast.ai is cheap and works very good, I have a bash script that install, copies and setups everything in a couple of minutes, RTX cards are very fast.\n- Mixup and fp16 was used all the time.\n- We should have chosen better models to submit, I forgot to click the better ones.\n- The model f1 score was almost identical to the LB score.\n- We used BCE, focal Loss and soft F1 loss.\n- Only single fold models, did not have time to do CV.\n- 90% train, 10% test was used.\n- fastai library changed a lot during the competition, we started with V1.06 and ended in V1.39 (using the datablock API). It was a personal choice to use V1 and not V0.7, thanks @sgugger\n- The highest Private Leaderboard model was a mix of Resnet18, Resnet34 and Resnet50 (0.509)\n\n# Hardware\nI used mostly GCP with P4 GPUs and some P100 at the end. It was the first time I used GCP, I was mostly an AWS user, but discovered that we get $300 free credits, so I decided to try. The only thing I would say is that the P4 cards are not very fast, roughly equivalent to a GTX 1070, but you can use up to 4 of them, and they are cheap $0.24/hour. The SSD in the VM is not very fast, and was expensive ($1/day) for 200GB) for a hobbyst like me).\nThe RTX cards available at vast.ai are very fast for fp16 training, I found that the RTX2080 (not TI) was the best value for the money, almost 3x faster than the P4.\nI would love google added some value cards, like Titan's or even RTX 2080's...\n- Total personal Cost: $250 credits in GCP and $50 in vast.ai\n\n\nThanks you to everyone, the forums at fastai and the Discussion here was very inspiring.\n" } }