Hi everyone,
\nSince some weeks now, I am trying to catch up with the CNN baseline to improve my performance.
\nTo do so, I am trying to train a pre-trained efficientnet b4, and I am limited to a batch-size of size 16.
\nI have tried many things so far, as passing RGB, RG-NIR, RGBNIR, RGBNIR+land, RGBNIR+land + feature fusion in the classifier of the model with the environmental features vector given, training one model on fauna, one on flora and merging the top15 predictions of them together at the end…
\nBut the top30 error rate at the end doesn't go below 0.8 on average.
\nI have also tried to limit the learning to the 500 most populated species, which works quite fine and sees a real decrease in error rate through learning, but using all the species classes just don't work, whatever the training parameters I have tried.
I seriously think I am missing one idea that would make a significant progress. I don't know if it is working on the dataset (which I haven't done a lot since there are already so many patches), reduce patches size, using a RF to classify my CNN output… I just don't get why I cannot catch up with the baseline whether with the same training architecture, or more data, or a \"better\" model.
\nIf you have any hint to provide, I would be really greatful. If not, I hope the best candidate will post their code soon after the deadline so I can read it!
", "messages": [ { "id": "1794742", "postDate": "05/19/2022 05:59:26", "content": "Hi everyone,
\nSince some weeks now, I am trying to catch up with the CNN baseline to improve my performance.
\nTo do so, I am trying to train a pre-trained efficientnet b4, and I am limited to a batch-size of size 16.
\nI have tried many things so far, as passing RGB, RG-NIR, RGBNIR, RGBNIR+land, RGBNIR+land + feature fusion in the classifier of the model with the environmental features vector given, training one model on fauna, one on flora and merging the top15 predictions of them together at the end…
\nBut the top30 error rate at the end doesn't go below 0.8 on average.
\nI have also tried to limit the learning to the 500 most populated species, which works quite fine and sees a real decrease in error rate through learning, but using all the species classes just don't work, whatever the training parameters I have tried.
I seriously think I am missing one idea that would make a significant progress. I don't know if it is working on the dataset (which I haven't done a lot since there are already so many patches), reduce patches size, using a RF to classify my CNN output… I just don't get why I cannot catch up with the baseline whether with the same training architecture, or more data, or a \"better\" model.
\nIf you have any hint to provide, I would be really greatful. If not, I hope the best candidate will post their code soon after the deadline so I can read it!
", "rawMarkdown": "Hi everyone,\n\nSince some weeks now, I am trying to catch up with the CNN baseline to improve my performance.\nTo do so, I am trying to train a pre-trained efficientnet b4, and I am limited to a batch-size of size 16.\nI have tried many things so far, as passing RGB, RG-NIR, RGBNIR, RGBNIR+land, RGBNIR+land + feature fusion in the classifier of the model with the environmental features vector given, training one model on fauna, one on flora and merging the top15 predictions of them together at the end...\nBut the top30 error rate at the end doesn't go below 0.8 on average.\nI have also tried to limit the learning to the 500 most populated species, which works quite fine and sees a real decrease in error rate through learning, but using all the species classes just don't work, whatever the training parameters I have tried.\n\nI seriously think I am missing one idea that would make a significant progress. I don't know if it is working on the dataset (which I haven't done a lot since there are already so many patches), reduce patches size, using a RF to classify my CNN output... I just don't get why I cannot catch up with the baseline whether with the same training architecture, or more data, or a \"better\" model.\n\nIf you have any hint to provide, I would be really greatful. If not, I hope the best candidate will post their code soon after the deadline so I can read it!", "votes": null }, { "id": "1795044", "postDate": "05/19/2022 11:28:48", "content": "Hi Louis! Have you tried any simpler model with the same data layers you mentioned as input? I would suggest trying a pretrained resnet18 first to see if you can achieve better results with that, also you will be able to increase the batch size and train faster and smoother. I think that that will give you an improvement of you current results and you would be able to build on that :), I hope this helps you!
", "rawMarkdown": "Hi Louis! Have you tried any simpler model with the same data layers you mentioned as input? I would suggest trying a pretrained resnet18 first to see if you can achieve better results with that, also you will be able to increase the batch size and train faster and smoother. I think that that will give you an improvement of you current results and you would be able to build on that :), I hope this helps you!", "votes": null }, { "id": "1795110", "postDate": "05/19/2022 12:23:23", "content": "There is nothing fancy in the provided CNN baselines. So you are not missing a big idea if you don't manage to reproduce them. The only information not explicitly stated in the leaderboard description is the use of (very standard) data augmentation. But if your model does not converge and does not beat (or come very close to) the random forest, the issue is likely to be somewhere else.
", "rawMarkdown": "There is nothing fancy in the provided CNN baselines. So you are not missing a big idea if you don't manage to reproduce them. The only information not explicitly stated in the leaderboard description is the use of (very standard) data augmentation. But if your model does not converge and does not beat (or come very close to) the random forest, the issue is likely to be somewhere else.", "votes": null }, { "id": "1795142", "postDate": "05/19/2022 13:12:28", "content": "Thank you for your answers. Actually, the loss converges quite fast to some value, and the curve is nice looking, but the top30 error rate shortly stops to decrease as well, and does not approach the random forest indeed.
\nMaybe the efficientnet was a choice a bit too ambitious given my ressources (128Gb of RAM, RTX3060 with 12Gb VRAM), I could not perform too long trainings So I will try indeed the resnet18 which should be faster to train, you're right! Thanks again to the two of you :)
Hi Louis! Have you tried any simpler model with the same data layers you mentioned as input? I would suggest trying a pretrained resnet18 first to see if you can achieve better results with that, also you will be able to increase the batch size and train faster and smoother. I think that that will give you an improvement of you current results and you would be able to build on that :), I hope this helps you!
", "votes": null, "replies": [] }, { "id": 1795110, "author_name": "tlorieul", "author_url": "", "post_date": "05/19/2022 12:23:23", "content": "There is nothing fancy in the provided CNN baselines. So you are not missing a big idea if you don't manage to reproduce them. The only information not explicitly stated in the leaderboard description is the use of (very standard) data augmentation. But if your model does not converge and does not beat (or come very close to) the random forest, the issue is likely to be somewhere else.
", "votes": null, "replies": [] }, { "id": 1795142, "author_name": "louischirol", "author_url": "", "post_date": "05/19/2022 13:12:28", "content": "Thank you for your answers. Actually, the loss converges quite fast to some value, and the curve is nice looking, but the top30 error rate shortly stops to decrease as well, and does not approach the random forest indeed.
\nMaybe the efficientnet was a choice a bit too ambitious given my ressources (128Gb of RAM, RTX3060 with 12Gb VRAM), I could not perform too long trainings So I will try indeed the resnet18 which should be faster to train, you're right! Thanks again to the two of you :)