Hello everyone! I thought it might be good to have a discussion space for sharing to help improve all of our ensembles by good architectures or good ensembles in general. I'm barely finishing my se_resnext50 model and it is giving around .90 score just by itself, I was also thinking on training an inceptionv4 since I've read on several papers it works good with melanomas. Any net or ensemble is welcomed!\nAlso if you have a particular ensemble technique that works well please also share, since I was planning to do a regular average ensemble!
", "messages": [ { "id": "912764", "postDate": "07/02/2020 17:55:35", "content": "Hello everyone! I thought it might be good to have a discussion space for sharing to help improve all of our ensembles by good architectures or good ensembles in general. I'm barely finishing my se_resnext50 model and it is giving around .90 score just by itself, I was also thinking on training an inceptionv4 since I've read on several papers it works good with melanomas. Any net or ensemble is welcomed!\nAlso if you have a particular ensemble technique that works well please also share, since I was planning to do a regular average ensemble!
", "rawMarkdown": "Hello everyone! I thought it might be good to have a discussion space for sharing to help improve all of our ensembles by good architectures or good ensembles in general. I'm barely finishing my se_resnext50 model and it is giving around .90 score just by itself, I was also thinking on training an inceptionv4 since I've read on several papers it works good with melanomas. Any net or ensemble is welcomed!\nAlso if you have a particular ensemble technique that works well please also share, since I was planning to do a regular average ensemble!", "votes": null }, { "id": "913774", "postDate": "07/03/2020 12:14:16", "content": "Now I'm working with an ensemble of 4 classifiers: each one works with different image sizes (256, 386, 512, 768). Then I do a weighted average of the prediction from each classifier, where the weight is proportional to the individual LB score. It improves if you have in the ensemble all classifiers with \"enough good\" val_acc.
", "rawMarkdown": "Now I'm working with an ensemble of 4 classifiers: each one works with different image sizes (256, 386, 512, 768). Then I do a weighted average of the prediction from each classifier, where the weight is proportional to the individual LB score. It improves if you have in the ensemble all classifiers with \"enough good\" val_acc.", "votes": null }, { "id": "913777", "postDate": "07/03/2020 12:15:50", "content": "Just to be clear: the idea is not entirely mine. I have read somewhere a comment from C. Deotte where he explains in a nice way how with different image sizes you can get different details in the image.
", "rawMarkdown": "Just to be clear: the idea is not entirely mine. I have read somewhere a comment from C. Deotte where he explains in a nice way how with different image sizes you can get different details in the image.", "votes": null }, { "id": "914023", "postDate": "07/03/2020 15:21:14", "content": "Nice, yea I actually read that discussion thread too and I also want to implement the resizing and retraining too! But first I want to build some models with basic 224x224 and ensemble those to see which models get me the best score, and then use those models for resizing. Which model did you use for the resizing? Oh by the way, you may want to consider doing Boosting for ensembling, instead of assigning weights based on the LB you get to build a model using XGBoost with the predictions of your previous models and choose the weights of each model based on their feature importance!
", "rawMarkdown": "Nice, yea I actually read that discussion thread too and I also want to implement the resizing and retraining too! But first I want to build some models with basic 224x224 and ensemble those to see which models get me the best score, and then use those models for resizing. Which model did you use for the resizing? Oh by the way, you may want to consider doing Boosting for ensembling, instead of assigning weights based on the LB you get to build a model using XGBoost with the predictions of your previous models and choose the weights of each model based on their feature importance!", "votes": null }, { "id": "914042", "postDate": "07/03/2020 15:34:23", "content": "If you want to learn more about Boosting ensembling, I recommend you this notebook:\nhttps://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python
", "rawMarkdown": "If you want to learn more about Boosting ensembling, I recommend you this notebook:\nhttps://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python", "votes": null }, { "id": "914289", "postDate": "07/03/2020 18:19:10", "content": "One of the condition to have good ensemble is that the classifiers are independent. Therefore I trained the 4 classifiers independently, trying to get as high LB as possible. Then I ensembles. Yes, a good idea to \"train\" the super-classifier. Maybe if I get enough close to the top 3 I'll do it (joking).
", "rawMarkdown": "One of the condition to have good ensemble is that the classifiers are independent. Therefore I trained the 4 classifiers independently, trying to get as high LB as possible. Then I ensembles. Yes, a good idea to \"train\" the super-classifier. Maybe if I get enough close to the top 3 I'll do it (joking).", "votes": null }, { "id": "914627", "postDate": "07/04/2020 05:27:33", "content": "It is well documented (see numerous ImageNet papers) that using larger Image Sizes does in fact improve the model. However, it has also been shown that using a smaller image size with a more complex model can work and/or additionally adjusting the maxpool stride to account for the smaller image size.
", "rawMarkdown": "It is well documented (see numerous ImageNet papers) that using larger Image Sizes does in fact improve the model. However, it has also been shown that using a smaller image size with a more complex model can work and/or additionally adjusting the maxpool stride to account for the smaller image size.", "votes": null }, { "id": "915401", "postDate": "07/04/2020 17:45:27", "content": "Little update. Both inceptionv4 and se_resnext50 using basic augmentation (flipping and zooming) and using 5 K-Folds gave me .89 and .88 in same order in the LB. Im gonna try now EfficientNet B7 and maybe noisy student models.
", "rawMarkdown": "Little update. Both inceptionv4 and se_resnext50 using basic augmentation (flipping and zooming) and using 5 K-Folds gave me .89 and .88 in same order in the LB. Im gonna try now EfficientNet B7 and maybe noisy student models.", "votes": null } ], "comments": [ { "id": 913774, "author_name": "luigisaetta", "author_url": "", "post_date": "07/03/2020 12:14:16", "content": "Now I'm working with an ensemble of 4 classifiers: each one works with different image sizes (256, 386, 512, 768). Then I do a weighted average of the prediction from each classifier, where the weight is proportional to the individual LB score. It improves if you have in the ensemble all classifiers with \"enough good\" val_acc.
", "votes": null, "replies": [ { "id": 914023, "author_name": "guillermocampollo", "author_url": "", "post_date": "07/03/2020 15:21:14", "content": "Nice, yea I actually read that discussion thread too and I also want to implement the resizing and retraining too! But first I want to build some models with basic 224x224 and ensemble those to see which models get me the best score, and then use those models for resizing. Which model did you use for the resizing? Oh by the way, you may want to consider doing Boosting for ensembling, instead of assigning weights based on the LB you get to build a model using XGBoost with the predictions of your previous models and choose the weights of each model based on their feature importance!
", "votes": null, "replies": [] }, { "id": 914042, "author_name": "guillermocampollo", "author_url": "", "post_date": "07/03/2020 15:34:23", "content": "If you want to learn more about Boosting ensembling, I recommend you this notebook:\nhttps://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python
", "votes": null, "replies": [] }, { "id": 914289, "author_name": "luigisaetta", "author_url": "", "post_date": "07/03/2020 18:19:10", "content": "One of the condition to have good ensemble is that the classifiers are independent. Therefore I trained the 4 classifiers independently, trying to get as high LB as possible. Then I ensembles. Yes, a good idea to \"train\" the super-classifier. Maybe if I get enough close to the top 3 I'll do it (joking).
", "votes": null, "replies": [] } ] }, { "id": 913777, "author_name": "luigisaetta", "author_url": "", "post_date": "07/03/2020 12:15:50", "content": "Just to be clear: the idea is not entirely mine. I have read somewhere a comment from C. Deotte where he explains in a nice way how with different image sizes you can get different details in the image.
", "votes": null, "replies": [ { "id": 914627, "author_name": "brianfeeny", "author_url": "", "post_date": "07/04/2020 05:27:33", "content": "It is well documented (see numerous ImageNet papers) that using larger Image Sizes does in fact improve the model. However, it has also been shown that using a smaller image size with a more complex model can work and/or additionally adjusting the maxpool stride to account for the smaller image size.
", "votes": null, "replies": [] } ] }, { "id": 915401, "author_name": "guillermocampollo", "author_url": "", "post_date": "07/04/2020 17:45:27", "content": "Little update. Both inceptionv4 and se_resnext50 using basic augmentation (flipping and zooming) and using 5 K-Folds gave me .89 and .88 in same order in the LB. Im gonna try now EfficientNet B7 and maybe noisy student models.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "912764": "Hello everyone! I thought it might be good to have a discussion space for sharing to help improve all of our ensembles by good architectures or good ensembles in general. I'm barely finishing my se_resnext50 model and it is giving around .90 score just by itself, I was also thinking on training an inceptionv4 since I've read on several papers it works good with melanomas. Any net or ensemble is welcomed!\nAlso if you have a particular ensemble technique that works well please also share, since I was planning to do a regular average ensemble!", "913774": "Now I'm working with an ensemble of 4 classifiers: each one works with different image sizes (256, 386, 512, 768). Then I do a weighted average of the prediction from each classifier, where the weight is proportional to the individual LB score. It improves if you have in the ensemble all classifiers with \"enough good\" val_acc.", "913777": "Just to be clear: the idea is not entirely mine. I have read somewhere a comment from C. Deotte where he explains in a nice way how with different image sizes you can get different details in the image.", "914023": "Nice, yea I actually read that discussion thread too and I also want to implement the resizing and retraining too! But first I want to build some models with basic 224x224 and ensemble those to see which models get me the best score, and then use those models for resizing. Which model did you use for the resizing? Oh by the way, you may want to consider doing Boosting for ensembling, instead of assigning weights based on the LB you get to build a model using XGBoost with the predictions of your previous models and choose the weights of each model based on their feature importance!", "914042": "If you want to learn more about Boosting ensembling, I recommend you this notebook:\nhttps://www.kaggle.com/arthurtok/introduction-to-ensembling-stacking-in-python", "914289": "One of the condition to have good ensemble is that the classifiers are independent. Therefore I trained the 4 classifiers independently, trying to get as high LB as possible. Then I ensembles. Yes, a good idea to \"train\" the super-classifier. Maybe if I get enough close to the top 3 I'll do it (joking).", "914627": "It is well documented (see numerous ImageNet papers) that using larger Image Sizes does in fact improve the model. However, it has also been shown that using a smaller image size with a more complex model can work and/or additionally adjusting the maxpool stride to account for the smaller image size.", "915401": "Little update. Both inceptionv4 and se_resnext50 using basic augmentation (flipping and zooming) and using 5 K-Folds gave me .89 and .88 in same order in the LB. Im gonna try now EfficientNet B7 and maybe noisy student models." }, "source": "meta" }