I was wondering how would ensemble approaches are performing for this dataset. If you have experimented and don't mind sharing, then please do! :)
", "messages": [ { "id": "1321817", "postDate": "05/25/2021 02:32:01", "content": "I was wondering how would ensemble approaches are performing for this dataset. If you have experimented and don't mind sharing, then please do! :)
", "rawMarkdown": "I was wondering how would ensemble approaches are performing for this dataset. If you have experimented and don't mind sharing, then please do! :)", "votes": null }, { "id": "1322163", "postDate": "05/25/2021 08:29:28", "content": "I'm using a mean of all predictions.
", "rawMarkdown": "I'm using a mean of all predictions.", "votes": null }, { "id": "1323028", "postDate": "05/25/2021 22:13:32", "content": "I have tried two approaches:
\nSimple mean of predictions for soft outputs and then apply thresholds to produce solid labels.
The second approach I called \"Label Vote Ensemble\" for a lack of better term. The idea is to produce solid labels by each individual model and then pick final label based on how many models predicted that label. Let's say you have 5 models out of your 5-fold CV training, now you make 5 predictions with those models for an image, then for each class-label there can be up to 5 predictions - votes made by all 5 models. Use majority vote to decide on final prediction. So, if for class-label scab you get 4 votes of 5 but for other class-labels there were less than 3 votes of 5, then final prediction for the image will be scab.
I found that Label Vote Ensemble worked better than averaging based on LB scores I observed. Although, the majority vote logic was not the best one, in practice even if lower number of models predicted a certain class-label this was the true class-label for the most samples. I found that, for example, in the case of 5 models if at least 2 models predicted certain label it resulted in the highest score on public LB.
", "rawMarkdown": "I have tried two approaches:\n\n1. Simple mean of predictions for soft outputs and then apply thresholds to produce solid labels.\n\n2. The second approach I called \"Label Vote Ensemble\" for a lack of better term. The idea is to produce solid labels by each individual model and then pick final label based on how many models predicted that label. Let's say you have 5 models out of your 5-fold CV training, now you make 5 predictions with those models for an image, then for each class-label there can be up to 5 predictions - votes made by all 5 models. Use majority vote to decide on final prediction. So, if for class-label **scab** you get 4 votes of 5 but for other class-labels there were less than 3 votes of 5, then final prediction for the image will be **scab**.\n\nI found that Label Vote Ensemble worked better than averaging based on LB scores I observed. Although, the majority vote logic was not the best one, in practice even if lower number of models predicted a certain class-label this was the true class-label for the most samples. I found that, for example, in the case of 5 models if at least 2 models predicted certain label it resulted in the highest score on public LB.", "votes": null }, { "id": "1324441", "postDate": "05/27/2021 00:24:49", "content": "@datasciencegeek Let me go through this and get back to you. Thanks! :)
", "rawMarkdown": "datasciencegeek Let me go through this and get back to you. Thanks! :)", "votes": null }, { "id": "1327650", "postDate": "05/29/2021 13:50:56", "content": "So, in a nutshell, you are saying the ideal approach was to add another hyperparameter? you have the threshold for the probabilities (to compute a vote) and you have a threshold for the minimum number of votes? That is indeed interesting and I will use this approach in the next competition.
", "rawMarkdown": "So, in a nutshell, you are saying the ideal approach was to add another hyperparameter? you have the threshold for the probabilities (to compute a vote) and you have a threshold for the minimum number of votes? That is indeed interesting and I will use this approach in the next competition.", "votes": null } ], "comments": [ { "id": 1322163, "author_name": "atamazian", "author_url": "", "post_date": "05/25/2021 08:29:28", "content": "I'm using a mean of all predictions.
", "votes": null, "replies": [] }, { "id": 1323028, "author_name": "datasciencegeek", "author_url": "", "post_date": "05/25/2021 22:13:32", "content": "I have tried two approaches:
\nSimple mean of predictions for soft outputs and then apply thresholds to produce solid labels.
The second approach I called \"Label Vote Ensemble\" for a lack of better term. The idea is to produce solid labels by each individual model and then pick final label based on how many models predicted that label. Let's say you have 5 models out of your 5-fold CV training, now you make 5 predictions with those models for an image, then for each class-label there can be up to 5 predictions - votes made by all 5 models. Use majority vote to decide on final prediction. So, if for class-label scab you get 4 votes of 5 but for other class-labels there were less than 3 votes of 5, then final prediction for the image will be scab.
I found that Label Vote Ensemble worked better than averaging based on LB scores I observed. Although, the majority vote logic was not the best one, in practice even if lower number of models predicted a certain class-label this was the true class-label for the most samples. I found that, for example, in the case of 5 models if at least 2 models predicted certain label it resulted in the highest score on public LB.
", "votes": null, "replies": [ { "id": 1324441, "author_name": "antoreepjana", "author_url": "", "post_date": "05/27/2021 00:24:49", "content": "@datasciencegeek Let me go through this and get back to you. Thanks! :)
", "votes": null, "replies": [] }, { "id": 1327650, "author_name": "coldfir3", "author_url": "", "post_date": "05/29/2021 13:50:56", "content": "So, in a nutshell, you are saying the ideal approach was to add another hyperparameter? you have the threshold for the probabilities (to compute a vote) and you have a threshold for the minimum number of votes? That is indeed interesting and I will use this approach in the next competition.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1321817": "I was wondering how would ensemble approaches are performing for this dataset. If you have experimented and don't mind sharing, then please do! :)", "1322163": "I'm using a mean of all predictions.", "1323028": "I have tried two approaches:\n\n1. Simple mean of predictions for soft outputs and then apply thresholds to produce solid labels.\n\n2. The second approach I called \"Label Vote Ensemble\" for a lack of better term. The idea is to produce solid labels by each individual model and then pick final label based on how many models predicted that label. Let's say you have 5 models out of your 5-fold CV training, now you make 5 predictions with those models for an image, then for each class-label there can be up to 5 predictions - votes made by all 5 models. Use majority vote to decide on final prediction. So, if for class-label **scab** you get 4 votes of 5 but for other class-labels there were less than 3 votes of 5, then final prediction for the image will be **scab**.\n\nI found that Label Vote Ensemble worked better than averaging based on LB scores I observed. Although, the majority vote logic was not the best one, in practice even if lower number of models predicted a certain class-label this was the true class-label for the most samples. I found that, for example, in the case of 5 models if at least 2 models predicted certain label it resulted in the highest score on public LB.", "1324441": "datasciencegeek Let me go through this and get back to you. Thanks! :)", "1327650": "So, in a nutshell, you are saying the ideal approach was to add another hyperparameter? you have the threshold for the probabilities (to compute a vote) and you have a threshold for the minimum number of votes? That is indeed interesting and I will use this approach in the next competition." }, "source": "meta" }