I was working on creating a working baseline using public cv strategy as suggested by @yasufuminakama which uses GroupKfold splits where I encountered with this error ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.
\nWhile digging it further, I realized there aren't enough positive sample values in some targets.
I was wondering as the data is imbalanced, it could be hard for models to learn if there aren't enough positive sample. I believe having enough positive samples in every class of a fold should also help in the learning process of model. So, what I really wanted to know is, shouldn't we prefer using Stratified+ Group Kfold for making splits in this competition?
\nAs I still lack experience and not a professional, I would really love to hear thoughts about it from the community.
\nThank you.
", "messages": [ { "id": "1176028", "postDate": "01/29/2021 13:14:56", "content": "I was working on creating a working baseline using public cv strategy as suggested by @yasufuminakama which uses GroupKfold splits where I encountered with this error ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.
\nWhile digging it further, I realized there aren't enough positive sample values in some targets.
I was wondering as the data is imbalanced, it could be hard for models to learn if there aren't enough positive sample. I believe having enough positive samples in every class of a fold should also help in the learning process of model. So, what I really wanted to know is, shouldn't we prefer using Stratified+ Group Kfold for making splits in this competition?
\nAs I still lack experience and not a professional, I would really love to hear thoughts about it from the community.
\nThank you.
", "rawMarkdown": "I was working on creating a working baseline using public cv strategy as suggested by @yasufuminakama which uses GroupKfold splits where I encountered with this error *ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.*\nWhile digging it further, I realized there aren't enough positive sample values in some targets.\n\nI was wondering as the data is imbalanced, it could be hard for models to learn if there aren't enough positive sample. I believe having enough positive samples in every class of a fold should also help in the learning process of model. So, what I really wanted to know is, shouldn't we prefer using **Stratified+ Group Kfold** for making splits in this competition?\n\nAs I still lack experience and not a professional, I would really love to hear thoughts about it from the community. \n\nThank you.", "votes": null }, { "id": "1176985", "postDate": "01/30/2021 00:03:18", "content": "GroupKFold doesn't enforce stratification but I think it is ok to use since it is almost stratified(https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/204638) If you want to do StratifiedGroupKFold, you can refer to https://www.kaggle.com/underwearfitting/how-to-properly-split-folds
", "rawMarkdown": "GroupKFold doesn't enforce stratification but I think it is ok to use since it is almost stratified(https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/204638) If you want to do StratifiedGroupKFold, you can refer to https://www.kaggle.com/underwearfitting/how-to-properly-split-folds", "votes": null }, { "id": "1177825", "postDate": "01/30/2021 14:32:30", "content": "Thank you for your feedback @harangdev.
", "rawMarkdown": "Thank you for your feedback @harangdev.", "votes": null } ], "comments": [ { "id": 1176985, "author_name": "harangdev", "author_url": "", "post_date": "01/30/2021 00:03:18", "content": "GroupKFold doesn't enforce stratification but I think it is ok to use since it is almost stratified(https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/204638) If you want to do StratifiedGroupKFold, you can refer to https://www.kaggle.com/underwearfitting/how-to-properly-split-folds
", "votes": null, "replies": [ { "id": 1177825, "author_name": "nischaydnk", "author_url": "", "post_date": "01/30/2021 14:32:30", "content": "Thank you for your feedback @harangdev.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1176028": "I was working on creating a working baseline using public cv strategy as suggested by @yasufuminakama which uses GroupKfold splits where I encountered with this error *ValueError: Only one class present in y_true. ROC AUC score is not defined in that case.*\nWhile digging it further, I realized there aren't enough positive sample values in some targets.\n\nI was wondering as the data is imbalanced, it could be hard for models to learn if there aren't enough positive sample. I believe having enough positive samples in every class of a fold should also help in the learning process of model. So, what I really wanted to know is, shouldn't we prefer using **Stratified+ Group Kfold** for making splits in this competition?\n\nAs I still lack experience and not a professional, I would really love to hear thoughts about it from the community. \n\nThank you.", "1176985": "GroupKFold doesn't enforce stratification but I think it is ok to use since it is almost stratified(https://www.kaggle.com/c/ranzcr-clip-catheter-line-classification/discussion/204638) If you want to do StratifiedGroupKFold, you can refer to https://www.kaggle.com/underwearfitting/how-to-properly-split-folds", "1177825": "Thank you for your feedback @harangdev." }, "source": "meta" }