When I was experimenting with multiple model to check which model is able to perform well & which is giving random guess , got to found LightGBM model train method was giving above 0.8 ROC AUC score same param with LGBMClassifier fit method could not able to get score above 0.5 of ROC AUC is their any difference!!
\nPlease have a look at below notebook in Light GBM Model Section :
\nhttps://www.kaggle.com/code/kunduruanil/credit-risk-model-feature-selection
Please check if you are using predict/ predict_proba for your scikit-learn interface @kunduruanil
\nUsing predict usually produces low scores.
\nIn the native booster syntax, you can freely use predict for regression and classification problems @kunduruanil
Currently I am using predict , try with predict_proba !!
\nI think you are tagging haileynah not mee !!
", "rawMarkdown": "Currently I am using predict , try with predict_proba !! \n\nI think you are tagging haileynah not mee !! ", "votes": 1, "replies": [ { "id": 2793994, "postDate": "2024-05-05T05:27:09.857Z", "content": "So sorry for the wrong tagging, rectified it @kunduruanil 😀
\nGood that the suggestion helped! I prefer the native booster syntax for this very reason. You can use the same syntax for classification and regression and use model.predict() for any type of predictions as well
I am using below code to evaluate could not found any improvement :
\ndef evalvate(model,X,y):
\n y_prob = model.predict_proba(X)
\n threshold = 0.5
\n y_pred = np.where(y_prob[:, 1] >= threshold, 1, 0)
\n print(f\"ROC AUC Score : {roc_auc_score(y,y_pred)}\")
\n y_pred = np.where(y_pred>=0.5,1,0)
\n print(f\"Precision Score : {precision_score(y,y_pred)}\")
\n print(\"\"50)
please check notebook as well : https://www.kaggle.com/code/kunduruanil/credit-risk-model-feature-selection
", "rawMarkdown": "I am using below code to evaluate could not found any improvement :\n\ndef evalvate(model,X,y):\n y_prob = model.predict_proba(X)\n threshold = 0.5\n y_pred = np.where(y_prob[:, 1] >= threshold, 1, 0)\n print(f\"ROC AUC Score : {roc_auc_score(y,y_pred)}\")\n y_pred = np.where(y_pred>=0.5,1,0)\n print(f\"Precision Score : {precision_score(y,y_pred)}\")\n print(\"*\"*50)\n\nplease check notebook as well : [https://www.kaggle.com/code/kunduruanil/credit-risk-model-feature-selection](url)" }, { "id": 2797015, "postDate": "2024-05-06T14:00:13.930Z", "content": "Check your embedded links… a click on these links don't take people to where you think they should.
", "rawMarkdown": "Check your embedded links... a click on these links don't take people to where you think they should. " } ] } ] }, { "id": 2793378, "postDate": "2024-05-04T17:21:48.220Z", "content": "When I was experimenting with multiple model to check which model is able to perform well & which is giving random guess , got to found LightGBM model train method was giving above 0.8 ROC AUC score same param with LGBMClassifier fit method could not able to get score above 0.5 of ROC AUC is their any difference!!
\nPlease have a look at below notebook in Light GBM Model Section :
\nhttps://www.kaggle.com/code/kunduruanil/credit-risk-model-feature-selection
lgb train also seems to provide easier ways to use custom metrics. By comparison, using custom metric in LGBMClassifier requires writing callback functions and such
", "rawMarkdown": "lgb train also seems to provide easier ways to use custom metrics. By comparison, using custom metric in LGBMClassifier requires writing callback functions and such", "replies": [ { "id": 2813553, "postDate": "2024-05-14T20:23:11.470Z", "content": "With fit method it's implemented in this notebook https://www.kaggle.com/code/eu1234/creditrisk-sklearn-pipeline-integrated-metric
", "rawMarkdown": "With fit method it's implemented in this notebook https://www.kaggle.com/code/eu1234/creditrisk-sklearn-pipeline-integrated-metric" } ] }, { "id": 2797038, "postDate": "2024-05-06T14:06:50.923Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2793504, "postDate": "2024-05-04T18:52:35.253Z", "content": "Thank you for sharing!
", "rawMarkdown": "Thank you for sharing!", "votes": 1 } ], "comments": [ { "id": 2793969, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-05-05T05:10:33.143000", "content": "Please check if you are using predict/ predict_proba for your scikit-learn interface @kunduruanil
\nUsing predict usually produces low scores.
\nIn the native booster syntax, you can freely use predict for regression and classification problems @kunduruanil
Currently I am using predict , try with predict_proba !!
\nI think you are tagging haileynah not mee !!
", "votes": 1, "replies": [ { "id": 2793994, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-05-05T05:27:09.857000", "content": "So sorry for the wrong tagging, rectified it @kunduruanil 😀
\nGood that the suggestion helped! I prefer the native booster syntax for this very reason. You can use the same syntax for classification and regression and use model.predict() for any type of predictions as well
I am using below code to evaluate could not found any improvement :
\ndef evalvate(model,X,y):
\n y_prob = model.predict_proba(X)
\n threshold = 0.5
\n y_pred = np.where(y_prob[:, 1] >= threshold, 1, 0)
\n print(f\"ROC AUC Score : {roc_auc_score(y,y_pred)}\")
\n y_pred = np.where(y_pred>=0.5,1,0)
\n print(f\"Precision Score : {precision_score(y,y_pred)}\")
\n print(\"\"50)
please check notebook as well : https://www.kaggle.com/code/kunduruanil/credit-risk-model-feature-selection
", "votes": 0, "replies": [] }, { "id": 2797015, "author_name": "Eduard Stefanescu", "author_url": "", "post_date": "2024-05-06T14:00:13.930000", "content": "Check your embedded links… a click on these links don't take people to where you think they should.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2799899, "author_name": "TkiJoestar", "author_url": "", "post_date": "2024-05-08T03:04:32.393000", "content": "lgb train also seems to provide easier ways to use custom metrics. By comparison, using custom metric in LGBMClassifier requires writing callback functions and such
", "votes": 0, "replies": [ { "id": 2813553, "author_name": "Danu A.", "author_url": "", "post_date": "2024-05-14T20:23:11.470000", "content": "With fit method it's implemented in this notebook https://www.kaggle.com/code/eu1234/creditrisk-sklearn-pipeline-integrated-metric
", "votes": 0, "replies": [] } ] }, { "id": 2797038, "author_name": "", "author_url": "", "post_date": "2024-05-06T14:06:50.923000", "content": "", "votes": 0, "replies": [] }, { "id": 2793504, "author_name": "haileynah", "author_url": "", "post_date": "2024-05-04T18:52:35.253000", "content": "Thank you for sharing!
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2793969": "Please check if you are using **predict**/ **predict_proba** for your scikit-learn interface @kunduruanil \nUsing **predict** usually produces low scores. \nIn the native booster syntax, you can freely use predict for regression and classification problems @kunduruanil ", "2793378": "When I was experimenting with multiple model to check which model is able to perform well & which is giving random guess , got to found LightGBM model train method was giving above 0.8 ROC AUC score same param with LGBMClassifier fit method could not able to get score above 0.5 of ROC AUC is their any difference!! \n\n\nPlease have a look at below notebook in Light GBM Model Section : \n[https://www.kaggle.com/code/kunduruanil/credit-risk-model-feature-selection](url)", "2799899": "lgb train also seems to provide easier ways to use custom metrics. By comparison, using custom metric in LGBMClassifier requires writing callback functions and such", "2797038": "", "2793504": "Thank you for sharing!" } }