\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F877700%2F9d627bb3d011ce1c845c30c3eccda351%2Ftemp.png?generation=1611230617255165&alt=media\")
def get_score(y_true, y_pred):
\n scores = []
\n for i in range(y_true.shape[1]):
\n score = roc_auc_score(y_true[:,i], y_pred[:,i])
\n scores.append(score)
\n avg_score = np.mean(scores)
\n return avg_score, scores
Predictions and labels are same size.Predictions are between 0 and 1, labels are 0 or 1.
\nUsing this function when i try to calcuate accuracy i always get this:
\nOnly one class present in y_true. ROC AUC score is not defined in that case.
I have run into this error too because i was computing AUC after every batch.
\nI solved this issue by computing the AUC score after every epoch rather than every batch. The limitation with my i approach is that you cannot apply learning rate scheduler at every batch during training.
when you call roc_auc_score you need both 0 and 1 presented in your y_true[:, i]
otherwise you'll get that error.
", "rawMarkdown": "when you call `roc_auc_score ` you need both `0` and `1` presented in your `y_true[:, i]`\n\notherwise you'll get that error.", "replies": [ { "id": 1162709, "postDate": "2021-01-21T09:36:00.747Z", "content": "But how to avoid that, for example:
\ndf = df.sample(frac=0.001).reset_index(drop=True)
\nI take little sample and check training proces.
\nNow when i want to get labels for getting score later
\nlabels = df[target_cols].values
\nNow in labels i got one column with only zeros and cant get column wise score
To avoid the error during fast debugging, I use:
\n
Text:
\n
", "rawMarkdown": "To avoid the error during fast debugging, I use:\n\n\nText:\n
\ndef macro_multilabel_auc(label, pred):\n aucs = []\n for i in range(len(target_cols)):\n try:\n auc_class = roc_auc_score(label[:, i], pred[:, i])\n except:\n logprint(f\"couldn't compute auc for {i}th label. Assigning '0.5'\")\n auc_class = 0.5\n aucs.append(auc_class)\n return np.mean(aucs), aucs
\ndef macro_multilabel_auc(label, pred):\n aucs = []\n for i in range(len(target_cols)):\n try:\n auc_class = roc_auc_score(label[:, i], pred[:, i])\n except:\n logprint(f\"couldn't compute auc for {i}th label. Assigning '0.5'\")\n auc_class = 0.5\n aucs.append(auc_class)\n return np.mean(aucs), aucs"
},
{
"id": 1162928,
"postDate": "2021-01-21T12:06:10.627Z",
"content": "Thanks mate
",
"rawMarkdown": "Thanks mate "
}
]
},
{
"id": 1161828,
"postDate": "2021-01-20T19:11:43.473Z",
"content": "def get_score(y_true, y_pred):
\n scores = []
\n for i in range(y_true.shape[1]):
\n score = roc_auc_score(y_true[:,i], y_pred[:,i])
\n scores.append(score)
\n avg_score = np.mean(scores)
\n return avg_score, scores
\nPredictions and labels are same size.Predictions are between 0 and 1, labels are 0 or 1.
\nUsing this function when i try to calcuate accuracy i always get this:
\nOnly one class present in y_true. ROC AUC score is not defined in that case.
",
"rawMarkdown": "def get_score(y_true, y_pred):\n scores = []\n for i in range(y_true.shape[1]):\n score = roc_auc_score(y_true[:,i], y_pred[:,i])\n scores.append(score)\n avg_score = np.mean(scores)\n return avg_score, scores\n\nPredictions and labels are same size.Predictions are between 0 and 1, labels are 0 or 1.\n\nUsing this function when i try to calcuate accuracy i always get this:\nOnly one class present in y_true. ROC AUC score is not defined in that case.\n"
},
{
"id": 1162900,
"postDate": "2021-01-21T11:53:16.073Z",
"rawMarkdown": "",
"isDeleted": true
}
],
"comments": [
{
"id": 1163375,
"author_name": "Aravind P",
"author_url": "",
"post_date": "2021-01-21T16:52:17.493000",
"content": "I have run into this error too because i was computing AUC after every batch.
\nI solved this issue by computing the AUC score after every epoch rather than every batch. The limitation with my i approach is that you cannot apply learning rate scheduler at every batch during training.
",
"votes": 0,
"replies": []
},
{
"id": 1162356,
"author_name": "Qishen Ha",
"author_url": "",
"post_date": "2021-01-21T05:39:55.193000",
"content": "when you call roc_auc_score you need both 0 and 1 presented in your y_true[:, i]
\notherwise you'll get that error.
",
"votes": 0,
"replies": [
{
"id": 1162709,
"author_name": "wawepapi",
"author_url": "",
"post_date": "2021-01-21T09:36:00.747000",
"content": "But how to avoid that, for example:
\ndf = df.sample(frac=0.001).reset_index(drop=True)
\nI take little sample and check training proces.
\nNow when i want to get labels for getting score later
\nlabels = df[target_cols].values
\nNow in labels i got one column with only zeros and cant get column wise score
",
"votes": 0,
"replies": []
},
{
"id": 1162925,
"author_name": "Issagali Konysbayev [dsmlkz]",
"author_url": "",
"post_date": "2021-01-21T12:04:17.660000",
"content": "To avoid the error during fast debugging, I use:
\n
\nText:
\n
\ndef macro_multilabel_auc(label, pred):\n aucs = []\n for i in range(len(target_cols)):\n try:\n auc_class = roc_auc_score(label[:, i], pred[:, i])\n except:\n logprint(f\"couldn't compute auc for {i}th label. Assigning '0.5'\")\n auc_class = 0.5\n aucs.append(auc_class)\n return np.mean(aucs), aucs
",
"votes": 0,
"replies": []
},
{
"id": 1162928,
"author_name": "wawepapi",
"author_url": "",
"post_date": "2021-01-21T12:06:10.627000",
"content": "Thanks mate
",
"votes": 0,
"replies": []
}
]
},
{
"id": 1162900,
"author_name": "",
"author_url": "",
"post_date": "2021-01-21T11:53:16.073000",
"content": "",
"votes": 0,
"replies": []
}
],
"raw_markdown_by_id": {
"1163375": "I have run into this error too because i was computing AUC after every batch. \nI solved this issue by computing the AUC score after every epoch rather than every batch. The limitation with my i approach is that you cannot apply learning rate scheduler at every batch during training. ",
"1162356": "when you call `roc_auc_score ` you need both `0` and `1` presented in your `y_true[:, i]`\n\notherwise you'll get that error.",
"1161828": "def get_score(y_true, y_pred):\n scores = []\n for i in range(y_true.shape[1]):\n score = roc_auc_score(y_true[:,i], y_pred[:,i])\n scores.append(score)\n avg_score = np.mean(scores)\n return avg_score, scores\n\nPredictions and labels are same size.Predictions are between 0 and 1, labels are 0 or 1.\n\nUsing this function when i try to calcuate accuracy i always get this:\nOnly one class present in y_true. ROC AUC score is not defined in that case.\n",
"1162900": ""
}
}