{
  "id": 370741,
  "title": "About Evaluation Metrics",
  "url": "/competitions/nfl-player-contact-detection/discussion/370741",
  "author_name": "Gaju Ahmed",
  "post_date": "2022-12-06T09:02:15.428000",
  "votes": 9,
  "comment_count": 0,
  "views": 0,
  "content": "<p><strong>Why MCC :</strong> These results show that, while accuracy and F1 score often generate high scores that do not inform the user about ongoing prediction issues, the MCC is a robust, useful, reliable, truthful statistical measure able to correctly reflect the deficiency of any prediction in any dataset.</p>\n<p>How to Calculate MCC using Sklearn <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.matthews_corrcoef.html\">here</a></p>\n<p>Lets try with Code:</p>\n<pre><code>from sklearn.metrics import matthews_corrcoef \n\nmatthews_set = []\n\n#Evaluate each test batch using Matthew's correlation coefficient\nprint('Calculating Matthews Corr. Coef. for each batch...')\n\n#For each input batch...\nfor i in range(len(true_labels)):\n        \"\"\"\n       The predictions for this batch are a 2-column ndarray (one column for \"0\" \n       and one column for \"1\"). Pick the label with the highest value and turn this\n       in to a list of 0s and 1s.\n       \"\"\"\n       pred_labels_i = np.argmax(predictions[i], axis=1).flatten()\n\n        # Calculate and store the coef for this batch.  \n        matthews = matthews_corrcoef(true_labels[i], pred_labels_i)                \n        matthews_set.append(matthews)\n\n\n# Combine the predictions for each batch into a single list of 0s and 1s.\nflat_predictions = [item for sublist in predictions for item in sublist]\nflat_predictions = np.argmax(flat_predictions, axis=1).flatten()\n\n# Combine the correct labels for each batch into a single list.\nflat_true_labels = [item for sublist in true_labels for item in sublist]\n\n# Calculate the MCC\nmcc = matthews_corrcoef(flat_true_labels, flat_predictions)\n\nprint('MCC: %.3f' % mcc)\n</code></pre>",
  "messages": [
    {
      "id": 2056577,
      "postDate": "2022-12-06T09:02:15.430Z",
      "content": "<p><strong>Why MCC :</strong> These results show that, while accuracy and F1 score often generate high scores that do not inform the user about ongoing prediction issues, the MCC is a robust, useful, reliable, truthful statistical measure able to correctly reflect the deficiency of any prediction in any dataset.</p>\n<p>How to Calculate MCC using Sklearn <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.matthews_corrcoef.html\">here</a></p>\n<p>Lets try with Code:</p>\n<pre><code>from sklearn.metrics import matthews_corrcoef \n\nmatthews_set = []\n\n#Evaluate each test batch using Matthew's correlation coefficient\nprint('Calculating Matthews Corr. Coef. for each batch...')\n\n#For each input batch...\nfor i in range(len(true_labels)):\n        \"\"\"\n       The predictions for this batch are a 2-column ndarray (one column for \"0\" \n       and one column for \"1\"). Pick the label with the highest value and turn this\n       in to a list of 0s and 1s.\n       \"\"\"\n       pred_labels_i = np.argmax(predictions[i], axis=1).flatten()\n\n        # Calculate and store the coef for this batch.  \n        matthews = matthews_corrcoef(true_labels[i], pred_labels_i)                \n        matthews_set.append(matthews)\n\n\n# Combine the predictions for each batch into a single list of 0s and 1s.\nflat_predictions = [item for sublist in predictions for item in sublist]\nflat_predictions = np.argmax(flat_predictions, axis=1).flatten()\n\n# Combine the correct labels for each batch into a single list.\nflat_true_labels = [item for sublist in true_labels for item in sublist]\n\n# Calculate the MCC\nmcc = matthews_corrcoef(flat_true_labels, flat_predictions)\n\nprint('MCC: %.3f' % mcc)\n</code></pre>",
      "rawMarkdown": "**Why MCC :** These results show that, while accuracy and F1 score often generate high scores that do not inform the user about ongoing prediction issues, the MCC is a robust, useful, reliable, truthful statistical measure able to correctly reflect the deficiency of any prediction in any dataset.\n\nHow to Calculate MCC using Sklearn <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.matthews_corrcoef.html\">here</a>\n\nLets try with Code:\n\n```\nfrom sklearn.metrics import matthews_corrcoef \n\nmatthews_set = []\n\n#Evaluate each test batch using Matthew's correlation coefficient\nprint('Calculating Matthews Corr. Coef. for each batch...')\n\n#For each input batch...\nfor i in range(len(true_labels)):\n        \"\"\"\n       The predictions for this batch are a 2-column ndarray (one column for \"0\" \n       and one column for \"1\"). Pick the label with the highest value and turn this\n       in to a list of 0s and 1s.\n       \"\"\"\n       pred_labels_i = np.argmax(predictions[i], axis=1).flatten()\n\n        # Calculate and store the coef for this batch.  \n        matthews = matthews_corrcoef(true_labels[i], pred_labels_i)                \n        matthews_set.append(matthews)\n\n\n# Combine the predictions for each batch into a single list of 0s and 1s.\nflat_predictions = [item for sublist in predictions for item in sublist]\nflat_predictions = np.argmax(flat_predictions, axis=1).flatten()\n\n# Combine the correct labels for each batch into a single list.\nflat_true_labels = [item for sublist in true_labels for item in sublist]\n\n# Calculate the MCC\nmcc = matthews_corrcoef(flat_true_labels, flat_predictions)\n\nprint('MCC: %.3f' % mcc)\n\n```\n\n\n",
      "votes": 9
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2056577": "**Why MCC :** These results show that, while accuracy and F1 score often generate high scores that do not inform the user about ongoing prediction issues, the MCC is a robust, useful, reliable, truthful statistical measure able to correctly reflect the deficiency of any prediction in any dataset.\n\nHow to Calculate MCC using Sklearn <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.matthews_corrcoef.html\">here</a>\n\nLets try with Code:\n\n```\nfrom sklearn.metrics import matthews_corrcoef \n\nmatthews_set = []\n\n#Evaluate each test batch using Matthew's correlation coefficient\nprint('Calculating Matthews Corr. Coef. for each batch...')\n\n#For each input batch...\nfor i in range(len(true_labels)):\n        \"\"\"\n       The predictions for this batch are a 2-column ndarray (one column for \"0\" \n       and one column for \"1\"). Pick the label with the highest value and turn this\n       in to a list of 0s and 1s.\n       \"\"\"\n       pred_labels_i = np.argmax(predictions[i], axis=1).flatten()\n\n        # Calculate and store the coef for this batch.  \n        matthews = matthews_corrcoef(true_labels[i], pred_labels_i)                \n        matthews_set.append(matthews)\n\n\n# Combine the predictions for each batch into a single list of 0s and 1s.\nflat_predictions = [item for sublist in predictions for item in sublist]\nflat_predictions = np.argmax(flat_predictions, axis=1).flatten()\n\n# Combine the correct labels for each batch into a single list.\nflat_true_labels = [item for sublist in true_labels for item in sublist]\n\n# Calculate the MCC\nmcc = matthews_corrcoef(flat_true_labels, flat_predictions)\n\nprint('MCC: %.3f' % mcc)\n\n```\n\n\n"
  }
}