{
  "id": 83321,
  "title": "evaluate Matthew's Correlation outside of Keras",
  "url": "/competitions/vsb-power-line-fault-detection/discussion/83321",
  "author_name": "",
  "post_date": "2019-03-08T15:12:59.661623700Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>def mattews_cor_eval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))</p>\n\n<p>which references:</p>\n\n<p>def matthews_correlation(y_true, y_pred):\n    '''Calculates the Matthews correlation coefficient measure for quality\n    of binary classification problems.\n    '''</p>\n\n<pre><code>y_pred_pos = K.round(K.clip(y_pred, 0, 1))\ny_pred_neg = 1 - y_pred_pos\n\ny_pos = K.round(K.clip(y_true, 0, 1))\ny_neg = 1 - y_pos\n\ntp = K.sum(y_pos * y_pred_pos)\ntn = K.sum(y_neg * y_pred_neg)\n\nfp = K.sum(y_neg * y_pred_pos)\nfn = K.sum(y_pos * y_pred_neg)\n\nnumerator = (tp * tn - fp * fn)\ndenominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\nreturn numerator / (denominator + K.epsilon())\n</code></pre>\n\n<p>You could use sklearn's version as well, but need to turn predictions into binaries first...</p>\n\n<p>Hope this helps someone...</p>",
  "messages": [
    {
      "id": "486301",
      "postDate": "03/08/2019 15:12:59",
      "content": "<p>def mattews_cor_eval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))</p>\n\n<p>which references:</p>\n\n<p>def matthews_correlation(y_true, y_pred):\n    '''Calculates the Matthews correlation coefficient measure for quality\n    of binary classification problems.\n    '''</p>\n\n<pre><code>y_pred_pos = K.round(K.clip(y_pred, 0, 1))\ny_pred_neg = 1 - y_pred_pos\n\ny_pos = K.round(K.clip(y_true, 0, 1))\ny_neg = 1 - y_pos\n\ntp = K.sum(y_pos * y_pred_pos)\ntn = K.sum(y_neg * y_pred_neg)\n\nfp = K.sum(y_neg * y_pred_pos)\nfn = K.sum(y_pos * y_pred_neg)\n\nnumerator = (tp * tn - fp * fn)\ndenominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\nreturn numerator / (denominator + K.epsilon())\n</code></pre>\n\n<p>You could use sklearn's version as well, but need to turn predictions into binaries first...</p>\n\n<p>Hope this helps someone...</p>",
      "rawMarkdown": "def mattews_cor_eval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))\n\nwhich references:\n\ndef matthews_correlation(y_true, y_pred):\n    '''Calculates the Matthews correlation coefficient measure for quality\n    of binary classification problems.\n    '''\n\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\n    return numerator / (denominator + K.epsilon())\n\nYou could use sklearn's version as well, but need to turn predictions into binaries first...\n\nHope this helps someone...",
      "votes": null
    },
    {
      "id": "486321",
      "postDate": "03/08/2019 15:57:37",
      "content": "<p>screwed up the code blocking.. here it is again...</p>\n\n<p><code>def mattewscoreval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))</code></p>\n\n<p>for </p>\n\n<p>`def matthews_correlation(y_true, y_pred):</p>\n\n<pre><code>y_pred_pos = K.round(K.clip(y_pred, 0, 1))\ny_pred_neg = 1 - y_pred_pos\n\ny_pos = K.round(K.clip(y_true, 0, 1))\ny_neg = 1 - y_pos\n\ntp = K.sum(y_pos * y_pred_pos)\ntn = K.sum(y_neg * y_pred_neg)\n\nfp = K.sum(y_neg * y_pred_pos)\nfn = K.sum(y_pos * y_pred_neg)\n\nnumerator = (tp * tn - fp * fn)\ndenominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\nreturn numerator / (denominator + K.epsilon())`\n</code></pre>",
      "rawMarkdown": "screwed up the code blocking.. here it is again...\n\n`def mattewscoreval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))`\n\nfor \n\n`def matthews_correlation(y_true, y_pred):\n\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\n    return numerator / (denominator + K.epsilon())`",
      "votes": null
    },
    {
      "id": "486617",
      "postDate": "03/09/2019 04:40:06",
      "content": "<p>I think you meant to show this:</p>\n\n<p><code>\ndef mattewscoreval(y_true, y_pred): \n    p = K.variable(np.array(y_pred), dtype='float32') \n    a = K.variable(np.array(y_true), dtype='float32') \n    return K.eval(matthews_correlation(a, p))\n</code>\nwhich references:</p>\n\n<p>```\ndef matthews_correlation(ytrue, y_pred):\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos</p>\n\n<pre><code>y_pos = K.round(K.clip(y_true, 0, 1))\ny_neg = 1 - y_pos\n\ntp = K.sum(y_pos * y_pred_pos)\ntn = K.sum(y_neg * y_pred_neg)\n\nfp = K.sum(y_neg * y_pred_pos)\nfn = K.sum(y_pos * y_pred_neg)\n\nnumerator = (tp * tn - fp * fn)\ndenominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\nreturn numerator / (denominator + K.epsilon())\n</code></pre>\n\n<p>```</p>",
      "rawMarkdown": "I think you meant to show this:\n\n```\ndef mattewscoreval(y_true, y_pred): \n    p = K.variable(np.array(y_pred), dtype='float32') \n    a = K.variable(np.array(y_true), dtype='float32') \n    return K.eval(matthews_correlation(a, p))\n```\nwhich references:\n\n```\ndef matthews_correlation(ytrue, y_pred):\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n    \n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n    \n    return numerator / (denominator + K.epsilon())\n```",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 486321,
      "author_name": "dr01alikat",
      "author_url": "",
      "post_date": "03/08/2019 15:57:37",
      "content": "<p>screwed up the code blocking.. here it is again...</p>\n\n<p><code>def mattewscoreval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))</code></p>\n\n<p>for </p>\n\n<p>`def matthews_correlation(y_true, y_pred):</p>\n\n<pre><code>y_pred_pos = K.round(K.clip(y_pred, 0, 1))\ny_pred_neg = 1 - y_pred_pos\n\ny_pos = K.round(K.clip(y_true, 0, 1))\ny_neg = 1 - y_pos\n\ntp = K.sum(y_pos * y_pred_pos)\ntn = K.sum(y_neg * y_pred_neg)\n\nfp = K.sum(y_neg * y_pred_pos)\nfn = K.sum(y_pos * y_pred_neg)\n\nnumerator = (tp * tn - fp * fn)\ndenominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\nreturn numerator / (denominator + K.epsilon())`\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 486617,
      "author_name": "huyunwei",
      "author_url": "",
      "post_date": "03/09/2019 04:40:06",
      "content": "<p>I think you meant to show this:</p>\n\n<p><code>\ndef mattewscoreval(y_true, y_pred): \n    p = K.variable(np.array(y_pred), dtype='float32') \n    a = K.variable(np.array(y_true), dtype='float32') \n    return K.eval(matthews_correlation(a, p))\n</code>\nwhich references:</p>\n\n<p>```\ndef matthews_correlation(ytrue, y_pred):\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos</p>\n\n<pre><code>y_pos = K.round(K.clip(y_true, 0, 1))\ny_neg = 1 - y_pos\n\ntp = K.sum(y_pos * y_pred_pos)\ntn = K.sum(y_neg * y_pred_neg)\n\nfp = K.sum(y_neg * y_pred_pos)\nfn = K.sum(y_pos * y_pred_neg)\n\nnumerator = (tp * tn - fp * fn)\ndenominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\nreturn numerator / (denominator + K.epsilon())\n</code></pre>\n\n<p>```</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "486301": "def mattews_cor_eval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))\n\nwhich references:\n\ndef matthews_correlation(y_true, y_pred):\n    '''Calculates the Matthews correlation coefficient measure for quality\n    of binary classification problems.\n    '''\n\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\n    return numerator / (denominator + K.epsilon())\n\nYou could use sklearn's version as well, but need to turn predictions into binaries first...\n\nHope this helps someone...",
    "486321": "screwed up the code blocking.. here it is again...\n\n`def mattewscoreval(y_true, y_pred):\n    p = K.variable(np.array(y_pred), dtype='float32')\n    a = K.variable(np.array(y_true), dtype='float32')\n    return K.eval(matthews_correlation(a, p))`\n\nfor \n\n`def matthews_correlation(y_true, y_pred):\n\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n\n    return numerator / (denominator + K.epsilon())`",
    "486617": "I think you meant to show this:\n\n```\ndef mattewscoreval(y_true, y_pred): \n    p = K.variable(np.array(y_pred), dtype='float32') \n    a = K.variable(np.array(y_true), dtype='float32') \n    return K.eval(matthews_correlation(a, p))\n```\nwhich references:\n\n```\ndef matthews_correlation(ytrue, y_pred):\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n    \n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n    \n    return numerator / (denominator + K.epsilon())\n```"
  },
  "source": "meta"
}