{
  "id": 81818,
  "title": "How to fix keras error AttributeError: 'numpy.dtype' object has no attribute 'base_dtype'",
  "url": "/competitions/vsb-power-line-fault-detection/discussion/81818",
  "author_name": "",
  "post_date": "2019-02-25T09:25:44.357188500Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>All of a sudden I started getting the following error, while using the matthews_correlation function. The same code was working earlier.</p>\n\n<p>Anyone experienced the same issue and knows how to solve? Please help. </p>\n\n<p>Google says some version issue with Keras, but since I am using Kaggle Kernel, not sure how I can make changes to Keras version.</p>\n\n<p>Thanks,\nSubrahmanyam</p>\n\n<hr>\n\n<p>AttributeError                            Traceback (most recent call last)\n in ()\n----&gt; 1 best_threshold, best_score = threshold_search(y_val, preds_val)\n      2 \n      3 print('best_threshold: {}  best_score: {}'.format(best_threshold,best_score))</p>\n\n<p> in threshold_search(y_true, y_proba)\n      6     best_score = 0\n      7     for threshold in tqdm([i * 0.01 for i in range(100)]):\n----&gt; 8         score = K.eval(matthews_correlation(y_true.astype(np.float64), (y_proba &gt; threshold).astype(np.float64)))\n      9         if score &gt; best_score:\n     10             best_threshold = threshold</p>\n\n<p> in matthews_correlation(y_true, y_pred)\n      5     of binary classification problems.\n      6     '''\n----&gt; 7     y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n      8     y_pred_neg = 1 - y_pred_pos\n      9 </p>\n\n<p>/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in clip(x, min_value, max_value)\n   1599     if max_value is None:\n   1600         max_value = np.inf\n-&gt; 1601     min_value = _to_tensor(min_value, x.dtype.base_dtype)\n   1602     max_value = _to_tensor(max_value, x.dtype.base_dtype)\n   1603     return tf.clip_by_value(x, min_value, max_value)</p>\n\n<h2>AttributeError: 'numpy.dtype' object has no attribute 'base_dtype'</h2>",
  "messages": [
    {
      "id": "477799",
      "postDate": "02/25/2019 09:25:44",
      "content": "<p>All of a sudden I started getting the following error, while using the matthews_correlation function. The same code was working earlier.</p>\n\n<p>Anyone experienced the same issue and knows how to solve? Please help. </p>\n\n<p>Google says some version issue with Keras, but since I am using Kaggle Kernel, not sure how I can make changes to Keras version.</p>\n\n<p>Thanks,\nSubrahmanyam</p>\n\n<hr>\n\n<p>AttributeError                            Traceback (most recent call last)\n in ()\n----&gt; 1 best_threshold, best_score = threshold_search(y_val, preds_val)\n      2 \n      3 print('best_threshold: {}  best_score: {}'.format(best_threshold,best_score))</p>\n\n<p> in threshold_search(y_true, y_proba)\n      6     best_score = 0\n      7     for threshold in tqdm([i * 0.01 for i in range(100)]):\n----&gt; 8         score = K.eval(matthews_correlation(y_true.astype(np.float64), (y_proba &gt; threshold).astype(np.float64)))\n      9         if score &gt; best_score:\n     10             best_threshold = threshold</p>\n\n<p> in matthews_correlation(y_true, y_pred)\n      5     of binary classification problems.\n      6     '''\n----&gt; 7     y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n      8     y_pred_neg = 1 - y_pred_pos\n      9 </p>\n\n<p>/opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in clip(x, min_value, max_value)\n   1599     if max_value is None:\n   1600         max_value = np.inf\n-&gt; 1601     min_value = _to_tensor(min_value, x.dtype.base_dtype)\n   1602     max_value = _to_tensor(max_value, x.dtype.base_dtype)\n   1603     return tf.clip_by_value(x, min_value, max_value)</p>\n\n<h2>AttributeError: 'numpy.dtype' object has no attribute 'base_dtype'</h2>",
      "rawMarkdown": "All of a sudden I started getting the following error, while using the matthews_correlation function. The same code was working earlier.\n\nAnyone experienced the same issue and knows how to solve? Please help. \n\nGoogle says some version issue with Keras, but since I am using Kaggle Kernel, not sure how I can make changes to Keras version.\n\nThanks,\nSubrahmanyam\n\n\n--------------------------------------------------------------------------\n\nAttributeError                            Traceback (most recent call last)",
      "votes": null
    },
    {
      "id": "477941",
      "postDate": "02/25/2019 14:35:05",
      "content": "<p>This one should work :</p>\n\n<pre><code>def MMC(y_true, y_pred):\n    y_pred = tf.convert_to_tensor(y_pred, np.float32)\n    y_true = tf.convert_to_tensor(y_true, np.float32)\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n    return numerator / (denominator + K.epsilon())\n</code></pre>",
      "rawMarkdown": "This one should work :\n\n<pre><code>def MMC(y_true, y_pred):\n    y_pred = tf.convert_to_tensor(y_pred, np.float32)\n    y_true = tf.convert_to_tensor(y_true, np.float32)\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n    return numerator / (denominator + K.epsilon())\n</code></pre>",
      "votes": null
    },
    {
      "id": "478460",
      "postDate": "02/26/2019 07:29:51",
      "content": "<p>Thanks Antoine. Worked like a charm.\nUsing your solution, I understood that with the new version of Keras, it is mandatory to pass tensors to clip function.</p>",
      "rawMarkdown": "Thanks Antoine. Worked like a charm.\nUsing your solution, I understood that with the new version of Keras, it is mandatory to pass tensors to clip function.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 477941,
      "author_name": "areveillon",
      "author_url": "",
      "post_date": "02/25/2019 14:35:05",
      "content": "<p>This one should work :</p>\n\n<pre><code>def MMC(y_true, y_pred):\n    y_pred = tf.convert_to_tensor(y_pred, np.float32)\n    y_true = tf.convert_to_tensor(y_true, np.float32)\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n    return numerator / (denominator + K.epsilon())\n</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 478460,
          "author_name": "subrahmanyamv",
          "author_url": "",
          "post_date": "02/26/2019 07:29:51",
          "content": "<p>Thanks Antoine. Worked like a charm.\nUsing your solution, I understood that with the new version of Keras, it is mandatory to pass tensors to clip function.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "477799": "All of a sudden I started getting the following error, while using the matthews_correlation function. The same code was working earlier.\n\nAnyone experienced the same issue and knows how to solve? Please help. \n\nGoogle says some version issue with Keras, but since I am using Kaggle Kernel, not sure how I can make changes to Keras version.\n\nThanks,\nSubrahmanyam\n\n\n--------------------------------------------------------------------------\n\nAttributeError                            Traceback (most recent call last)",
    "477941": "This one should work :\n\n<pre><code>def MMC(y_true, y_pred):\n    y_pred = tf.convert_to_tensor(y_pred, np.float32)\n    y_true = tf.convert_to_tensor(y_true, np.float32)\n    y_pred_pos = K.round(K.clip(y_pred, 0, 1))\n    y_pred_neg = 1 - y_pred_pos\n    y_pos = K.round(K.clip(y_true, 0, 1))\n    y_neg = 1 - y_pos\n    tp = K.sum(y_pos * y_pred_pos)\n    tn = K.sum(y_neg * y_pred_neg)\n    fp = K.sum(y_neg * y_pred_pos)\n    fn = K.sum(y_pos * y_pred_neg)\n    numerator = (tp * tn - fp * fn)\n    denominator = K.sqrt((tp + fp) * (tp + fn) * (tn + fp) * (tn + fn))\n    return numerator / (denominator + K.epsilon())\n</code></pre>",
    "478460": "Thanks Antoine. Worked like a charm.\nUsing your solution, I understood that with the new version of Keras, it is mandatory to pass tensors to clip function."
  },
  "source": "meta"
}