{
  "id": 70747,
  "title": "F1 Score Evaluation?",
  "url": "/competitions/quora-insincere-questions-classification/discussion/70747",
  "author_name": "Strideradu",
  "post_date": "2018-11-07T01:02:36.928000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I have used the following code for the F1 score evaluation, but found inconsistent with LB (val f1 is only 0.03), what am I missing?</p>\n\n<pre><code>import tensorflow as tf\n\ndef f1_score(y_true, y_pred):\n    y_true = tf.cast(y_true, \"int32\")\n    y_pred = tf.cast(tf.round(y_pred), \"int32\") # implicit 0.5 threshold via tf.round\n    y_correct = y_true * y_pred\n    sum_true = tf.reduce_sum(y_true, axis=1)\n    sum_pred = tf.reduce_sum(y_pred, axis=1)\n    sum_correct = tf.reduce_sum(y_correct, axis=1)\n    precision = sum_correct / sum_pred\n    recall = sum_correct / sum_true\n    f_score = 2 * precision * recall / (precision + recall)\n    f_score = tf.where(tf.is_nan(f_score), tf.zeros_like(f_score), f_score)\n    return tf.reduce_mean(f_score)\n</code></pre>",
  "messages": [
    {
      "id": 416606,
      "postDate": "2018-11-07T01:02:36.930Z",
      "content": "<p>I have used the following code for the F1 score evaluation, but found inconsistent with LB (val f1 is only 0.03), what am I missing?</p>\n\n<pre><code>import tensorflow as tf\n\ndef f1_score(y_true, y_pred):\n    y_true = tf.cast(y_true, \"int32\")\n    y_pred = tf.cast(tf.round(y_pred), \"int32\") # implicit 0.5 threshold via tf.round\n    y_correct = y_true * y_pred\n    sum_true = tf.reduce_sum(y_true, axis=1)\n    sum_pred = tf.reduce_sum(y_pred, axis=1)\n    sum_correct = tf.reduce_sum(y_correct, axis=1)\n    precision = sum_correct / sum_pred\n    recall = sum_correct / sum_true\n    f_score = 2 * precision * recall / (precision + recall)\n    f_score = tf.where(tf.is_nan(f_score), tf.zeros_like(f_score), f_score)\n    return tf.reduce_mean(f_score)\n</code></pre>",
      "rawMarkdown": "I have used the following code for the F1 score evaluation, but found inconsistent with LB (val f1 is only 0.03), what am I missing?\n\n    import tensorflow as tf\n    \n    def f1_score(y_true, y_pred):\n        y_true = tf.cast(y_true, \"int32\")\n        y_pred = tf.cast(tf.round(y_pred), \"int32\") # implicit 0.5 threshold via tf.round\n        y_correct = y_true * y_pred\n        sum_true = tf.reduce_sum(y_true, axis=1)\n        sum_pred = tf.reduce_sum(y_pred, axis=1)\n        sum_correct = tf.reduce_sum(y_correct, axis=1)\n        precision = sum_correct / sum_pred\n        recall = sum_correct / sum_true\n        f_score = 2 * precision * recall / (precision + recall)\n        f_score = tf.where(tf.is_nan(f_score), tf.zeros_like(f_score), f_score)\n        return tf.reduce_mean(f_score)",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "416606": "I have used the following code for the F1 score evaluation, but found inconsistent with LB (val f1 is only 0.03), what am I missing?\n\n    import tensorflow as tf\n    \n    def f1_score(y_true, y_pred):\n        y_true = tf.cast(y_true, \"int32\")\n        y_pred = tf.cast(tf.round(y_pred), \"int32\") # implicit 0.5 threshold via tf.round\n        y_correct = y_true * y_pred\n        sum_true = tf.reduce_sum(y_true, axis=1)\n        sum_pred = tf.reduce_sum(y_pred, axis=1)\n        sum_correct = tf.reduce_sum(y_correct, axis=1)\n        precision = sum_correct / sum_pred\n        recall = sum_correct / sum_true\n        f_score = 2 * precision * recall / (precision + recall)\n        f_score = tf.where(tf.is_nan(f_score), tf.zeros_like(f_score), f_score)\n        return tf.reduce_mean(f_score)"
  }
}