{
  "id": 79462,
  "title": "Help me figure out the f1_smart function",
  "url": "/competitions/quora-insincere-questions-classification/discussion/79462",
  "author_name": "",
  "post_date": "2019-02-04T11:19:10.533917Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi all, </p>\n\n<p>Can you help me figure this out ? Everyone seems to be using it but i haven't found a good explanation.</p>\n\n<p>What does \"fs\" stand for ? how is that equal to (precison*recall)/(precision+recall) ?\nthanks,</p>\n\n<blockquote>\n<pre><code>def f1_smart(y_true, y_pred):\n    args = np.argsort(y_pred)\n\n    tp = y_true.sum()\n\n    fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)\n\n    res_idx = np.argmax(fs)\n\n    f1 = 2 * fs[res_idx]\n\n    threshold = (y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2\n    return f1, threshold\n</code></pre>\n</blockquote>",
  "messages": [
    {
      "id": "465944",
      "postDate": "02/04/2019 11:19:10",
      "content": "<p>Hi all, </p>\n\n<p>Can you help me figure this out ? Everyone seems to be using it but i haven't found a good explanation.</p>\n\n<p>What does \"fs\" stand for ? how is that equal to (precison*recall)/(precision+recall) ?\nthanks,</p>\n\n<blockquote>\n<pre><code>def f1_smart(y_true, y_pred):\n    args = np.argsort(y_pred)\n\n    tp = y_true.sum()\n\n    fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)\n\n    res_idx = np.argmax(fs)\n\n    f1 = 2 * fs[res_idx]\n\n    threshold = (y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2\n    return f1, threshold\n</code></pre>\n</blockquote>",
      "rawMarkdown": "Hi all, \n\nCan you help me figure this out ? Everyone seems to be using it but i haven't found a good explanation.\n\nWhat does \"fs\" stand for ? how is that equal to (precison*recall)/(precision+recall) ?\nthanks,\n\n\n&gt;     def f1_smart(y_true, y_pred):\n&gt;         args = np.argsort(y_pred)\n&gt;     \n&gt;         tp = y_true.sum()\n&gt;     \n&gt;         fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)\n&gt;     \n&gt;         res_idx = np.argmax(fs)\n&gt;     \n&gt;         f1 = 2 * fs[res_idx]\n&gt;     \n&gt;         threshold = (y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2\n&gt;         return f1, threshold",
      "votes": null
    },
    {
      "id": "466302",
      "postDate": "02/05/2019 04:43:52",
      "content": "<p>Well based on it's name it certainly looks like it's related to the traditional <a href=\"https://en.wikipedia.org/wiki/F1_score\">F1 Score</a> evaluation metric. The 'smart' in the name could be referring to either a 'smart implementation' that might be a more efficient way of calculating the F1 Score, or it could be 'smart' in the sense that it's an improvement on the traditional F1 Score. If I were you, I'd examine the outputs of this function when compared to a normal F1 Score function (like <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html\">the one</a> from SciKit-Learn) and see if they're the same in terms of output and running time. </p>",
      "rawMarkdown": "Well based on it's name it certainly looks like it's related to the traditional [F1 Score](https://en.wikipedia.org/wiki/F1_score) evaluation metric. The 'smart' in the name could be referring to either a 'smart implementation' that might be a more efficient way of calculating the F1 Score, or it could be 'smart' in the sense that it's an improvement on the traditional F1 Score. If I were you, I'd examine the outputs of this function when compared to a normal F1 Score function (like [the one](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html) from SciKit-Learn) and see if they're the same in terms of output and running time.",
      "votes": null
    },
    {
      "id": "466372",
      "postDate": "02/05/2019 08:33:33",
      "content": "<p>hi Alec i did compare it with the output of F1 score from sklearn they are the same. I also went through it line by line and checked all the outputs i just cannot put it together in my head. The line with fs bothers me i don't see how that is equivalent to the the  (precison*recall)/(precision+recall).</p>",
      "rawMarkdown": "hi Alec i did compare it with the output of F1 score from sklearn they are the same. I also went through it line by line and checked all the outputs i just cannot put it together in my head. The line with fs bothers me i don't see how that is equivalent to the the  (precison*recall)/(precision+recall).",
      "votes": null
    },
    {
      "id": "466521",
      "postDate": "02/05/2019 14:53:55",
      "content": "<p>I assume they do it in multiple parts with shortcuts, lines like <code>fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)</code> and <code>(y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2</code> certainly look similar to the traditional F1 code to me. If you manually calculate precision and recall, their difference and their sum, maybe even all data from a confusion matrix (write them all down) and then compare them to the values that are being added or divided, you can see at what point they start to match up and how. </p>\n\n<p>Sorry I couldn't be of more help, I'm not familiar with that specific function, if you figure out how it's replicating F1 please do let me know I'd be very interested in finding out as well. </p>",
      "rawMarkdown": "I assume they do it in multiple parts with shortcuts, lines like `fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)` and `(y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2` certainly look similar to the traditional F1 code to me. If you manually calculate precision and recall, their difference and their sum, maybe even all data from a confusion matrix (write them all down) and then compare them to the values that are being added or divided, you can see at what point they start to match up and how. \n\nSorry I couldn't be of more help, I'm not familiar with that specific function, if you figure out how it's replicating F1 please do let me know I'd be very interested in finding out as well.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 466302,
      "author_name": "alecthekulak",
      "author_url": "",
      "post_date": "02/05/2019 04:43:52",
      "content": "<p>Well based on it's name it certainly looks like it's related to the traditional <a href=\"https://en.wikipedia.org/wiki/F1_score\">F1 Score</a> evaluation metric. The 'smart' in the name could be referring to either a 'smart implementation' that might be a more efficient way of calculating the F1 Score, or it could be 'smart' in the sense that it's an improvement on the traditional F1 Score. If I were you, I'd examine the outputs of this function when compared to a normal F1 Score function (like <a href=\"https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html\">the one</a> from SciKit-Learn) and see if they're the same in terms of output and running time. </p>",
      "votes": null,
      "replies": [
        {
          "id": 466372,
          "author_name": "vonneumann",
          "author_url": "",
          "post_date": "02/05/2019 08:33:33",
          "content": "<p>hi Alec i did compare it with the output of F1 score from sklearn they are the same. I also went through it line by line and checked all the outputs i just cannot put it together in my head. The line with fs bothers me i don't see how that is equivalent to the the  (precison*recall)/(precision+recall).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 466521,
          "author_name": "alecthekulak",
          "author_url": "",
          "post_date": "02/05/2019 14:53:55",
          "content": "<p>I assume they do it in multiple parts with shortcuts, lines like <code>fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)</code> and <code>(y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2</code> certainly look similar to the traditional F1 code to me. If you manually calculate precision and recall, their difference and their sum, maybe even all data from a confusion matrix (write them all down) and then compare them to the values that are being added or divided, you can see at what point they start to match up and how. </p>\n\n<p>Sorry I couldn't be of more help, I'm not familiar with that specific function, if you figure out how it's replicating F1 please do let me know I'd be very interested in finding out as well. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "465944": "Hi all, \n\nCan you help me figure this out ? Everyone seems to be using it but i haven't found a good explanation.\n\nWhat does \"fs\" stand for ? how is that equal to (precison*recall)/(precision+recall) ?\nthanks,\n\n\n&gt;     def f1_smart(y_true, y_pred):\n&gt;         args = np.argsort(y_pred)\n&gt;     \n&gt;         tp = y_true.sum()\n&gt;     \n&gt;         fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)\n&gt;     \n&gt;         res_idx = np.argmax(fs)\n&gt;     \n&gt;         f1 = 2 * fs[res_idx]\n&gt;     \n&gt;         threshold = (y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2\n&gt;         return f1, threshold",
    "466302": "Well based on it's name it certainly looks like it's related to the traditional [F1 Score](https://en.wikipedia.org/wiki/F1_score) evaluation metric. The 'smart' in the name could be referring to either a 'smart implementation' that might be a more efficient way of calculating the F1 Score, or it could be 'smart' in the sense that it's an improvement on the traditional F1 Score. If I were you, I'd examine the outputs of this function when compared to a normal F1 Score function (like [the one](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html) from SciKit-Learn) and see if they're the same in terms of output and running time.",
    "466372": "hi Alec i did compare it with the output of F1 score from sklearn they are the same. I also went through it line by line and checked all the outputs i just cannot put it together in my head. The line with fs bothers me i don't see how that is equivalent to the the  (precison*recall)/(precision+recall).",
    "466521": "I assume they do it in multiple parts with shortcuts, lines like `fs = (tp - np.cumsum(y_true[args[:-1]])) / np.arange(y_true.shape[0] + tp - 1, tp, -1)` and `(y_pred[args[res_idx]] + y_pred[args[res_idx + 1]]) / 2` certainly look similar to the traditional F1 code to me. If you manually calculate precision and recall, their difference and their sum, maybe even all data from a confusion matrix (write them all down) and then compare them to the values that are being added or divided, you can see at what point they start to match up and how. \n\nSorry I couldn't be of more help, I'm not familiar with that specific function, if you figure out how it's replicating F1 please do let me know I'd be very interested in finding out as well."
  },
  "source": "meta"
}