{
  "id": 72175,
  "title": "What is the normal range of f1 score in valid datasets during training?In training, I usually get a very low f1 score, about 0.3,what is your score?welcome to share your experience!",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/72175",
  "author_name": "",
  "post_date": "2018-11-21T03:39:54.720000",
  "votes": 0,
  "comment_count": 7,
  "views": 0,
  "content": "",
  "messages": [
    {
      "id": 425784,
      "postDate": "2018-11-22T05:15:33.080Z",
      "content": "<p>I search the threshold which get maximum f1 score on my validation, my f1 score on validation set about 0.74</p>",
      "rawMarkdown": "I search the threshold which get maximum f1 score on my validation, my f1 score on validation set about 0.74",
      "votes": 2,
      "replies": [
        {
          "id": 427098,
          "postDate": "2018-11-24T15:06:02.913Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 425601,
      "postDate": "2018-11-21T20:42:13.977Z",
      "content": "<p>I’m getting f1 scores of ~.77-.78 on my validation set to get public LB of .478. I think it’s been mentioned before that class imbalances are contributing to this huge difference</p>",
      "rawMarkdown": "I’m getting f1 scores of ~.77-.78 on my validation set to get public LB of .478. I think it’s been mentioned before that class imbalances are contributing to this huge difference",
      "replies": [
        {
          "id": 425641,
          "postDate": "2018-11-21T22:12:00.667Z",
          "content": "<p>Are you computing macro F1 or micro F1 scores? I ask that because I am computing both and my macro is around .30 and my micro is around .70. This same result yields .40 in the public LB.</p>",
          "rawMarkdown": "Are you computing macro F1 or micro F1 scores? I ask that because I am computing both and my macro is around .30 and my micro is around .70. This same result yields .40 in the public LB.\n"
        },
        {
          "id": 425785,
          "postDate": "2018-11-22T05:17:12.953Z",
          "content": "<p>I'm using the implementation of fbeta (where in this case beta=1) in fastai here <a href=\"https://github.com/fastai/fastai/blob/master/fastai/metrics.py#L7\">https://github.com/fastai/fastai/blob/master/fastai/metrics.py#L7</a>, I'm pretty sure it's macro F1</p>",
          "rawMarkdown": "I'm using the implementation of fbeta (where in this case beta=1) in fastai here https://github.com/fastai/fastai/blob/master/fastai/metrics.py#L7, I'm pretty sure it's macro F1",
          "votes": 1
        },
        {
          "id": 426138,
          "postDate": "2018-11-22T17:49:59.010Z",
          "content": "<p>In the link you sent, what dim=1 refers to? Is it the samples axis or the labels axis?</p>\n\n<pre><code>TP = (y_pred*y_true).sum(dim=1)\nprec = TP/(y_pred.sum(dim=1)+eps)\nrec = TP/(y_true.sum(dim=1)+eps)\n</code></pre>\n\n<p>Macro F1 should be the non-weighted average of the F1 from each class. In keras, this would be dim=0. dim=1 would first compute each image's F1, then average them.\nBut I'm not familiar with fast.ai code, so there's a good chance I'm wrong. Please, let me know if I'm wrong.</p>",
          "rawMarkdown": "In the link you sent, what dim=1 refers to? Is it the samples axis or the labels axis?\n\n    TP = (y_pred*y_true).sum(dim=1)\n    prec = TP/(y_pred.sum(dim=1)+eps)\n    rec = TP/(y_true.sum(dim=1)+eps)\n\nMacro F1 should be the non-weighted average of the F1 from each class. In keras, this would be dim=0. dim=1 would first compute each image's F1, then average them.\nBut I'm not familiar with fast.ai code, so there's a good chance I'm wrong. Please, let me know if I'm wrong."
        },
        {
          "id": 446928,
          "postDate": "2018-12-28T22:03:40.807Z",
          "content": "<p>Agree about the dim in FastAI library, I did not see this discussion before,  I bought this up at <a href=\"https://forums.fast.ai/t/is-fbeta-metric-summing-along-the-correct-axis/32684\">FastAI forums</a>(with an example and comparing it to sklearn f1 macro call), nobody has replied yet. </p>",
          "rawMarkdown": "Agree about the dim in FastAI library, I did not see this discussion before,  I bought this up at [FastAI forums][1](with an example and comparing it to sklearn f1 macro call), nobody has replied yet. \n\n\n  [1]: https://forums.fast.ai/t/is-fbeta-metric-summing-along-the-correct-axis/32684"
        }
      ]
    },
    {
      "id": 425038,
      "postDate": "2018-11-21T03:39:54.720Z",
      "rawMarkdown": ""
    }
  ],
  "comments": [
    {
      "id": 425784,
      "author_name": "yelan",
      "author_url": "",
      "post_date": "2018-11-22T05:15:33.080000",
      "content": "<p>I search the threshold which get maximum f1 score on my validation, my f1 score on validation set about 0.74</p>",
      "votes": 2,
      "replies": [
        {
          "id": 427098,
          "author_name": "",
          "author_url": "",
          "post_date": "2018-11-24T15:06:02.913000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 425601,
      "author_name": "William Horton",
      "author_url": "",
      "post_date": "2018-11-21T20:42:13.977000",
      "content": "<p>I’m getting f1 scores of ~.77-.78 on my validation set to get public LB of .478. I think it’s been mentioned before that class imbalances are contributing to this huge difference</p>",
      "votes": 0,
      "replies": [
        {
          "id": 425641,
          "author_name": "FelipeKitamura, MD, PhD",
          "author_url": "",
          "post_date": "2018-11-21T22:12:00.667000",
          "content": "<p>Are you computing macro F1 or micro F1 scores? I ask that because I am computing both and my macro is around .30 and my micro is around .70. This same result yields .40 in the public LB.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 425785,
          "author_name": "William Horton",
          "author_url": "",
          "post_date": "2018-11-22T05:17:12.953000",
          "content": "<p>I'm using the implementation of fbeta (where in this case beta=1) in fastai here <a href=\"https://github.com/fastai/fastai/blob/master/fastai/metrics.py#L7\">https://github.com/fastai/fastai/blob/master/fastai/metrics.py#L7</a>, I'm pretty sure it's macro F1</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 426138,
          "author_name": "FelipeKitamura, MD, PhD",
          "author_url": "",
          "post_date": "2018-11-22T17:49:59.010000",
          "content": "<p>In the link you sent, what dim=1 refers to? Is it the samples axis or the labels axis?</p>\n\n<pre><code>TP = (y_pred*y_true).sum(dim=1)\nprec = TP/(y_pred.sum(dim=1)+eps)\nrec = TP/(y_true.sum(dim=1)+eps)\n</code></pre>\n\n<p>Macro F1 should be the non-weighted average of the F1 from each class. In keras, this would be dim=0. dim=1 would first compute each image's F1, then average them.\nBut I'm not familiar with fast.ai code, so there's a good chance I'm wrong. Please, let me know if I'm wrong.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 446928,
          "author_name": "Shiv Gowda",
          "author_url": "",
          "post_date": "2018-12-28T22:03:40.807000",
          "content": "<p>Agree about the dim in FastAI library, I did not see this discussion before,  I bought this up at <a href=\"https://forums.fast.ai/t/is-fbeta-metric-summing-along-the-correct-axis/32684\">FastAI forums</a>(with an example and comparing it to sklearn f1 macro call), nobody has replied yet. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "425784": "I search the threshold which get maximum f1 score on my validation, my f1 score on validation set about 0.74",
    "425601": "I’m getting f1 scores of ~.77-.78 on my validation set to get public LB of .478. I think it’s been mentioned before that class imbalances are contributing to this huge difference",
    "425038": ""
  }
}