Hi all, I was wondering, since the competition is less than half way from finish line, a lot of competitors might have the understanding about the competition metric, it will be very helpful if anyone share some insights about how much the competition metric is deviated from F1 score (I used basic predictions like if prob > 0.5 then 1 and if < 0.5 then 0).
\nIn my understanding, all the tricks lies in how to put the probabilities on 0s and 1s so that after sorting probabilities it gives best result which I can not relate with F1 score. So the ask!
", "messages": [ { "id": 1862889, "postDate": "2022-07-20T04:35:14.133Z", "content": "Hi all, I was wondering, since the competition is less than half way from finish line, a lot of competitors might have the understanding about the competition metric, it will be very helpful if anyone share some insights about how much the competition metric is deviated from F1 score (I used basic predictions like if prob > 0.5 then 1 and if < 0.5 then 0).
\nIn my understanding, all the tricks lies in how to put the probabilities on 0s and 1s so that after sorting probabilities it gives best result which I can not relate with F1 score. So the ask!
", "rawMarkdown": "Hi all, I was wondering, since the competition is less than half way from finish line, a lot of competitors might have the understanding about the competition metric, it will be very helpful if anyone share some insights about how much the competition metric is deviated from F1 score (I used basic predictions like if prob > 0.5 then 1 and if < 0.5 then 0).\n\nIn my understanding, all the tricks lies in how to put the probabilities on 0s and 1s so that after sorting probabilities it gives best result which I can not relate with F1 score. So the ask!", "votes": 1 }, { "id": 1868125, "postDate": "2022-07-23T18:12:23.340Z", "content": "F1 score is usually a weird metric… there is two kinds of metric: differentiable and non differentiable. Those that are differentiable are usually more interesting because they can be used to tune models. So non differentiable metrics aren’t much used unless they are really business oriented. F1 score is not differentiable and not really business oriented (arithmetic geometric mean of precision and recall doesn’t easily translate to $). More specifically, F1 score needs a threshold (or be optimised over all possible thresholds) which doesn’t occur here.
", "rawMarkdown": "F1 score is usually a weird metric... there is two kinds of metric: differentiable and non differentiable. Those that are differentiable are usually more interesting because they can be used to tune models. So non differentiable metrics aren’t much used unless they are really business oriented. F1 score is not differentiable and not really business oriented (arithmetic geometric mean of precision and recall doesn’t easily translate to $). More specifically, F1 score needs a threshold (or be optimised over all possible thresholds) which doesn’t occur here." }, { "id": 1867440, "postDate": "2022-07-23T08:03:53.060Z", "content": "Hi Tamique de Brito, I could not find reply button on your comment so I am writing this comment:
\nI wasn't intend to use 0's and 1's in submission file, I was using that to check how F1 score is working compare to amex metric. That's all.
", "rawMarkdown": "Hi Tamique de Brito, I could not find reply button on your comment so I am writing this comment:\n\nI wasn't intend to use 0's and 1's in submission file, I was using that to check how F1 score is working compare to amex metric. That's all.", "replies": [ { "id": 1868020, "postDate": "2022-07-23T16:26:03.550Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 1866654, "postDate": "2022-07-22T16:20:53.980Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 1866915, "postDate": "2022-07-22T20:24:37.733Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 1868125, "author_name": "Lucas Morin", "author_url": "", "post_date": "2022-07-23T18:12:23.340000", "content": "F1 score is usually a weird metric… there is two kinds of metric: differentiable and non differentiable. Those that are differentiable are usually more interesting because they can be used to tune models. So non differentiable metrics aren’t much used unless they are really business oriented. F1 score is not differentiable and not really business oriented (arithmetic geometric mean of precision and recall doesn’t easily translate to $). More specifically, F1 score needs a threshold (or be optimised over all possible thresholds) which doesn’t occur here.
", "votes": 0, "replies": [] }, { "id": 1867440, "author_name": "Youktinath Bhowmick", "author_url": "", "post_date": "2022-07-23T08:03:53.060000", "content": "Hi Tamique de Brito, I could not find reply button on your comment so I am writing this comment:
\nI wasn't intend to use 0's and 1's in submission file, I was using that to check how F1 score is working compare to amex metric. That's all.
", "votes": 0, "replies": [ { "id": 1868020, "author_name": "", "author_url": "", "post_date": "2022-07-23T16:26:03.550000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 1866654, "author_name": "", "author_url": "", "post_date": "2022-07-22T16:20:53.980000", "content": "", "votes": 0, "replies": [ { "id": 1866915, "author_name": "", "author_url": "", "post_date": "2022-07-22T20:24:37.733000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1862889": "Hi all, I was wondering, since the competition is less than half way from finish line, a lot of competitors might have the understanding about the competition metric, it will be very helpful if anyone share some insights about how much the competition metric is deviated from F1 score (I used basic predictions like if prob > 0.5 then 1 and if < 0.5 then 0).\n\nIn my understanding, all the tricks lies in how to put the probabilities on 0s and 1s so that after sorting probabilities it gives best result which I can not relate with F1 score. So the ask!", "1868125": "F1 score is usually a weird metric... there is two kinds of metric: differentiable and non differentiable. Those that are differentiable are usually more interesting because they can be used to tune models. So non differentiable metrics aren’t much used unless they are really business oriented. F1 score is not differentiable and not really business oriented (arithmetic geometric mean of precision and recall doesn’t easily translate to $). More specifically, F1 score needs a threshold (or be optimised over all possible thresholds) which doesn’t occur here.", "1867440": "Hi Tamique de Brito, I could not find reply button on your comment so I am writing this comment:\n\nI wasn't intend to use 0's and 1's in submission file, I was using that to check how F1 score is working compare to amex metric. That's all.", "1866654": "" } }