{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"### F1 Score:\n\nF1 Score is harmonic mean of Precision and Recall.\n\n![F1 Score](https://svgshare.com/i/M7d.svg)\n","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"## Precision: \nIt tells you how weak is your model in predicting positive Classes. If number of False Positives are more than True Positives then your model has less precision.\n\n## Recall:\nIt tells you how weak is your model in covering True Classes. If number of False Negatives are more than True Positives then your model has less recall.\n\nFor more info on Precision and Recall, Please go through: https://towardsdatascience.com/multi-class-metrics-made-simple-part-i-precision-and-recall-9250280bddc2","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"*  F1-Score can be easily calculated for a binary classification Problem.\n*  What if we have multi-class classification ? How can we calculate F1-Score and if calculated for each label how can we merge / consolidate them ?","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's try to address these questions by considering an Example:\n\nWe have a dataset of 3 Labels Cat, Dog and Fish. We have built a multi-class classifier that predicts one label among these 3 labels and let's consider below as the the outputs for each Label. (One vs All)\n\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The above values can be represented as below confusion Matrix:\n\n![Confusion Matrix](https://i.ibb.co/swZrksj/Screenshot-from-2020-06-17-14-13-01.png)\n\nFrom the above table we can deduce the following for each Label:\n\nCat: 10 TP; 15 FP; 13 FN\n\nDog: 20 TP; 11 FP; 12 FN\n\nFish: 13 TP; 12 FP; 13 FN","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Now F-1 Score for each label can be calculated by calculating individual Precisions and Recalls by\n\nPrecision = T.P / (T.P + F.P)\nRecall = T.P / (T.P + F.N)\n\nEx: For Cat we have the following:\nPrecision = 10 / (10+15) = 0.4\nRecall = 10 / (10+13) = 0.43\n\nTherefore F1-Score for ** Cat is 0.414 **\n\nSimilarly F1-Scores for other labels are ** Dog 0.635, Fish 0.51 **","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We acheived F1 Scores for independent Labels. How can we aggregate these F1-Scores into a single F1-Score for evaluating the Model ?","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We have two type of Aggregations:\n\n**Macro Averaged F1-Score and Micro-Averaged F1-Score**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Macro Averaged F1-Score:\nHere we simple average all the F1-Scores and calculate a mean F1-Score.\n\nAverage of all the F1-Scores result in **0.52**\n\nBut simply weighing all the F1-Scores isn't a fair way to estimate a model perfomance. Because this metric doesn't perform good on imbalanced dataset. A good explaination on the same is provided below:\n\nhttps://datascience.stackexchange.com/questions/15989/micro-average-vs-macro-average-performance-in-a-multiclass-classification-settin","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Micro Averaged F1-Score:\n\nHere instead of calculating each label's F1-Score, we derive the F1-Score by calculating Precision and Recall by summing all the TPs and Type Errors instead of calculating for each Label:\n\nTotal T.Ps = 10+20+13 = 43\nTotal F.Ps = 15+11+12 = 38\nTotal F.Ns = 13+12+13 = 39\n\nPrecision = 43/(43+38) = 0.53\nRecall = 0.524\n\nF1-Score = 0.526\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}