```\nwe have CLASSES[0,1] melanoma or not \nthe problem is like i have accuracy with 0.9836 with valid_acc =0.9851 , but when i try to calculate score, precision, recall i got , score : 0.49, precision: 0.51, recall :0.50, which is incorrect ,so can any when tell me if there is any error in this code or if there is any other way to calculate score, precision, recall (to specify i want to calculate the sensitivity and specificity)
\n\nany help ll be very grateful & helpful thansk\n\n\ndef display_confusion_matrix(cmat, score, precision, recall):\n plt.figure(figsize=(15,15))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = \"\"\n if score is not None:\n titlestring += 'f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\nprecision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\nrecall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 18, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n```h
```\nwe have CLASSES[0,1] melanoma or not \nthe problem is like i have accuracy with 0.9836 with valid_acc =0.9851 , but when i try to calculate score, precision, recall i got , score : 0.49, precision: 0.51, recall :0.50, which is incorrect ,so can any when tell me if there is any error in this code or if there is any other way to calculate score, precision, recall (to specify i want to calculate the sensitivity and specificity)
\n\nany help ll be very grateful & helpful thansk\n\n\ndef display_confusion_matrix(cmat, score, precision, recall):\n plt.figure(figsize=(15,15))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = \"\"\n if score is not None:\n titlestring += 'f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\nprecision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\nrecall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 18, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n```h
The training set is very unbalanced with 584 melanoma images out of a total of 33126 images. If you classify all images as benign you will recieve an accuracy of 98.2% but the score will be 0.5. In this task it is very important to take care of this class imbalance. A simple test is to undersample your training set so you have 584 images in both classes and see what accuaracy you get (and score). If this works, try a better way of balancing the training set.
", "rawMarkdown": "The training set is very unbalanced with 584 melanoma images out of a total of 33126 images. If you classify all images as benign you will recieve an accuracy of 98.2% but the score will be 0.5. In this task it is very important to take care of this class imbalance. A simple test is to undersample your training set so you have 584 images in both classes and see what accuaracy you get (and score). If this works, try a better way of balancing the training set.", "votes": null }, { "id": "960738", "postDate": "08/06/2020 16:38:56", "content": "ok thanks brother
", "rawMarkdown": "ok thanks brother", "votes": null }, { "id": "964467", "postDate": "08/09/2020 22:56:54", "content": "sir how can i apply the undersampling or the over-sampling for the tfrec files ???
", "rawMarkdown": "sir how can i apply the undersampling or the over-sampling for the **tfrec** files ???", "votes": null }, { "id": "964797", "postDate": "08/10/2020 07:13:31", "content": "@tikoboss , there are many ways to try to take care of imbalance. If you had used PyTorch your files with lie in folders and it would have been super easy to test under-sampling. Now, when you are using tfrecords, I can't say from my experiance how to undersample. There are however other methods for taking care of imbalance which are easier to test together with tfrecords. However, you don't have many days left, so I would recommend that you take a look at some of the uploaded notebooks that are well written and take care of this. Some of the uploaded notebooks receive a LB of 0.95, which I think could be a good way for you to start experimenting. Some of these take care of imbalancing by using auc as a metric.
", "rawMarkdown": "tikoboss , there are many ways to try to take care of imbalance. If you had used PyTorch your files with lie in folders and it would have been super easy to test under-sampling. Now, when you are using tfrecords, I can't say from my experiance how to undersample. There are however other methods for taking care of imbalance which are easier to test together with tfrecords. However, you don't have many days left, so I would recommend that you take a look at some of the uploaded notebooks that are well written and take care of this. Some of the uploaded notebooks receive a LB of 0.95, which I think could be a good way for you to start experimenting. Some of these take care of imbalancing by using auc as a metric.", "votes": null } ], "comments": [ { "id": 960441, "author_name": "andersericssongnosco", "author_url": "", "post_date": "08/06/2020 12:06:33", "content": "The training set is very unbalanced with 584 melanoma images out of a total of 33126 images. If you classify all images as benign you will recieve an accuracy of 98.2% but the score will be 0.5. In this task it is very important to take care of this class imbalance. A simple test is to undersample your training set so you have 584 images in both classes and see what accuaracy you get (and score). If this works, try a better way of balancing the training set.
", "votes": null, "replies": [ { "id": 960738, "author_name": "tikoboss", "author_url": "", "post_date": "08/06/2020 16:38:56", "content": "ok thanks brother
", "votes": null, "replies": [] }, { "id": 964467, "author_name": "tikoboss", "author_url": "", "post_date": "08/09/2020 22:56:54", "content": "sir how can i apply the undersampling or the over-sampling for the tfrec files ???
", "votes": null, "replies": [] } ] }, { "id": 964797, "author_name": "andersericssongnosco", "author_url": "", "post_date": "08/10/2020 07:13:31", "content": "@tikoboss , there are many ways to try to take care of imbalance. If you had used PyTorch your files with lie in folders and it would have been super easy to test under-sampling. Now, when you are using tfrecords, I can't say from my experiance how to undersample. There are however other methods for taking care of imbalance which are easier to test together with tfrecords. However, you don't have many days left, so I would recommend that you take a look at some of the uploaded notebooks that are well written and take care of this. Some of the uploaded notebooks receive a LB of 0.95, which I think could be a good way for you to start experimenting. Some of these take care of imbalancing by using auc as a metric.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "960291": "```\nwe have CLASSES[0,1] melanoma or not \nthe problem is like i have accuracy with 0.9836 with valid_acc =0.9851 , but when i try to calculate score, precision, recall i got , score : 0.49, precision: 0.51, recall :0.50, which is incorrect ,so can any when tell me if there is any error in this code or if there is any other way to calculate score, precision, recall (**to specify i want to calculate the sensitivity and specificity**)\n\n**any help ll be very grateful & helpful thansk**\n```\n```\ndef display_confusion_matrix(cmat, score, precision, recall):\n plt.figure(figsize=(15,15))\n ax = plt.gca()\n ax.matshow(cmat, cmap='Reds')\n ax.set_xticks(range(len(CLASSES)))\n ax.set_xticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_xticklabels(), rotation=45, ha=\"left\", rotation_mode=\"anchor\")\n ax.set_yticks(range(len(CLASSES)))\n ax.set_yticklabels(CLASSES, fontdict={'fontsize': 7})\n plt.setp(ax.get_yticklabels(), rotation=45, ha=\"right\", rotation_mode=\"anchor\")\n titlestring = \"\"\n if score is not None:\n titlestring += 'f1 = {:.3f} '.format(score)\n if precision is not None:\n titlestring += '\\nprecision = {:.3f} '.format(precision)\n if recall is not None:\n titlestring += '\\nrecall = {:.3f} '.format(recall)\n if len(titlestring) > 0:\n ax.text(101, 1, titlestring, fontdict={'fontsize': 18, 'horizontalalignment':'right', 'verticalalignment':'top', 'color':'#804040'})\n plt.show()\n```h", "960441": "The training set is very unbalanced with 584 melanoma images out of a total of 33126 images. If you classify all images as benign you will recieve an accuracy of 98.2% but the score will be 0.5. In this task it is very important to take care of this class imbalance. A simple test is to undersample your training set so you have 584 images in both classes and see what accuaracy you get (and score). If this works, try a better way of balancing the training set.", "960738": "ok thanks brother", "964467": "sir how can i apply the undersampling or the over-sampling for the **tfrec** files ???", "964797": "tikoboss , there are many ways to try to take care of imbalance. If you had used PyTorch your files with lie in folders and it would have been super easy to test under-sampling. Now, when you are using tfrecords, I can't say from my experiance how to undersample. There are however other methods for taking care of imbalance which are easier to test together with tfrecords. However, you don't have many days left, so I would recommend that you take a look at some of the uploaded notebooks that are well written and take care of this. Some of the uploaded notebooks receive a LB of 0.95, which I think could be a good way for you to start experimenting. Some of these take care of imbalancing by using auc as a metric." }, "source": "meta" }