{
  "id": 80318,
  "title": "how do you handle a “none of these” class in a CNN - softamax, sigmoid, openmax",
  "url": "/competitions/humpback-whale-identification/discussion/80318",
  "author_name": "",
  "post_date": "2019-02-12T16:05:09.474074500Z",
  "votes": 13,
  "comment_count": 1,
  "views": 0,
  "content": "<p>we have an open-set image classification task in this challenge. How do you handle the \"new whale\" class (aka “none of these” class)</p>\n\n<p>We have 5004 whale id</p>\n\n<hr>\n\n<p>A. Dustbin Class</p>\n\n<p>Simply treat \"new whale\" as a new class, use softmax on 5004+1 class</p>\n\n<p>B. Use sigmoid + softmax</p>\n\n<p>use softmax on 5004 class. Then train another 5004 sigmoid classifier for using \"one vs all\". At inference: </p>\n\n<ul>\n<li><p>use softmax-classifier to determine candidate class :  c = argmax(soft_<strong>p1,  soft</strong><em>p2 ... soft</em>_p5004)</p></li>\n<li><p>select c-th sigmoid-classifier to see if it is a new whale or not:  sigmoid__pc(x) &gt; 0.5?</p></li>\n</ul>\n\n<p>( alternatively, use one universal sigmoid classifier to determine it is a new whale or not. if it is, use softmax to determine its id)</p>\n\n<hr>\n\n<p>C. there is also the method of openmax</p>\n\n<p>see: <br>\n“Towards Open Set Deep Networks,” CVPR 2016.\n<a href=\"http://vast.uccs.edu/~abendale/\">http://vast.uccs.edu/~abendale/</a></p>\n\n<p>\" The closed set nature of deep networks forces them to choose from one of\nthe known classes leading to such artifacts. Recognition in the real world is open set, i.e. the recognition system should reject unknown/unseen classes at test time. We present a\nmethodology to adapt deep networks for open set recognition, by introducing a new model layer, OpenMax, which estimates the probability of an input being from an unknown class.\"</p>\n\n<p>see also:</p>\n\n<p><a href=\"https://github.com/PeiqinZhuang/WildFish\">https://github.com/PeiqinZhuang/WildFish</a></p>\n\n<p>1) Open-Set Fish Classification:</p>\n\n<p><img src=\"https://github.com/PeiqinZhuang/WildFish/raw/master/paper/OpenSet_Framework.png\" alt=\"enter image description here\"></p>\n\n<p>2)Fine-grained Fish Recognition with Pairwise Text Descriptions</p>\n\n<p><img src=\"https://github.com/PeiqinZhuang/WildFish/raw/master/paper/pairwise.png\" alt=\"enter image description here\"></p>\n\n<p>other reference:</p>\n\n<p>\"LEARNING A NEURAL-NETWORK-BASED REPRESENTATION\nFOR OPEN SET RECOGNITION\" -  ICLR 2019</p>\n\n<p>\"Universum Prescription: Regularization Using Unlabeled Data\"\n<a href=\"https://arxiv.org/pdf/1511.03719.pdf\">https://arxiv.org/pdf/1511.03719.pdf</a></p>\n\n<p><a href=\"https://www.wjscheirer.com/projects/openset-recognition/\">https://www.wjscheirer.com/projects/openset-recognition/</a></p>\n\n<hr>\n\n<p>Note: the above is for classification-based approach. if you are using metric learning based approach, you do not need these.</p>",
  "messages": [
    {
      "id": "470225",
      "postDate": "02/12/2019 16:05:09",
      "content": "<p>we have an open-set image classification task in this challenge. How do you handle the \"new whale\" class (aka “none of these” class)</p>\n\n<p>We have 5004 whale id</p>\n\n<hr>\n\n<p>A. Dustbin Class</p>\n\n<p>Simply treat \"new whale\" as a new class, use softmax on 5004+1 class</p>\n\n<p>B. Use sigmoid + softmax</p>\n\n<p>use softmax on 5004 class. Then train another 5004 sigmoid classifier for using \"one vs all\". At inference: </p>\n\n<ul>\n<li><p>use softmax-classifier to determine candidate class :  c = argmax(soft_<strong>p1,  soft</strong><em>p2 ... soft</em>_p5004)</p></li>\n<li><p>select c-th sigmoid-classifier to see if it is a new whale or not:  sigmoid__pc(x) &gt; 0.5?</p></li>\n</ul>\n\n<p>( alternatively, use one universal sigmoid classifier to determine it is a new whale or not. if it is, use softmax to determine its id)</p>\n\n<hr>\n\n<p>C. there is also the method of openmax</p>\n\n<p>see: <br>\n“Towards Open Set Deep Networks,” CVPR 2016.\n<a href=\"http://vast.uccs.edu/~abendale/\">http://vast.uccs.edu/~abendale/</a></p>\n\n<p>\" The closed set nature of deep networks forces them to choose from one of\nthe known classes leading to such artifacts. Recognition in the real world is open set, i.e. the recognition system should reject unknown/unseen classes at test time. We present a\nmethodology to adapt deep networks for open set recognition, by introducing a new model layer, OpenMax, which estimates the probability of an input being from an unknown class.\"</p>\n\n<p>see also:</p>\n\n<p><a href=\"https://github.com/PeiqinZhuang/WildFish\">https://github.com/PeiqinZhuang/WildFish</a></p>\n\n<p>1) Open-Set Fish Classification:</p>\n\n<p><img src=\"https://github.com/PeiqinZhuang/WildFish/raw/master/paper/OpenSet_Framework.png\" alt=\"enter image description here\"></p>\n\n<p>2)Fine-grained Fish Recognition with Pairwise Text Descriptions</p>\n\n<p><img src=\"https://github.com/PeiqinZhuang/WildFish/raw/master/paper/pairwise.png\" alt=\"enter image description here\"></p>\n\n<p>other reference:</p>\n\n<p>\"LEARNING A NEURAL-NETWORK-BASED REPRESENTATION\nFOR OPEN SET RECOGNITION\" -  ICLR 2019</p>\n\n<p>\"Universum Prescription: Regularization Using Unlabeled Data\"\n<a href=\"https://arxiv.org/pdf/1511.03719.pdf\">https://arxiv.org/pdf/1511.03719.pdf</a></p>\n\n<p><a href=\"https://www.wjscheirer.com/projects/openset-recognition/\">https://www.wjscheirer.com/projects/openset-recognition/</a></p>\n\n<hr>\n\n<p>Note: the above is for classification-based approach. if you are using metric learning based approach, you do not need these.</p>",
      "rawMarkdown": "we have an open-set image classification task in this challenge. How do you handle the \"new whale\" class (aka “none of these” class)\n\nWe have 5004 whale id\n\n---\n\nA. Dustbin Class\n\nSimply treat \"new whale\" as a new class, use softmax on 5004+1 class\n\n\nB. Use sigmoid + softmax\n\nuse softmax on 5004 class. Then train another 5004 sigmoid classifier for using \"one vs all\". At inference: \n\n- use softmax-classifier to determine candidate class :  c = argmax(soft___p1,  soft___p2 ... soft__p5004)\n\n- select c-th sigmoid-classifier to see if it is a new whale or not:  sigmoid__pc(x) &gt; 0.5?\n\n\n( alternatively, use one universal sigmoid classifier to determine it is a new whale or not. if it is, use softmax to determine its id)\n\n---\n\nC. there is also the method of openmax\n\nsee:  \n“Towards Open Set Deep Networks,” CVPR 2016.\nhttp://vast.uccs.edu/~abendale/\n\n\" The closed set nature of deep networks forces them to choose from one of\nthe known classes leading to such artifacts. Recognition in the real world is open set, i.e. the recognition system should reject unknown/unseen classes at test time. We present a\nmethodology to adapt deep networks for open set recognition, by introducing a new model layer, OpenMax, which estimates the probability of an input being from an unknown class.\"\n\nsee also:\n\nhttps://github.com/PeiqinZhuang/WildFish\n\n\n1) Open-Set Fish Classification:\n\n   ![enter image description here][1]\n\n2)Fine-grained Fish Recognition with Pairwise Text Descriptions\n\n   ![enter image description here][2]\n\nother reference:\n\n\"LEARNING A NEURAL-NETWORK-BASED REPRESENTATION\nFOR OPEN SET RECOGNITION\" -  ICLR 2019\n\n\"Universum Prescription: Regularization Using Unlabeled Data\"\nhttps://arxiv.org/pdf/1511.03719.pdf\n\nhttps://www.wjscheirer.com/projects/openset-recognition/\n\n---\nNote: the above is for classification-based approach. if you are using metric learning based approach, you do not need these.\n\n\n  [1]: https://github.com/PeiqinZhuang/WildFish/raw/master/paper/OpenSet_Framework.png\n  [2]: https://github.com/PeiqinZhuang/WildFish/raw/master/paper/pairwise.png",
      "votes": null
    },
    {
      "id": "470253",
      "postDate": "02/12/2019 16:45:12",
      "content": "<p>performance of softmax vs openmax in rejecting image of class not in training (aka unknown class)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/470253/11267/openmax.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "performance of softmax vs openmax in rejecting image of class not in training (aka unknown class)\n\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/470253/11267/openmax.png",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 470253,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/12/2019 16:45:12",
      "content": "<p>performance of softmax vs openmax in rejecting image of class not in training (aka unknown class)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/470253/11267/openmax.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "470225": "we have an open-set image classification task in this challenge. How do you handle the \"new whale\" class (aka “none of these” class)\n\nWe have 5004 whale id\n\n---\n\nA. Dustbin Class\n\nSimply treat \"new whale\" as a new class, use softmax on 5004+1 class\n\n\nB. Use sigmoid + softmax\n\nuse softmax on 5004 class. Then train another 5004 sigmoid classifier for using \"one vs all\". At inference: \n\n- use softmax-classifier to determine candidate class :  c = argmax(soft___p1,  soft___p2 ... soft__p5004)\n\n- select c-th sigmoid-classifier to see if it is a new whale or not:  sigmoid__pc(x) &gt; 0.5?\n\n\n( alternatively, use one universal sigmoid classifier to determine it is a new whale or not. if it is, use softmax to determine its id)\n\n---\n\nC. there is also the method of openmax\n\nsee:  \n“Towards Open Set Deep Networks,” CVPR 2016.\nhttp://vast.uccs.edu/~abendale/\n\n\" The closed set nature of deep networks forces them to choose from one of\nthe known classes leading to such artifacts. Recognition in the real world is open set, i.e. the recognition system should reject unknown/unseen classes at test time. We present a\nmethodology to adapt deep networks for open set recognition, by introducing a new model layer, OpenMax, which estimates the probability of an input being from an unknown class.\"\n\nsee also:\n\nhttps://github.com/PeiqinZhuang/WildFish\n\n\n1) Open-Set Fish Classification:\n\n   ![enter image description here][1]\n\n2)Fine-grained Fish Recognition with Pairwise Text Descriptions\n\n   ![enter image description here][2]\n\nother reference:\n\n\"LEARNING A NEURAL-NETWORK-BASED REPRESENTATION\nFOR OPEN SET RECOGNITION\" -  ICLR 2019\n\n\"Universum Prescription: Regularization Using Unlabeled Data\"\nhttps://arxiv.org/pdf/1511.03719.pdf\n\nhttps://www.wjscheirer.com/projects/openset-recognition/\n\n---\nNote: the above is for classification-based approach. if you are using metric learning based approach, you do not need these.\n\n\n  [1]: https://github.com/PeiqinZhuang/WildFish/raw/master/paper/OpenSet_Framework.png\n  [2]: https://github.com/PeiqinZhuang/WildFish/raw/master/paper/pairwise.png",
    "470253": "performance of softmax vs openmax in rejecting image of class not in training (aka unknown class)\n\n  ![enter image description here][1]\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/470253/11267/openmax.png"
  },
  "source": "meta"
}