{
  "id": 81454,
  "title": "classification enhancements that improve results",
  "url": "/competitions/humpback-whale-identification/discussion/81454",
  "author_name": "",
  "post_date": "2019-02-21T16:27:18.554667500Z",
  "votes": 13,
  "comment_count": 5,
  "views": 0,
  "content": "<p>i would like to share some of my successful experiments on classification model</p>\n\n<p>1) convolution features as bag of word</p>\n\n<p>reference:</p>\n\n<ul>\n<li>\"Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet\"</li>\n</ul>\n\n<p><a href=\"https://openreview.net/forum?id=SkfMWhAqYQ\">https://openreview.net/forum?id=SkfMWhAqYQ</a></p>\n\n<p>-\"Bags of Local Convolutional Features for Scalable Instance Search\"</p>\n\n<p><a href=\"https://core.ac.uk/download/pdf/147609064.pdf\">https://core.ac.uk/download/pdf/147609064.pdf</a></p>\n\n<p>..</p>\n\n<p>results:  bag-of-local-feature-CNN is better than normal CNN by +0.01~0.02 public LB</p>\n\n<p>a densenet169 at 384x384 input without much train parameter tunning,etc will give you LB 0.81 to 0.82 using 5005 classification model (5004 id + 1 new-whale). single model without TTA, ensemble</p>\n\n<hr>\n\n<p>normal CNN (e.g. resnet) (each image is classified)</p>\n\n<p>input --&gt;[ encoder] --&gt;feature --&gt; ave pool --&gt;linear classifier ---&gt; softmax</p>\n\n<p>..</p>\n\n<p>bag-of-local-feature-CNN (each patch is classified into 5004 class)</p>\n\n<p>input --&gt;[ encoder] --&gt;feature --&gt; conv classifier  --&gt; ave pool  ---&gt; softmax</p>",
  "messages": [
    {
      "id": "476124",
      "postDate": "02/21/2019 16:27:18",
      "content": "<p>i would like to share some of my successful experiments on classification model</p>\n\n<p>1) convolution features as bag of word</p>\n\n<p>reference:</p>\n\n<ul>\n<li>\"Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet\"</li>\n</ul>\n\n<p><a href=\"https://openreview.net/forum?id=SkfMWhAqYQ\">https://openreview.net/forum?id=SkfMWhAqYQ</a></p>\n\n<p>-\"Bags of Local Convolutional Features for Scalable Instance Search\"</p>\n\n<p><a href=\"https://core.ac.uk/download/pdf/147609064.pdf\">https://core.ac.uk/download/pdf/147609064.pdf</a></p>\n\n<p>..</p>\n\n<p>results:  bag-of-local-feature-CNN is better than normal CNN by +0.01~0.02 public LB</p>\n\n<p>a densenet169 at 384x384 input without much train parameter tunning,etc will give you LB 0.81 to 0.82 using 5005 classification model (5004 id + 1 new-whale). single model without TTA, ensemble</p>\n\n<hr>\n\n<p>normal CNN (e.g. resnet) (each image is classified)</p>\n\n<p>input --&gt;[ encoder] --&gt;feature --&gt; ave pool --&gt;linear classifier ---&gt; softmax</p>\n\n<p>..</p>\n\n<p>bag-of-local-feature-CNN (each patch is classified into 5004 class)</p>\n\n<p>input --&gt;[ encoder] --&gt;feature --&gt; conv classifier  --&gt; ave pool  ---&gt; softmax</p>",
      "rawMarkdown": "i would like to share some of my successful experiments on classification model\n\n1) convolution features as bag of word\n\nreference:\n\n- \"Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet\"\n\nhttps://openreview.net/forum?id=SkfMWhAqYQ\n\n-\"Bags of Local Convolutional Features for Scalable Instance Search\"\n\nhttps://core.ac.uk/download/pdf/147609064.pdf\n\n\n..\n\nresults:  bag-of-local-feature-CNN is better than normal CNN by +0.01~0.02 public LB\n\na densenet169 at 384x384 input without much train parameter tunning,etc will give you LB 0.81 to 0.82 using 5005 classification model (5004 id + 1 new-whale). single model without TTA, ensemble\n\n---\n\n\nnormal CNN (e.g. resnet) (each image is classified)\n\ninput --&gt;[ encoder] --&gt;feature --&gt; ave pool --&gt;linear classifier ---&gt; softmax\n\n..\n\n\nbag-of-local-feature-CNN (each patch is classified into 5004 class)\n\ninput --&gt;[ encoder] --&gt;feature --&gt; conv classifier  --&gt; ave pool  ---&gt; softmax",
      "votes": null
    },
    {
      "id": "476129",
      "postDate": "02/21/2019 16:41:17",
      "content": "<p>Great idea!  I've tried BagNet in other classification task and it works pretty well.</p>",
      "rawMarkdown": "Great idea!  I've tried BagNet in other classification task and it works pretty well.",
      "votes": null
    },
    {
      "id": "476139",
      "postDate": "02/21/2019 16:50:04",
      "content": "<p>bagnet is supposed to learn useful patch </p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/476139/11371/1_afeo0XPiT1sPRRn_SYXEbw.jpeg\" alt=\"enter image description here\"></p>\n\n<p>my idea is:</p>\n\n<p>olden days computer vision:</p>\n\n<p>sift feature + knn clustering --&gt; visual bag of word --&gt;Gaussian mixture modeling</p>\n\n<p>..</p>\n\n<p>modern days of deep learning</p>\n\n<p>convolution features --&gt; center prototype --&gt; large margin Gaussian mixture loss</p>",
      "rawMarkdown": "bagnet is supposed to learn useful patch \n\n\n  ![enter image description here][1]\n\nmy idea is:\n\nolden days computer vision:\n\nsift feature + knn clustering --&gt; visual bag of word --&gt;Gaussian mixture modeling\n\n..\n\nmodern days of deep learning\n\nconvolution features --&gt; center prototype --&gt; large margin Gaussian mixture loss\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/476139/11371/1_afeo0XPiT1sPRRn_SYXEbw.jpeg",
      "votes": null
    },
    {
      "id": "478285",
      "postDate": "02/26/2019 01:13:19",
      "content": "<p>Did you use the pretrained weights (224x224 imagenet) when you trained the densenet169 with 384x384 input? Or did you retrain the whole network?</p>",
      "rawMarkdown": "Did you use the pretrained weights (224x224 imagenet) when you trained the densenet169 with 384x384 input? Or did you retrain the whole network?",
      "votes": null
    },
    {
      "id": "478309",
      "postDate": "02/26/2019 02:17:03",
      "content": "<p>use imagenet pretrain</p>",
      "rawMarkdown": "use imagenet pretrain",
      "votes": null
    },
    {
      "id": "480072",
      "postDate": "02/27/2019 18:48:57",
      "content": "<p>Heng, do you have any advice on training the bagnets? I've been working on training a bagnet33 from <a href=\"https://github.com/wielandbrendel/bag-of-local-features-models\">https://github.com/wielandbrendel/bag-of-local-features-models</a></p>\n\n<p>Valid map5 stalls around 0.87. At first I thought it was not learning at all, but after changing image size, adding mixup, retraining, repeat, its doing well but still cannot get past that 0.87.</p>",
      "rawMarkdown": "Heng, do you have any advice on training the bagnets? I've been working on training a bagnet33 from https://github.com/wielandbrendel/bag-of-local-features-models\n\nValid map5 stalls around 0.87. At first I thought it was not learning at all, but after changing image size, adding mixup, retraining, repeat, its doing well but still cannot get past that 0.87.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 476129,
      "author_name": "shentao",
      "author_url": "",
      "post_date": "02/21/2019 16:41:17",
      "content": "<p>Great idea!  I've tried BagNet in other classification task and it works pretty well.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 476139,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "02/21/2019 16:50:04",
      "content": "<p>bagnet is supposed to learn useful patch </p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/476139/11371/1_afeo0XPiT1sPRRn_SYXEbw.jpeg\" alt=\"enter image description here\"></p>\n\n<p>my idea is:</p>\n\n<p>olden days computer vision:</p>\n\n<p>sift feature + knn clustering --&gt; visual bag of word --&gt;Gaussian mixture modeling</p>\n\n<p>..</p>\n\n<p>modern days of deep learning</p>\n\n<p>convolution features --&gt; center prototype --&gt; large margin Gaussian mixture loss</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 478285,
      "author_name": "alamjs",
      "author_url": "",
      "post_date": "02/26/2019 01:13:19",
      "content": "<p>Did you use the pretrained weights (224x224 imagenet) when you trained the densenet169 with 384x384 input? Or did you retrain the whole network?</p>",
      "votes": null,
      "replies": [
        {
          "id": 478309,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "02/26/2019 02:17:03",
          "content": "<p>use imagenet pretrain</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 480072,
      "author_name": "interneuron",
      "author_url": "",
      "post_date": "02/27/2019 18:48:57",
      "content": "<p>Heng, do you have any advice on training the bagnets? I've been working on training a bagnet33 from <a href=\"https://github.com/wielandbrendel/bag-of-local-features-models\">https://github.com/wielandbrendel/bag-of-local-features-models</a></p>\n\n<p>Valid map5 stalls around 0.87. At first I thought it was not learning at all, but after changing image size, adding mixup, retraining, repeat, its doing well but still cannot get past that 0.87.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "476124": "i would like to share some of my successful experiments on classification model\n\n1) convolution features as bag of word\n\nreference:\n\n- \"Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet\"\n\nhttps://openreview.net/forum?id=SkfMWhAqYQ\n\n-\"Bags of Local Convolutional Features for Scalable Instance Search\"\n\nhttps://core.ac.uk/download/pdf/147609064.pdf\n\n\n..\n\nresults:  bag-of-local-feature-CNN is better than normal CNN by +0.01~0.02 public LB\n\na densenet169 at 384x384 input without much train parameter tunning,etc will give you LB 0.81 to 0.82 using 5005 classification model (5004 id + 1 new-whale). single model without TTA, ensemble\n\n---\n\n\nnormal CNN (e.g. resnet) (each image is classified)\n\ninput --&gt;[ encoder] --&gt;feature --&gt; ave pool --&gt;linear classifier ---&gt; softmax\n\n..\n\n\nbag-of-local-feature-CNN (each patch is classified into 5004 class)\n\ninput --&gt;[ encoder] --&gt;feature --&gt; conv classifier  --&gt; ave pool  ---&gt; softmax",
    "476129": "Great idea!  I've tried BagNet in other classification task and it works pretty well.",
    "476139": "bagnet is supposed to learn useful patch \n\n\n  ![enter image description here][1]\n\nmy idea is:\n\nolden days computer vision:\n\nsift feature + knn clustering --&gt; visual bag of word --&gt;Gaussian mixture modeling\n\n..\n\nmodern days of deep learning\n\nconvolution features --&gt; center prototype --&gt; large margin Gaussian mixture loss\n\n\n  [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/476139/11371/1_afeo0XPiT1sPRRn_SYXEbw.jpeg",
    "478285": "Did you use the pretrained weights (224x224 imagenet) when you trained the densenet169 with 384x384 input? Or did you retrain the whole network?",
    "478309": "use imagenet pretrain",
    "480072": "Heng, do you have any advice on training the bagnets? I've been working on training a bagnet33 from https://github.com/wielandbrendel/bag-of-local-features-models\n\nValid map5 stalls around 0.87. At first I thought it was not learning at all, but after changing image size, adding mixup, retraining, repeat, its doing well but still cannot get past that 0.87."
  },
  "source": "meta"
}