{
  "id": 41632,
  "title": "any one trying multiple classifier approach?",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/41632",
  "author_name": "",
  "post_date": "2017-10-21T16:39:51.058229700Z",
  "votes": 2,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Given 5270 classes, i create subsets S1, S2 ... Sn, ....SN. These subsets can be overlapping. The subset can be randomly created, or based on class distribution,etc</p>\n\n<p>for example, Sn = { background, class0, class5, class5000, .... }</p>\n\n<p>for each subset, i train a classifier Cn.</p>\n\n<p>At prediction, given a test image, N classifiers will produce N set of scores. then i can either:</p>\n\n<ol>\n<li><p>choose the label that give highest score (ignore background)</p></li>\n<li><p>build another classifier/deep network to make a prediction based on these set of scores.</p></li>\n</ol>\n\n<p>[ why doing this? ]</p>\n\n<p>Say you have several small gpus or you are in a team with several members, you can split the big classification problem into smaller ones. Then you can combine the predictions into a stronger one.</p>",
  "messages": [
    {
      "id": "233873",
      "postDate": "10/21/2017 16:39:51",
      "content": "<p>Given 5270 classes, i create subsets S1, S2 ... Sn, ....SN. These subsets can be overlapping. The subset can be randomly created, or based on class distribution,etc</p>\n\n<p>for example, Sn = { background, class0, class5, class5000, .... }</p>\n\n<p>for each subset, i train a classifier Cn.</p>\n\n<p>At prediction, given a test image, N classifiers will produce N set of scores. then i can either:</p>\n\n<ol>\n<li><p>choose the label that give highest score (ignore background)</p></li>\n<li><p>build another classifier/deep network to make a prediction based on these set of scores.</p></li>\n</ol>\n\n<p>[ why doing this? ]</p>\n\n<p>Say you have several small gpus or you are in a team with several members, you can split the big classification problem into smaller ones. Then you can combine the predictions into a stronger one.</p>",
      "rawMarkdown": "Given 5270 classes, i create subsets S1, S2 ... Sn, ....SN. These subsets can be overlapping. The subset can be randomly created, or based on class distribution,etc\n\nfor example, Sn = { background, class0, class5, class5000, .... }\n\nfor each subset, i train a classifier Cn.\n\nAt prediction, given a test image, N classifiers will produce N set of scores. then i can either:\n\n1. choose the label that give highest score (ignore background)\n\n2. build another classifier/deep network to make a prediction based on these set of scores.\n\n[ why doing this? ]\n\nSay you have several small gpus or you are in a team with several members, you can split the big classification problem into smaller ones. Then you can combine the predictions into a stronger one.",
      "votes": null
    },
    {
      "id": "233899",
      "postDate": "10/21/2017 18:02:44",
      "content": "<p>An interesting idea would be to split subsets (S1, S2, ...) based on the hierarchical structure of categories.  E.g. there are family categories defining a subtree for sports, fashion, technology etc. What do you think? The intuition is, that this way more specific classifiers may be able to identify more domain specific features in images such as gender in fashion etc.</p>",
      "rawMarkdown": "An interesting idea would be to split subsets (S1, S2, ...) based on the hierarchical structure of categories.  E.g. there are family categories defining a subtree for sports, fashion, technology etc. What do you think? The intuition is, that this way more specific classifiers may be able to identify more domain specific features in images such as gender in fashion etc.",
      "votes": null
    },
    {
      "id": "234212",
      "postDate": "10/22/2017 17:08:05",
      "content": "<p>I'm planning something like that, probably splits based on number of images in a category, to have more balanced categories. A hierarchical loss layer is another possibility, that would take into account the hierarchies too.\nSince I have limited resources, 6GB 1060, and want to go green, this time, by not training for days on it :) I'm trying out all kind of non-traditional (take big net and fine-tune it) approaches. I did not submit to public LB as my val scores are not stellar, yet :D</p>",
      "rawMarkdown": "I'm planning something like that, probably splits based on number of images in a category, to have more balanced categories. A hierarchical loss layer is another possibility, that would take into account the hierarchies too.\nSince I have limited resources, 6GB 1060, and want to go green, this time, by not training for days on it :) I'm trying out all kind of non-traditional (take big net and fine-tune it) approaches. I did not submit to public LB as my val scores are not stellar, yet :D",
      "votes": null
    },
    {
      "id": "236009",
      "postDate": "10/26/2017 14:07:26",
      "content": "<p>I am trying a different approach by combine the softmax of multiple model together to have an ensemble. I have seen a couple of papers about that but with 5270 classes and N models, the fitting is taking forever and the results are most of the time worst than the best model. Any idea/ model / link to help ? \nPS: base models are from different architecture\n@Heng CherKeng: Thanks a lot for your comments/help in the forum, it is very helpful !!!</p>",
      "rawMarkdown": "I am trying a different approach by combine the softmax of multiple model together to have an ensemble. I have seen a couple of papers about that but with 5270 classes and N models, the fitting is taking forever and the results are most of the time worst than the best model. Any idea/ model / link to help ? \nPS: base models are from different architecture\n@Heng CherKeng: Thanks a lot for your comments/help in the forum, it is very helpful !!!",
      "votes": null
    },
    {
      "id": "236241",
      "postDate": "10/26/2017 21:40:42",
      "content": "<p>you could combine the features (e.g. the last feature map that is input to the fc classifier) instead of the softmax probability, and build a classifier on the combined features. you can also use multiple feature maps per model, instead of the last feature map.</p>",
      "rawMarkdown": "you could combine the features (e.g. the last feature map that is input to the fc classifier) instead of the softmax probability, and build a classifier on the combined features. you can also use multiple feature maps per model, instead of the last feature map.",
      "votes": null
    },
    {
      "id": "237138",
      "postDate": "10/29/2017 15:24:28",
      "content": "<p>I tried building multiple classifiers but without much success. I made 5 classifiers that each did 1054 categories plus 1 \"other\". The training and validation accuracy for each individual classifier were great, since predicting \"other\" is often a good choice. However, if all 5 classifiers predict \"other\", then that is not a very useful result. I tried different things here, but everything was worse than just using one big classifier. However, I like the idea of training another classifier on top that will decide which of the 5 classifiers to use for the final prediction.</p>",
      "rawMarkdown": "I tried building multiple classifiers but without much success. I made 5 classifiers that each did 1054 categories plus 1 \"other\". The training and validation accuracy for each individual classifier were great, since predicting \"other\" is often a good choice. However, if all 5 classifiers predict \"other\", then that is not a very useful result. I tried different things here, but everything was worse than just using one big classifier. However, I like the idea of training another classifier on top that will decide which of the 5 classifiers to use for the final prediction.",
      "votes": null
    },
    {
      "id": "240930",
      "postDate": "11/07/2017 16:51:23",
      "content": "<p>Do you balance this 5 classifier i.e. first block most popular classes, or its just random classes in block?\nDo you try to create classifier with intersections?</p>",
      "rawMarkdown": "Do you balance this 5 classifier i.e. first block most popular classes, or its just random classes in block?\nDo you try to create classifier with intersections?",
      "votes": null
    },
    {
      "id": "241204",
      "postDate": "11/08/2017 09:31:19",
      "content": "<p>I grouped by most popular classes. I also tried different group sizes (so some classifiers only had 100 classes but with many images, and other classifiers had 1000 classes but smaller ones).</p>",
      "rawMarkdown": "I grouped by most popular classes. I also tried different group sizes (so some classifiers only had 100 classes but with many images, and other classifiers had 1000 classes but smaller ones).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 233899,
      "author_name": "andreaslup",
      "author_url": "",
      "post_date": "10/21/2017 18:02:44",
      "content": "<p>An interesting idea would be to split subsets (S1, S2, ...) based on the hierarchical structure of categories.  E.g. there are family categories defining a subtree for sports, fashion, technology etc. What do you think? The intuition is, that this way more specific classifiers may be able to identify more domain specific features in images such as gender in fashion etc.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 234212,
      "author_name": "vzsolt",
      "author_url": "",
      "post_date": "10/22/2017 17:08:05",
      "content": "<p>I'm planning something like that, probably splits based on number of images in a category, to have more balanced categories. A hierarchical loss layer is another possibility, that would take into account the hierarchies too.\nSince I have limited resources, 6GB 1060, and want to go green, this time, by not training for days on it :) I'm trying out all kind of non-traditional (take big net and fine-tune it) approaches. I did not submit to public LB as my val scores are not stellar, yet :D</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 236009,
      "author_name": "elberi",
      "author_url": "",
      "post_date": "10/26/2017 14:07:26",
      "content": "<p>I am trying a different approach by combine the softmax of multiple model together to have an ensemble. I have seen a couple of papers about that but with 5270 classes and N models, the fitting is taking forever and the results are most of the time worst than the best model. Any idea/ model / link to help ? \nPS: base models are from different architecture\n@Heng CherKeng: Thanks a lot for your comments/help in the forum, it is very helpful !!!</p>",
      "votes": null,
      "replies": [
        {
          "id": 236241,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "10/26/2017 21:40:42",
          "content": "<p>you could combine the features (e.g. the last feature map that is input to the fc classifier) instead of the softmax probability, and build a classifier on the combined features. you can also use multiple feature maps per model, instead of the last feature map.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 237138,
      "author_name": "humananalog",
      "author_url": "",
      "post_date": "10/29/2017 15:24:28",
      "content": "<p>I tried building multiple classifiers but without much success. I made 5 classifiers that each did 1054 categories plus 1 \"other\". The training and validation accuracy for each individual classifier were great, since predicting \"other\" is often a good choice. However, if all 5 classifiers predict \"other\", then that is not a very useful result. I tried different things here, but everything was worse than just using one big classifier. However, I like the idea of training another classifier on top that will decide which of the 5 classifiers to use for the final prediction.</p>",
      "votes": null,
      "replies": [
        {
          "id": 240930,
          "author_name": "nicksergievskiy",
          "author_url": "",
          "post_date": "11/07/2017 16:51:23",
          "content": "<p>Do you balance this 5 classifier i.e. first block most popular classes, or its just random classes in block?\nDo you try to create classifier with intersections?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 241204,
          "author_name": "humananalog",
          "author_url": "",
          "post_date": "11/08/2017 09:31:19",
          "content": "<p>I grouped by most popular classes. I also tried different group sizes (so some classifiers only had 100 classes but with many images, and other classifiers had 1000 classes but smaller ones).</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "233873": "Given 5270 classes, i create subsets S1, S2 ... Sn, ....SN. These subsets can be overlapping. The subset can be randomly created, or based on class distribution,etc\n\nfor example, Sn = { background, class0, class5, class5000, .... }\n\nfor each subset, i train a classifier Cn.\n\nAt prediction, given a test image, N classifiers will produce N set of scores. then i can either:\n\n1. choose the label that give highest score (ignore background)\n\n2. build another classifier/deep network to make a prediction based on these set of scores.\n\n[ why doing this? ]\n\nSay you have several small gpus or you are in a team with several members, you can split the big classification problem into smaller ones. Then you can combine the predictions into a stronger one.",
    "233899": "An interesting idea would be to split subsets (S1, S2, ...) based on the hierarchical structure of categories.  E.g. there are family categories defining a subtree for sports, fashion, technology etc. What do you think? The intuition is, that this way more specific classifiers may be able to identify more domain specific features in images such as gender in fashion etc.",
    "234212": "I'm planning something like that, probably splits based on number of images in a category, to have more balanced categories. A hierarchical loss layer is another possibility, that would take into account the hierarchies too.\nSince I have limited resources, 6GB 1060, and want to go green, this time, by not training for days on it :) I'm trying out all kind of non-traditional (take big net and fine-tune it) approaches. I did not submit to public LB as my val scores are not stellar, yet :D",
    "236009": "I am trying a different approach by combine the softmax of multiple model together to have an ensemble. I have seen a couple of papers about that but with 5270 classes and N models, the fitting is taking forever and the results are most of the time worst than the best model. Any idea/ model / link to help ? \nPS: base models are from different architecture\n@Heng CherKeng: Thanks a lot for your comments/help in the forum, it is very helpful !!!",
    "236241": "you could combine the features (e.g. the last feature map that is input to the fc classifier) instead of the softmax probability, and build a classifier on the combined features. you can also use multiple feature maps per model, instead of the last feature map.",
    "237138": "I tried building multiple classifiers but without much success. I made 5 classifiers that each did 1054 categories plus 1 \"other\". The training and validation accuracy for each individual classifier were great, since predicting \"other\" is often a good choice. However, if all 5 classifiers predict \"other\", then that is not a very useful result. I tried different things here, but everything was worse than just using one big classifier. However, I like the idea of training another classifier on top that will decide which of the 5 classifiers to use for the final prediction.",
    "240930": "Do you balance this 5 classifier i.e. first block most popular classes, or its just random classes in block?\nDo you try to create classifier with intersections?",
    "241204": "I grouped by most popular classes. I also tried different group sizes (so some classifiers only had 100 classes but with many images, and other classifiers had 1000 classes but smaller ones)."
  },
  "source": "meta"
}