{
  "id": 45710,
  "title": "Trainable and not trainable models",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/45710",
  "author_name": "",
  "post_date": "2017-12-14T23:56:57.980560900Z",
  "votes": 6,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hey all, I was wondering if anyone else experienced substantially different behavior when training models. For example, for me inceptionv4 was training like crap, while inception-resnet was working perfectly. Another model that I was not able to train properly was Densenet. Was anyone able to train a Densenet? On the other hand ResNets worked perfectly.</p>",
  "messages": [
    {
      "id": "257778",
      "postDate": "12/14/2017 23:56:57",
      "content": "<p>Hey all, I was wondering if anyone else experienced substantially different behavior when training models. For example, for me inceptionv4 was training like crap, while inception-resnet was working perfectly. Another model that I was not able to train properly was Densenet. Was anyone able to train a Densenet? On the other hand ResNets worked perfectly.</p>",
      "rawMarkdown": "Hey all, I was wondering if anyone else experienced substantially different behavior when training models. For example, for me inceptionv4 was training like crap, while inception-resnet was working perfectly. Another model that I was not able to train properly was Densenet. Was anyone able to train a Densenet? On the other hand ResNets worked perfectly.",
      "votes": null
    },
    {
      "id": "257787",
      "postDate": "12/15/2017 00:18:24",
      "content": "<p>InceptionResnet - great; best single model 0.739</p>\n\n<p>Xception - great; best single model 0.735</p>\n\n<p>inceptionV4 - OK; best single model 0.732</p>\n\n<p>resnet50 - OK/poor; 0.705</p>\n\n<p>resnet152 - poor: didn't submit, 0.701 on validation</p>\n\n<p>densenet - poor: didn't submit, highly unstable</p>\n\n<p>All models seemed to benefit from dropout (0.20) and horizontal flips.  Additional augmentations seemed to make things worse.</p>",
      "rawMarkdown": "InceptionResnet - great; best single model 0.739\n\nXception - great; best single model 0.735\n\ninceptionV4 - OK; best single model 0.732\n\nresnet50 - OK/poor; 0.705\n\nresnet152 - poor: didn't submit, 0.701 on validation\n\ndensenet - poor: didn't submit, highly unstable\n\nAll models seemed to benefit from dropout (0.20) and horizontal flips.  Additional augmentations seemed to make things worse.",
      "votes": null
    },
    {
      "id": "257790",
      "postDate": "12/15/2017 00:24:55",
      "content": "<p>Nice list, thanks for sharing!</p>\n\n<p>Alson the novel <a href=\"https://arxiv.org/abs/1707.01629\">Dual path networks</a> worked nicely for us.</p>",
      "rawMarkdown": "Nice list, thanks for sharing!\n\nAlson the novel [Dual path networks][1] worked nicely for us.\n\n\n  [1]: https://arxiv.org/abs/1707.01629",
      "votes": null
    },
    {
      "id": "257795",
      "postDate": "12/15/2017 00:29:26",
      "content": "<p>Would you mind posting your models online. i try to extract your features and train classifiers on top of that. (give me your training and validation list as well)</p>",
      "rawMarkdown": "Would you mind posting your models online. i try to extract your features and train classifiers on top of that. (give me your training and validation list as well)",
      "votes": null
    },
    {
      "id": "257802",
      "postDate": "12/15/2017 00:36:21",
      "content": "<p>I trained several custom-built densenets somewhat successfully, but none of them broke 70%, mostly because they were too slow for me to train from scratch in the time given. One got pretty close to being usable, though.</p>",
      "rawMarkdown": "I trained several custom-built densenets somewhat successfully, but none of them broke 70%, mostly because they were too slow for me to train from scratch in the time given. One got pretty close to being usable, though.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 257787,
      "author_name": "tivfrvqhs5",
      "author_url": "",
      "post_date": "12/15/2017 00:18:24",
      "content": "<p>InceptionResnet - great; best single model 0.739</p>\n\n<p>Xception - great; best single model 0.735</p>\n\n<p>inceptionV4 - OK; best single model 0.732</p>\n\n<p>resnet50 - OK/poor; 0.705</p>\n\n<p>resnet152 - poor: didn't submit, 0.701 on validation</p>\n\n<p>densenet - poor: didn't submit, highly unstable</p>\n\n<p>All models seemed to benefit from dropout (0.20) and horizontal flips.  Additional augmentations seemed to make things worse.</p>",
      "votes": null,
      "replies": [
        {
          "id": 257790,
          "author_name": "mihaskalic",
          "author_url": "",
          "post_date": "12/15/2017 00:24:55",
          "content": "<p>Nice list, thanks for sharing!</p>\n\n<p>Alson the novel <a href=\"https://arxiv.org/abs/1707.01629\">Dual path networks</a> worked nicely for us.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 257795,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "12/15/2017 00:29:26",
          "content": "<p>Would you mind posting your models online. i try to extract your features and train classifiers on top of that. (give me your training and validation list as well)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 257802,
      "author_name": "eachshadow",
      "author_url": "",
      "post_date": "12/15/2017 00:36:21",
      "content": "<p>I trained several custom-built densenets somewhat successfully, but none of them broke 70%, mostly because they were too slow for me to train from scratch in the time given. One got pretty close to being usable, though.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "257778": "Hey all, I was wondering if anyone else experienced substantially different behavior when training models. For example, for me inceptionv4 was training like crap, while inception-resnet was working perfectly. Another model that I was not able to train properly was Densenet. Was anyone able to train a Densenet? On the other hand ResNets worked perfectly.",
    "257787": "InceptionResnet - great; best single model 0.739\n\nXception - great; best single model 0.735\n\ninceptionV4 - OK; best single model 0.732\n\nresnet50 - OK/poor; 0.705\n\nresnet152 - poor: didn't submit, 0.701 on validation\n\ndensenet - poor: didn't submit, highly unstable\n\nAll models seemed to benefit from dropout (0.20) and horizontal flips.  Additional augmentations seemed to make things worse.",
    "257790": "Nice list, thanks for sharing!\n\nAlson the novel [Dual path networks][1] worked nicely for us.\n\n\n  [1]: https://arxiv.org/abs/1707.01629",
    "257795": "Would you mind posting your models online. i try to extract your features and train classifiers on top of that. (give me your training and validation list as well)",
    "257802": "I trained several custom-built densenets somewhat successfully, but none of them broke 70%, mostly because they were too slow for me to train from scratch in the time given. One got pretty close to being usable, though."
  },
  "source": "meta"
}