{
  "id": 76594,
  "title": "SEnet and variants not performing well",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/76594",
  "author_name": "",
  "post_date": "2019-01-04T15:14:28.158186100Z",
  "votes": null,
  "comment_count": 6,
  "views": 0,
  "content": "<p>SE networks does not seem to work well, has someone used them succesufully?\nResnet18 works pretty well, but seResnet18 does not seem to cut it. I was hoping that Squeeze and Excitation would help to detect small features in the cell images.\nI am currently testing a Resnet18 with just one SE layer at the end, but It does not look promising.</p>",
  "messages": [
    {
      "id": "450252",
      "postDate": "01/04/2019 15:14:28",
      "content": "<p>SE networks does not seem to work well, has someone used them succesufully?\nResnet18 works pretty well, but seResnet18 does not seem to cut it. I was hoping that Squeeze and Excitation would help to detect small features in the cell images.\nI am currently testing a Resnet18 with just one SE layer at the end, but It does not look promising.</p>",
      "rawMarkdown": "SE networks does not seem to work well, has someone used them succesufully?\nResnet18 works pretty well, but seResnet18 does not seem to cut it. I was hoping that Squeeze and Excitation would help to detect small features in the cell images.\nI am currently testing a Resnet18 with just one SE layer at the end, but It does not look promising.",
      "votes": null
    },
    {
      "id": "450341",
      "postDate": "01/04/2019 18:07:34",
      "content": "<p>I think that that it is more difficult to retrain SE models than regular ones, since a model is <strong>paying attention</strong> to one thing but another should be used, and it is taking a while for the model to figure it out. For this particular competition, where images are completely different from ImageNet, it may play a role.</p>",
      "rawMarkdown": "I think that that it is more difficult to retrain SE models than regular ones, since a model is **paying attention** to one thing but another should be used, and it is taking a while for the model to figure it out. For this particular competition, where images are completely different from ImageNet, it may play a role.",
      "votes": null
    },
    {
      "id": "450362",
      "postDate": "01/04/2019 19:16:21",
      "content": "<p>I've got the same observation with both ResNet models and Inception models. \nGoing to try CBAM now.</p>",
      "rawMarkdown": "I've got the same observation with both ResNet models and Inception models. \nGoing to try CBAM now.",
      "votes": null
    },
    {
      "id": "450396",
      "postDate": "01/04/2019 21:13:06",
      "content": "<p>Regular Resnets train very well, but the simpler one perform better for me. Resnet18 gets me better results than Resnet50. I have a variant of Resnet18 with an aditional Resnet SE block (between the head and body) but no luck. \nA full SeResnet does not seem to perform that well either. Haven't tried Inceptions.\n@Iafoss I am training SE nets from scratch. I have some muscle, 2 x 2080s. We should team some day, I am also using fastai V1.</p>",
      "rawMarkdown": "Regular Resnets train very well, but the simpler one perform better for me. Resnet18 gets me better results than Resnet50. I have a variant of Resnet18 with an aditional Resnet SE block (between the head and body) but no luck. \nA full SeResnet does not seem to perform that well either. Haven't tried Inceptions.\n@Iafoss I am training SE nets from scratch. I have some muscle, 2 x 2080s. We should team some day, I am also using fastai V1.",
      "votes": null
    },
    {
      "id": "450401",
      "postDate": "01/04/2019 21:26:34",
      "content": "<p>Well, CBAM from scratch is not a good idea.</p>",
      "rawMarkdown": "Well, CBAM from scratch is not a good idea.",
      "votes": null
    },
    {
      "id": "450411",
      "postDate": "01/04/2019 22:15:22",
      "content": "<p>I'm a bit puzzled... my single best model in terms of public LB is actually a SE-ResNet50... </p>",
      "rawMarkdown": "I'm a bit puzzled... my single best model in terms of public LB is actually a SE-ResNet50...",
      "votes": null
    },
    {
      "id": "450467",
      "postDate": "01/05/2019 02:42:39",
      "content": "<p>In my case, res50 is better than res18.</p>",
      "rawMarkdown": "In my case, res50 is better than res18.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 450341,
      "author_name": "iafoss",
      "author_url": "",
      "post_date": "01/04/2019 18:07:34",
      "content": "<p>I think that that it is more difficult to retrain SE models than regular ones, since a model is <strong>paying attention</strong> to one thing but another should be used, and it is taking a while for the model to figure it out. For this particular competition, where images are completely different from ImageNet, it may play a role.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 450362,
      "author_name": "spsancti",
      "author_url": "",
      "post_date": "01/04/2019 19:16:21",
      "content": "<p>I've got the same observation with both ResNet models and Inception models. \nGoing to try CBAM now.</p>",
      "votes": null,
      "replies": [
        {
          "id": 450401,
          "author_name": "spsancti",
          "author_url": "",
          "post_date": "01/04/2019 21:26:34",
          "content": "<p>Well, CBAM from scratch is not a good idea.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 450396,
      "author_name": "tcapelle",
      "author_url": "",
      "post_date": "01/04/2019 21:13:06",
      "content": "<p>Regular Resnets train very well, but the simpler one perform better for me. Resnet18 gets me better results than Resnet50. I have a variant of Resnet18 with an aditional Resnet SE block (between the head and body) but no luck. \nA full SeResnet does not seem to perform that well either. Haven't tried Inceptions.\n@Iafoss I am training SE nets from scratch. I have some muscle, 2 x 2080s. We should team some day, I am also using fastai V1.</p>",
      "votes": null,
      "replies": [
        {
          "id": 450467,
          "author_name": "zjucor",
          "author_url": "",
          "post_date": "01/05/2019 02:42:39",
          "content": "<p>In my case, res50 is better than res18.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 450411,
      "author_name": "thundo",
      "author_url": "",
      "post_date": "01/04/2019 22:15:22",
      "content": "<p>I'm a bit puzzled... my single best model in terms of public LB is actually a SE-ResNet50... </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "450252": "SE networks does not seem to work well, has someone used them succesufully?\nResnet18 works pretty well, but seResnet18 does not seem to cut it. I was hoping that Squeeze and Excitation would help to detect small features in the cell images.\nI am currently testing a Resnet18 with just one SE layer at the end, but It does not look promising.",
    "450341": "I think that that it is more difficult to retrain SE models than regular ones, since a model is **paying attention** to one thing but another should be used, and it is taking a while for the model to figure it out. For this particular competition, where images are completely different from ImageNet, it may play a role.",
    "450362": "I've got the same observation with both ResNet models and Inception models. \nGoing to try CBAM now.",
    "450396": "Regular Resnets train very well, but the simpler one perform better for me. Resnet18 gets me better results than Resnet50. I have a variant of Resnet18 with an aditional Resnet SE block (between the head and body) but no luck. \nA full SeResnet does not seem to perform that well either. Haven't tried Inceptions.\n@Iafoss I am training SE nets from scratch. I have some muscle, 2 x 2080s. We should team some day, I am also using fastai V1.",
    "450401": "Well, CBAM from scratch is not a good idea.",
    "450411": "I'm a bit puzzled... my single best model in terms of public LB is actually a SE-ResNet50...",
    "450467": "In my case, res50 is better than res18."
  },
  "source": "meta"
}