{
  "id": 70024,
  "title": "Comparison of different architectures and tricks",
  "url": "/competitions/human-protein-atlas-image-classification/discussion/70024",
  "author_name": "",
  "post_date": "2018-10-30T01:43:53.292750600Z",
  "votes": 6,
  "comment_count": 14,
  "views": 0,
  "content": "<h2>LB performance:</h2>\n\n<p>ResNet34: 0.496</p>\n\n<h2>Tricks:</h2>\n\n<ol>\n<li><p>Adding Y channel + freezing</p>\n\n<p>RGB: train/validation loss gap: 0.15</p>\n\n<p>RGBY: train/validation loss gap: 0.05</p></li>\n</ol>\n\n<p>You are welcomed to add more.....</p>",
  "messages": [
    {
      "id": "412322",
      "postDate": "10/30/2018 01:43:53",
      "content": "<h2>LB performance:</h2>\n\n<p>ResNet34: 0.496</p>\n\n<h2>Tricks:</h2>\n\n<ol>\n<li><p>Adding Y channel + freezing</p>\n\n<p>RGB: train/validation loss gap: 0.15</p>\n\n<p>RGBY: train/validation loss gap: 0.05</p></li>\n</ol>\n\n<p>You are welcomed to add more.....</p>",
      "rawMarkdown": "LB performance:\n---------------\n\nResNet34: 0.496\n\nTricks:\n-------\n\n 1.  Adding Y channel + freezing\n\n RGB: train/validation loss gap: 0.15\n\n RGBY: train/validation loss gap: 0.05\n\nYou are welcomed to add more.....",
      "votes": null
    },
    {
      "id": "412356",
      "postDate": "10/30/2018 03:46:31",
      "content": "<p>Could you also specify the image resolution used for training, and, if possible, the loss function.</p>",
      "rawMarkdown": "Could you also specify the image resolution used for training, and, if possible, the loss function.",
      "votes": null
    },
    {
      "id": "412554",
      "postDate": "10/30/2018 11:25:51",
      "content": "<p>Hi, Alex, just finetuned ResNet34?</p>",
      "rawMarkdown": "Hi, Alex, just finetuned ResNet34?",
      "votes": null
    },
    {
      "id": "412555",
      "postDate": "10/30/2018 11:27:06",
      "content": "<p>AlexL said he used 512*512 in another discussion。</p>",
      "rawMarkdown": "AlexL said he used 512*512 in another discussion。",
      "votes": null
    },
    {
      "id": "412604",
      "postDate": "10/30/2018 13:37:46",
      "content": "<p>Not yet. I was busy transfering my workflow from fastai to pytorch.</p>",
      "rawMarkdown": "Not yet. I was busy transfering my workflow from fastai to pytorch.",
      "votes": null
    },
    {
      "id": "412605",
      "postDate": "10/30/2018 13:39:30",
      "content": "<p>Loss for now is focal loss with gamma=2. I've switched back from 512x512 to 256x256 because I cannot load all images into memory for now.</p>",
      "rawMarkdown": "Loss for now is focal loss with gamma=2. I've switched back from 512x512 to 256x256 because I cannot load all images into memory for now.",
      "votes": null
    },
    {
      "id": "412663",
      "postDate": "10/30/2018 15:25:59",
      "content": "<p>Has anyone done a one to one comparison for focal loss vs binary cross-entropy?</p>",
      "rawMarkdown": "Has anyone done a one to one comparison for focal loss vs binary cross-entropy?",
      "votes": null
    },
    {
      "id": "412683",
      "postDate": "10/30/2018 15:53:33",
      "content": "<p>Model: B-17 (17th try of my own design)</p>\n\n<p>Loss: Combination of F1 and positively weighted BCE</p>\n\n<p>Input: 512x512 RGB</p>\n\n<p>I've not been able to get focal loss to work properly for me in Keras.</p>",
      "rawMarkdown": "Model: B-17 (17th try of my own design)\n\nLoss: Combination of F1 and positively weighted BCE\n\nInput: 512x512 RGB\n\n\nI've not been able to get focal loss to work properly for me in Keras.",
      "votes": null
    },
    {
      "id": "412722",
      "postDate": "10/30/2018 17:04:05",
      "content": "<p>Check my response to WolfgangReuter in <a href=\"https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb#\">https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb#</a>, where I saw that BCE performs worse. However, how Khoi Nguyen pointed out, sets in Python are not ordered and behaves randomly, so the val data is appeared to be different. It looks that they are performing more or less similar.</p>",
      "rawMarkdown": "Check my response to WolfgangReuter in https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb#, where I saw that BCE performs worse. However, how Khoi Nguyen pointed out, sets in Python are not ordered and behaves randomly, so the val data is appeared to be different. It looks that they are performing more or less similar.",
      "votes": null
    },
    {
      "id": "412803",
      "postDate": "10/30/2018 19:58:19",
      "content": "<p>Hi Alex, can you talk about how you choose the threshold for submission?</p>",
      "rawMarkdown": "Hi Alex, can you talk about how you choose the threshold for submission?",
      "votes": null
    },
    {
      "id": "412913",
      "postDate": "10/31/2018 01:38:57",
      "content": "<p>Thanks!</p>",
      "rawMarkdown": "Thanks!",
      "votes": null
    },
    {
      "id": "413314",
      "postDate": "10/31/2018 17:15:58",
      "content": "<p>here is the trick:</p>\n\n<p>\"get every class correct instead of every image correct\".</p>\n\n<p>if you can get good F1 score for the minor class (or most of the class), you will win the challenge.</p>\n\n<p>in the extreme case, for the minority class (only few images), you can hand mark the segmentation for the train image. This is a way to in input more \"domain knowledge\".  Then you can train segmentation or use this to predict attention mask, etc.</p>\n\n<p>Also, the classes are not independent. Hierarchical classification should help.</p>",
      "rawMarkdown": "here is the trick:\n\n\"get every class correct instead of every image correct\".\n\nif you can get good F1 score for the minor class (or most of the class), you will win the challenge.\n\nin the extreme case, for the minority class (only few images), you can hand mark the segmentation for the train image. This is a way to in input more \"domain knowledge\".  Then you can train segmentation or use this to predict attention mask, etc.\n\n\n Also, the classes are not independent. Hierarchical classification should help.",
      "votes": null
    },
    {
      "id": "413766",
      "postDate": "11/01/2018 13:49:20",
      "content": "<p>I got some strange problem, when I used focal loss for pytorch, the loss will not change(bce can work), but in fastai there is no problem. </p>",
      "rawMarkdown": "I got some strange problem, when I used focal loss for pytorch, the loss will not change(bce can work), but in fastai there is no problem.",
      "votes": null
    },
    {
      "id": "413840",
      "postDate": "11/01/2018 16:21:52",
      "content": "<p>put the images in SSD drive. I do not preload images.</p>",
      "rawMarkdown": "put the images in SSD drive. I do not preload images.",
      "votes": null
    },
    {
      "id": "413850",
      "postDate": "11/01/2018 16:34:03",
      "content": "<p>Got it, thanks!</p>",
      "rawMarkdown": "Got it, thanks!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 412356,
      "author_name": "iafoss",
      "author_url": "",
      "post_date": "10/30/2018 03:46:31",
      "content": "<p>Could you also specify the image resolution used for training, and, if possible, the loss function.</p>",
      "votes": null,
      "replies": [
        {
          "id": 412555,
          "author_name": "garybios",
          "author_url": "",
          "post_date": "10/30/2018 11:27:06",
          "content": "<p>AlexL said he used 512*512 in another discussion。</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 412605,
          "author_name": "alexanderliao",
          "author_url": "",
          "post_date": "10/30/2018 13:39:30",
          "content": "<p>Loss for now is focal loss with gamma=2. I've switched back from 512x512 to 256x256 because I cannot load all images into memory for now.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 413766,
          "author_name": "garybios",
          "author_url": "",
          "post_date": "11/01/2018 13:49:20",
          "content": "<p>I got some strange problem, when I used focal loss for pytorch, the loss will not change(bce can work), but in fastai there is no problem. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 413840,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "11/01/2018 16:21:52",
          "content": "<p>put the images in SSD drive. I do not preload images.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 413850,
          "author_name": "alexanderliao",
          "author_url": "",
          "post_date": "11/01/2018 16:34:03",
          "content": "<p>Got it, thanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 412554,
      "author_name": "garybios",
      "author_url": "",
      "post_date": "10/30/2018 11:25:51",
      "content": "<p>Hi, Alex, just finetuned ResNet34?</p>",
      "votes": null,
      "replies": [
        {
          "id": 412604,
          "author_name": "alexanderliao",
          "author_url": "",
          "post_date": "10/30/2018 13:37:46",
          "content": "<p>Not yet. I was busy transfering my workflow from fastai to pytorch.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 412663,
      "author_name": "rgreenblatt",
      "author_url": "",
      "post_date": "10/30/2018 15:25:59",
      "content": "<p>Has anyone done a one to one comparison for focal loss vs binary cross-entropy?</p>",
      "votes": null,
      "replies": [
        {
          "id": 412722,
          "author_name": "iafoss",
          "author_url": "",
          "post_date": "10/30/2018 17:04:05",
          "content": "<p>Check my response to WolfgangReuter in <a href=\"https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb#\">https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb#</a>, where I saw that BCE performs worse. However, how Khoi Nguyen pointed out, sets in Python are not ordered and behaves randomly, so the val data is appeared to be different. It looks that they are performing more or less similar.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 412913,
          "author_name": "rgreenblatt",
          "author_url": "",
          "post_date": "10/31/2018 01:38:57",
          "content": "<p>Thanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 412683,
      "author_name": "ldm314",
      "author_url": "",
      "post_date": "10/30/2018 15:53:33",
      "content": "<p>Model: B-17 (17th try of my own design)</p>\n\n<p>Loss: Combination of F1 and positively weighted BCE</p>\n\n<p>Input: 512x512 RGB</p>\n\n<p>I've not been able to get focal loss to work properly for me in Keras.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 412803,
      "author_name": "zhijianli",
      "author_url": "",
      "post_date": "10/30/2018 19:58:19",
      "content": "<p>Hi Alex, can you talk about how you choose the threshold for submission?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 413314,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "10/31/2018 17:15:58",
      "content": "<p>here is the trick:</p>\n\n<p>\"get every class correct instead of every image correct\".</p>\n\n<p>if you can get good F1 score for the minor class (or most of the class), you will win the challenge.</p>\n\n<p>in the extreme case, for the minority class (only few images), you can hand mark the segmentation for the train image. This is a way to in input more \"domain knowledge\".  Then you can train segmentation or use this to predict attention mask, etc.</p>\n\n<p>Also, the classes are not independent. Hierarchical classification should help.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "412322": "LB performance:\n---------------\n\nResNet34: 0.496\n\nTricks:\n-------\n\n 1.  Adding Y channel + freezing\n\n RGB: train/validation loss gap: 0.15\n\n RGBY: train/validation loss gap: 0.05\n\nYou are welcomed to add more.....",
    "412356": "Could you also specify the image resolution used for training, and, if possible, the loss function.",
    "412554": "Hi, Alex, just finetuned ResNet34?",
    "412555": "AlexL said he used 512*512 in another discussion。",
    "412604": "Not yet. I was busy transfering my workflow from fastai to pytorch.",
    "412605": "Loss for now is focal loss with gamma=2. I've switched back from 512x512 to 256x256 because I cannot load all images into memory for now.",
    "412663": "Has anyone done a one to one comparison for focal loss vs binary cross-entropy?",
    "412683": "Model: B-17 (17th try of my own design)\n\nLoss: Combination of F1 and positively weighted BCE\n\nInput: 512x512 RGB\n\n\nI've not been able to get focal loss to work properly for me in Keras.",
    "412722": "Check my response to WolfgangReuter in https://www.kaggle.com/iafoss/pretrained-resnet34-with-rgby-0-460-public-lb#, where I saw that BCE performs worse. However, how Khoi Nguyen pointed out, sets in Python are not ordered and behaves randomly, so the val data is appeared to be different. It looks that they are performing more or less similar.",
    "412803": "Hi Alex, can you talk about how you choose the threshold for submission?",
    "412913": "Thanks!",
    "413314": "here is the trick:\n\n\"get every class correct instead of every image correct\".\n\nif you can get good F1 score for the minor class (or most of the class), you will win the challenge.\n\nin the extreme case, for the minority class (only few images), you can hand mark the segmentation for the train image. This is a way to in input more \"domain knowledge\".  Then you can train segmentation or use this to predict attention mask, etc.\n\n\n Also, the classes are not independent. Hierarchical classification should help.",
    "413766": "I got some strange problem, when I used focal loss for pytorch, the loss will not change(bce can work), but in fastai there is no problem.",
    "413840": "put the images in SSD drive. I do not preload images.",
    "413850": "Got it, thanks!"
  },
  "source": "meta"
}