{
  "id": 239048,
  "title": "31rd Place Solution: CAM only.",
  "url": "/competitions/hpa-single-cell-image-classification/discussion/239048",
  "author_name": "Chenglu",
  "post_date": "2021-05-14T12:25:19.579000",
  "votes": 15,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Thanks Kaggle for hosting this incredible game, I really learned a lot from it. And thanks the Kagglers for the awesome ideas. This is my highest rank in my Kaggle trip so I decide to write the journey down.</p>\n<h1>Summary</h1>\n<p>My final solution only use CAM(Class Activation Map) because other tricks does not work well for me. So it's a simple, e2e solution other than a complex pipeline. And it suprised me that the CAM only pipeline can give me a silver place.</p>\n<p>I used Knowledge Distillation for the CAM layer becuase I think it will help the model to generate more slid CAM, well, it does.</p>\n<p>I change the model downsampling size to 16(2**4) so for a 512 x 512 input, I will get a 32 x 32 CAM. And I use 3 layers output to get 3 CAMs at once.</p>\n<p>The only working backbone for me is ResNest and all its variant (Thank You Zhang Hang).</p>\n<h1>Training</h1>\n<p>The picture below describes the whole training pipeline:</p>\n<p><img src=\"http://github.com/louis-she/static/blob/main/hpasolution.png?raw=true\" alt=\"\"></p>\n<h5>CAM Extractor</h5>\n<p>The DRS and Dropout layer could force the model focusing on the whole image other than one or two cell. DRS can found here: <a href=\"https://github.com/qjadud1994/DRS\" target=\"_blank\">https://github.com/qjadud1994/DRS</a></p>\n<p><img src=\"https://github.com/qjadud1994/DRS/raw/main/docs/DRS_CAM.png\" alt=\"\"></p>\n<h5>Training Details</h5>\n<p>Input: 512<br>\nOptimizer: Adam <br>\nLoss: BCE<br>\nCV: 1 fold and then fine tune on whole dataset(5 fold is too expensive for me)<br>\nAugmentation: flip, random contrast<br>\nModel: ResNest and all the variant</p>\n<h1>Inference</h1>\n<p>I finally used 5 models toghether(all from timm) to ensemble, all the model input is 512 x 512.</p>\n<ol>\n<li>resnest50d_1s4x24d with DRS</li>\n<li>resnest101e</li>\n<li>resnest50d_1s4x24d</li>\n<li>resnest50d_4s2x40d with DRS</li>\n<li>resnest50d with DRS</li>\n</ol>\n<p>TTA: flip both vertically and horizontally, for every spatial changed image, the contrast is changed too, the contrast is changed as 0.64, 0.88, 1.12, 1.36.</p>\n<p>Cell Segment: original segmentator provided by the host.</p>\n<p>Cell Scoring: Sum the pixel value of the CAM in the cell area, and divided by the nonzero pixel numbers in that cell.</p>\n<p><strong>Thanks for reading.</strong></p>",
  "messages": [
    {
      "id": 1307399,
      "postDate": "2021-05-14T12:25:19.580Z",
      "content": "<p>Thanks Kaggle for hosting this incredible game, I really learned a lot from it. And thanks the Kagglers for the awesome ideas. This is my highest rank in my Kaggle trip so I decide to write the journey down.</p>\n<h1>Summary</h1>\n<p>My final solution only use CAM(Class Activation Map) because other tricks does not work well for me. So it's a simple, e2e solution other than a complex pipeline. And it suprised me that the CAM only pipeline can give me a silver place.</p>\n<p>I used Knowledge Distillation for the CAM layer becuase I think it will help the model to generate more slid CAM, well, it does.</p>\n<p>I change the model downsampling size to 16(2**4) so for a 512 x 512 input, I will get a 32 x 32 CAM. And I use 3 layers output to get 3 CAMs at once.</p>\n<p>The only working backbone for me is ResNest and all its variant (Thank You Zhang Hang).</p>\n<h1>Training</h1>\n<p>The picture below describes the whole training pipeline:</p>\n<p><img src=\"http://github.com/louis-she/static/blob/main/hpasolution.png?raw=true\" alt=\"\"></p>\n<h5>CAM Extractor</h5>\n<p>The DRS and Dropout layer could force the model focusing on the whole image other than one or two cell. DRS can found here: <a href=\"https://github.com/qjadud1994/DRS\" target=\"_blank\">https://github.com/qjadud1994/DRS</a></p>\n<p><img src=\"https://github.com/qjadud1994/DRS/raw/main/docs/DRS_CAM.png\" alt=\"\"></p>\n<h5>Training Details</h5>\n<p>Input: 512<br>\nOptimizer: Adam <br>\nLoss: BCE<br>\nCV: 1 fold and then fine tune on whole dataset(5 fold is too expensive for me)<br>\nAugmentation: flip, random contrast<br>\nModel: ResNest and all the variant</p>\n<h1>Inference</h1>\n<p>I finally used 5 models toghether(all from timm) to ensemble, all the model input is 512 x 512.</p>\n<ol>\n<li>resnest50d_1s4x24d with DRS</li>\n<li>resnest101e</li>\n<li>resnest50d_1s4x24d</li>\n<li>resnest50d_4s2x40d with DRS</li>\n<li>resnest50d with DRS</li>\n</ol>\n<p>TTA: flip both vertically and horizontally, for every spatial changed image, the contrast is changed too, the contrast is changed as 0.64, 0.88, 1.12, 1.36.</p>\n<p>Cell Segment: original segmentator provided by the host.</p>\n<p>Cell Scoring: Sum the pixel value of the CAM in the cell area, and divided by the nonzero pixel numbers in that cell.</p>\n<p><strong>Thanks for reading.</strong></p>",
      "rawMarkdown": "Thanks Kaggle for hosting this incredible game, I really learned a lot from it. And thanks the Kagglers for the awesome ideas. This is my highest rank in my Kaggle trip so I decide to write the journey down.\n\n# Summary\n\nMy final solution only use CAM(Class Activation Map) because other tricks does not work well for me. So it's a simple, e2e solution other than a complex pipeline. And it suprised me that the CAM only pipeline can give me a silver place.\n\nI used Knowledge Distillation for the CAM layer becuase I think it will help the model to generate more slid CAM, well, it does.\n\nI change the model downsampling size to 16(2**4) so for a 512 x 512 input, I will get a 32 x 32 CAM. And I use 3 layers output to get 3 CAMs at once.\n\nThe only working backbone for me is ResNest and all its variant (Thank You Zhang Hang).\n\n# Training\n\nThe picture below describes the whole training pipeline:\n\n![](http://github.com/louis-she/static/blob/main/hpasolution.png?raw=true)\n\n##### CAM Extractor\n\nThe DRS and Dropout layer could force the model focusing on the whole image other than one or two cell. DRS can found here: https://github.com/qjadud1994/DRS\n\n![](https://github.com/qjadud1994/DRS/raw/main/docs/DRS_CAM.png)\n\n##### Training Details\n\nInput: 512\nOptimizer: Adam \nLoss: BCE\nCV: 1 fold and then fine tune on whole dataset(5 fold is too expensive for me)\nAugmentation: flip, random contrast\nModel: ResNest and all the variant\n\n# Inference\n\nI finally used 5 models toghether(all from timm) to ensemble, all the model input is 512 x 512.\n\n1. resnest50d_1s4x24d with DRS\n2. resnest101e\n3. resnest50d_1s4x24d\n4. resnest50d_4s2x40d with DRS\n5. resnest50d with DRS\n\nTTA: flip both vertically and horizontally, for every spatial changed image, the contrast is changed too, the contrast is changed as 0.64, 0.88, 1.12, 1.36.\n\nCell Segment: original segmentator provided by the host.\n\nCell Scoring: Sum the pixel value of the CAM in the cell area, and divided by the nonzero pixel numbers in that cell.\n\n**Thanks for reading.**",
      "votes": 15
    },
    {
      "id": 1307777,
      "postDate": "2021-05-14T16:46:38.623Z",
      "content": "<p>This is interesting! Congratulations. </p>\n<p>From where did you get the idea for your training pipeline? Can you share some relevant sources.</p>",
      "rawMarkdown": "This is interesting! Congratulations. \n\nFrom where did you get the idea for your training pipeline? Can you share some relevant sources.",
      "replies": [
        {
          "id": 1308113,
          "postDate": "2021-05-15T02:42:55.717Z",
          "content": "<p>I come up with the pipline myself ; )</p>",
          "rawMarkdown": "I come up with the pipline myself ; )"
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1307777,
      "author_name": "antoreepjana",
      "author_url": "",
      "post_date": "2021-05-14T16:46:38.623000",
      "content": "<p>This is interesting! Congratulations. </p>\n<p>From where did you get the idea for your training pipeline? Can you share some relevant sources.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1308113,
          "author_name": "Chenglu",
          "author_url": "",
          "post_date": "2021-05-15T02:42:55.717000",
          "content": "<p>I come up with the pipline myself ; )</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1307399": "Thanks Kaggle for hosting this incredible game, I really learned a lot from it. And thanks the Kagglers for the awesome ideas. This is my highest rank in my Kaggle trip so I decide to write the journey down.\n\n# Summary\n\nMy final solution only use CAM(Class Activation Map) because other tricks does not work well for me. So it's a simple, e2e solution other than a complex pipeline. And it suprised me that the CAM only pipeline can give me a silver place.\n\nI used Knowledge Distillation for the CAM layer becuase I think it will help the model to generate more slid CAM, well, it does.\n\nI change the model downsampling size to 16(2**4) so for a 512 x 512 input, I will get a 32 x 32 CAM. And I use 3 layers output to get 3 CAMs at once.\n\nThe only working backbone for me is ResNest and all its variant (Thank You Zhang Hang).\n\n# Training\n\nThe picture below describes the whole training pipeline:\n\n![](http://github.com/louis-she/static/blob/main/hpasolution.png?raw=true)\n\n##### CAM Extractor\n\nThe DRS and Dropout layer could force the model focusing on the whole image other than one or two cell. DRS can found here: https://github.com/qjadud1994/DRS\n\n![](https://github.com/qjadud1994/DRS/raw/main/docs/DRS_CAM.png)\n\n##### Training Details\n\nInput: 512\nOptimizer: Adam \nLoss: BCE\nCV: 1 fold and then fine tune on whole dataset(5 fold is too expensive for me)\nAugmentation: flip, random contrast\nModel: ResNest and all the variant\n\n# Inference\n\nI finally used 5 models toghether(all from timm) to ensemble, all the model input is 512 x 512.\n\n1. resnest50d_1s4x24d with DRS\n2. resnest101e\n3. resnest50d_1s4x24d\n4. resnest50d_4s2x40d with DRS\n5. resnest50d with DRS\n\nTTA: flip both vertically and horizontally, for every spatial changed image, the contrast is changed too, the contrast is changed as 0.64, 0.88, 1.12, 1.36.\n\nCell Segment: original segmentator provided by the host.\n\nCell Scoring: Sum the pixel value of the CAM in the cell area, and divided by the nonzero pixel numbers in that cell.\n\n**Thanks for reading.**",
    "1307777": "This is interesting! Congratulations. \n\nFrom where did you get the idea for your training pipeline? Can you share some relevant sources."
  }
}