![\"cell](\"https://raw.githubusercontent.com/martin-chobanyan/hpa-single-cell/main/resources/cell-transformer-overview.png\")
Hey fellow Kaggle competitors!
\nI had a lot of fun with this unique competition and I'm looking forward to reading your own solutions to this problem. Here is an overview of my final approach + link to my code (open images in a new tab for higher res):
\n\n
First off, in all of my models I used the pre-trained backbone CNN of @bestfitting 's winning DenseNet-121 models from the previous competition (3 different pre-trained versions). I stacked all of the color stains into 4-channel images and resized them to 1536x1536 and fed them through the backbone CNN, resulting in 1024 feature maps.
\nI then extracted a feature vector for each cell in the image by performing Region-of-Interest (RoI) pooling over the feature maps using the cell segmentation masks from the HPA Cell Segmentator. The pooling layer consisted of a concatenation of both avg-pool and max-pool features (resulting in a 2048-dim vector per cell). I also applied an adaptive average pool over the cell masks to reduce them to 8x8, which I then flattened. These downsampled, flatten masks served as position encoding for the cell regions (see diagram below):
\n![\"roi](\"https://raw.githubusercontent.com/martin-chobanyan/hpa-single-cell/main/resources/cell-roi-pool-overview.png\")
At this stage, we have a sequence of cell vectors, which allows us to explore the use of a Transformer model. The cell feature vectors are passed through a feed-forward layer which maps them to a 1024-dim embedding. Then, four encoder layers are applied as they were defined in the original Transformer paper. A final feed forward layer then maps each cell to an 18-dimensional logit.
\nTo predict the image-level labels, a log-sum-exponential layer is applied which is defined as f(x) = (1/r) * ln(mean(e^(r*x))) with r=5 in our case. This serves as a mechanism to interpolate between average pooling and max pooling with different values for r.
\nFor training I used Focal Loss with sigmoid activation and flip/rotate/crop augmentations. During inference, I extracted the cell-level logits and applied a sigmoid activation before their reduction to image-level labels. The final class prediction for a cell was averaged across all three models (three different pre-trained backbones).
\nI also explored the use of Peak Response Maps as a CAM based approach to the problem. Though the localizations looked good, the model was a bit too sensitive to classes which were not present in the image.
\nThings I would have tried with more time:
\nHey fellow Kaggle competitors!
\nI had a lot of fun with this unique competition and I'm looking forward to reading your own solutions to this problem. Here is an overview of my final approach + link to my code (open images in a new tab for higher res):
\n\n
First off, in all of my models I used the pre-trained backbone CNN of @bestfitting 's winning DenseNet-121 models from the previous competition (3 different pre-trained versions). I stacked all of the color stains into 4-channel images and resized them to 1536x1536 and fed them through the backbone CNN, resulting in 1024 feature maps.
\nI then extracted a feature vector for each cell in the image by performing Region-of-Interest (RoI) pooling over the feature maps using the cell segmentation masks from the HPA Cell Segmentator. The pooling layer consisted of a concatenation of both avg-pool and max-pool features (resulting in a 2048-dim vector per cell). I also applied an adaptive average pool over the cell masks to reduce them to 8x8, which I then flattened. These downsampled, flatten masks served as position encoding for the cell regions (see diagram below):
\n
At this stage, we have a sequence of cell vectors, which allows us to explore the use of a Transformer model. The cell feature vectors are passed through a feed-forward layer which maps them to a 1024-dim embedding. Then, four encoder layers are applied as they were defined in the original Transformer paper. A final feed forward layer then maps each cell to an 18-dimensional logit.
\nTo predict the image-level labels, a log-sum-exponential layer is applied which is defined as f(x) = (1/r) * ln(mean(e^(r*x))) with r=5 in our case. This serves as a mechanism to interpolate between average pooling and max pooling with different values for r.
\nFor training I used Focal Loss with sigmoid activation and flip/rotate/crop augmentations. During inference, I extracted the cell-level logits and applied a sigmoid activation before their reduction to image-level labels. The final class prediction for a cell was averaged across all three models (three different pre-trained backbones).
\nI also explored the use of Peak Response Maps as a CAM based approach to the problem. Though the localizations looked good, the model was a bit too sensitive to classes which were not present in the image.
\nThings I would have tried with more time:
\nHi @mch123
\nI'm a newbie, I want to learn more and practice your source code, do you have any version above Kaggle notebook. If yes I hope you can share it so I can be more accessible. many thanks.
Hi @nguyenhuudat,
\nI just made my submission kernel public: https://www.kaggle.com/mch123/ensemble-cell-transformer-2
\nYou'll also have to use the model states in this dataset I made: https://www.kaggle.com/mch123/hparoimodels
", "rawMarkdown": "Hi @nguyenhuudat,\n\nI just made my submission kernel public: https://www.kaggle.com/mch123/ensemble-cell-transformer-2\n\nYou'll also have to use the model states in this dataset I made: https://www.kaggle.com/mch123/hparoimodels" } ] }, { "id": 1305022, "postDate": "2021-05-13T04:25:12.803Z", "content": "@mch123 Congratulations and Thanks for sharing the approach
", "rawMarkdown": "@mch123 Congratulations and Thanks for sharing the approach", "votes": 1, "replies": [ { "id": 1307633, "postDate": "2021-05-14T15:09:47.800Z", "content": "Thank you!
", "rawMarkdown": "Thank you!" } ] } ], "comments": [ { "id": 1309793, "author_name": "Dat Nguyen", "author_url": "", "post_date": "2021-05-16T09:57:34.680000", "content": "Hi @mch123
\nI'm a newbie, I want to learn more and practice your source code, do you have any version above Kaggle notebook. If yes I hope you can share it so I can be more accessible. many thanks.
Hi @nguyenhuudat,
\nI just made my submission kernel public: https://www.kaggle.com/mch123/ensemble-cell-transformer-2
\nYou'll also have to use the model states in this dataset I made: https://www.kaggle.com/mch123/hparoimodels
", "votes": 0, "replies": [] } ] }, { "id": 1305022, "author_name": "Tensor Girl", "author_url": "", "post_date": "2021-05-13T04:25:12.803000", "content": "@mch123 Congratulations and Thanks for sharing the approach
", "votes": 1, "replies": [ { "id": 1307633, "author_name": "Martin Chobanyan", "author_url": "", "post_date": "2021-05-14T15:09:47.800000", "content": "Thank you!
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1304735": "Hey fellow Kaggle competitors! \n\nI had a lot of fun with this unique competition and I'm looking forward to reading your own solutions to this problem. Here is an overview of my final approach + link to my code (open images in a new tab for higher res):\n\n[Github Repo](https://github.com/martin-chobanyan/hpa-single-cell)\n\n\n\nFirst off, in all of my models I used the pre-trained backbone CNN of @bestfitting 's winning DenseNet-121 models from the previous competition (3 different pre-trained versions). I stacked all of the color stains into 4-channel images and resized them to 1536x1536 and fed them through the backbone CNN, resulting in 1024 feature maps.\n\nI then extracted a feature vector for each cell in the image by performing Region-of-Interest (RoI) pooling over the feature maps using the cell segmentation masks from the HPA Cell Segmentator. The pooling layer consisted of a concatenation of both avg-pool and max-pool features (resulting in a 2048-dim vector per cell). I also applied an adaptive average pool over the cell masks to reduce them to 8x8, which I then flattened. These downsampled, flatten masks served as position encoding for the cell regions (see diagram below):\n\n\n\nAt this stage, we have a sequence of cell vectors, which allows us to explore the use of a Transformer model. The cell feature vectors are passed through a feed-forward layer which maps them to a 1024-dim embedding. Then, four encoder layers are applied as they were defined in the original Transformer paper. A final feed forward layer then maps each cell to an 18-dimensional logit.\n\nTo predict the image-level labels, a log-sum-exponential layer is applied which is defined as f(x) = (1/r) * ln(mean(e^(r*x))) with r=5 in our case. This serves as a mechanism to interpolate between average pooling and max pooling with different values for r.\n\nFor training I used Focal Loss with sigmoid activation and flip/rotate/crop augmentations. During inference, I extracted the cell-level logits and applied a sigmoid activation before their reduction to image-level labels. The final class prediction for a cell was averaged across all three models (three different pre-trained backbones).\n\nI also explored the use of [Peak Response Maps](https://arxiv.org/abs/1804.00880) as a CAM based approach to the problem. Though the localizations looked good, the model was a bit too sensitive to classes which were not present in the image.\n\nThings I would have tried with more time:\n\n- Ensembling with a cell-level classifier\n- Explore more ways of merging the peak-response-map results with the cell transformer model\n- Domain adaptation to the public test set (since we know the test set has higher variability in protein locations within a given image compared to the training set).", "1309793": "Hi @mch123 \nI'm a newbie, I want to learn more and practice your source code, do you have any version above Kaggle notebook. If yes I hope you can share it so I can be more accessible. many thanks.", "1305022": "@mch123 Congratulations and Thanks for sharing the approach" } }