{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":22990,"databundleVersionId":2048213,"sourceType":"competition"},{"sourceId":2216853,"sourceType":"datasetVersion","datasetId":1314001}],"dockerImageVersionId":30096,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"from matplotlib import pyplot as plt\n\ndef plot_image(img_path, title, figsize=(20, 25), title_pos=-0.2, fontsize=15):\n    \"\"\"\n    Helper to visualize images loaded from an external dataset\n    \"\"\"\n    img=plt.imread(img_path)\n    fig = plt.figure(figsize=figsize)\n    plt.axis('off')\n    plt.title(title, y=title_pos, fontsize=fontsize)\n    plt.imshow(img);","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Introduction\n\nThe Human Biomolecular Atlas Program ([HuBMAP](https://hubmapconsortium.org/about/)) is aiming to create an open global atlas of the human\nbody at the cellular level, to accelerate understanding of the relationships between cell and tissue\norganization, their functions, and human health. HuBMAP will create the next generation of molecular\nanalysis technologies and computational tools, enabling the generation of foundational 3D tissue maps\nand construction of an atlas of the function and relationships among cells in the human body. One\ncomponent of this overarching goal is to identify medically relevant functional tissue units (FTUs) within\nwhole slide microscopy images of human tissues. Once these FTUs are detected, information on size,\nshape, variability in number, and location within the tissue samples can be used to help in building a\nspatially accurate and semantically explicit model of the human body, [as pointed out](https://www.kaggle.com/leahscherschel/dataset-details) by organizers of\nthis challenge.\n\nOne example of an FTU, which is in the focus of this competition, is the glomerulus found in the outer\nlayer of kidney tissue. Glomeruli perform filtration of waste products out of blood and are represented\nby three-dimensional blocks of cells centered around a capillary, such that each cell in this block is\nwithin diffusion distance from any other cell in the same block. The objective of this challenge is the\nsegmentation of regions with glomeruli in human kidney tissue images across different tissue\npreparation pipelines. Below we provide an overview of the main data challenges arising in this task\nas well as a brief solution description and a list of external data used to build our models. Next, we\ndiscuss the main results and the limitations of the proposed approach, including confidence estimation.\nFinally, we applied the developed method for segmentation of nucleus cells in Line-field Confocal\nOptical Coherence Tomography (LC-OCT) images, and provide the main conclusions of our work.\n\n\n## About our team : deeplive.exe\n\n- Theo Viel (@theoviel) is one of the three French Kaggle Competitions Grandmaster and\ncurrently ranks 17. His Kaggle achievements gave him a lot of experience in Computer Vision\nand NLP. He currently works as a Computer Vision Researcher at DAMAE Medical where he\nuses his modeling skills to improve the understanding of healthy and pathological skin.\n- Maxim Shugaev (@Iafoss) is a Computer Vision Researcher at Intelligent Automation, Inc. In\n2019 he has received his PhD degree from the University of Virginia, USA, in Applied Physics.\nBeyond that Maxim has an extensive experience in more than 15 computer vision related\nprojects ranging from cancer grade assessment based on biopsies and pneumothorax\nsegmentation to deepfake detection, satellite image segmentation, and few-shot learning.\n- Sebastien Fischman (@Optimo) is a 30-year-old French Computer Vision Researcher at\nDAMAE Medical. Before working with medical images, he had several Machine Learning\nexperiences on very different topics ranging from Auto Machine Learning, stock market\npredictions using sentiment analysis based on Tweets, add retargeting and user segmentations.\n\n## Thanks to the organisers\n\nWe would start by thanking the organizers and the Kaggle team for hosting this competition. We\nbelieve that leveraging Artificial Intelligence in medical imaging will play a key role in improving the\noverall understanding of the human body.\n\n\n# Part 1: Data challenges\n\nIn this section we highlight the key challenges provided by the competition data to build a practically\nusable model.\n\n## Data quality variation\n\nBecause of the nature of the data, it is very likely that images have artefacts. Indeed, PAS kidney images are quite hard to acquire and a lot of variability can be observed. Specifically, such quality issues\ncan be found both in train and test datasets, and the built models, therefore, should be robust to\nunexpected image changes. To achieve this objective, we trained our models with an extensive image\naugmentation, including MixUp [[1]](https://arxiv.org/abs/1710.09412), which helps distinguishing glomeruli from the background in\nregions affected by image artefacts. As an illustration we plot model predictions on parts of the *aa05346ff*\nand *afa5e8098* images (from test and train datasets, respectively), which include regions affected by\nluminosity, contrast and blur issues. In all cases the model continues giving meaningful predictions missing only one\nglomerulus in the *aa05346ff*. Moreover, the model is able to predict masks for glomeruli in the dark\nregion of *afa5e8098*, even if the ground truth masks were intentionally excluded during annotation\nbecause of the insufficient image quality (see Fig 1).\n","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 1: Example of quality issues in aa05346ff (test) and afa5e8098 (train) images. Blue and red\ncolors correspond to model predictions of healthy and unhealthy glomeruli. Green color in right figure\ndepicts glomeruli not included into annotation because of insufficient image quality but still predicted\nby the model. The zoomed in image depicts one of these glomeruli.\"\"\"\nplot_image(\n    \"../input/presentation-images/image_quality_issues.png\",\n    title,\n    (20, 25),\n    -0.2, \n    fontsize=18\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Non-Sclerotic, Sclerotic, and Fibrous Crescent Glomeruli\n\nOne of the challenges of glomeruli detection is the significant variation of tissue peculiarities from one\npatient to another, which is deteriorated by the various tissue processing methods used. One more layer\nof complexity, meanwhile, is added by the fact that not all glomeruli are healthy. Variety of glomeruli\ndefects, including Fibrous Crescent (FC), Epithelial Crescent, Glomerulosclerosis, Necrosis, etc., as\nwell as a different time passed after glomeruli degradation and similarity of degraded glomeruli with\nother nephron components makes detection of unhealthy glomeruli extremely challenging.\n\nIn this competition participants are not asked to detect unhealthy glomeruli. Leah Scherschel, one of\nthe competition hosts, [pointed out](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/228993) that globally sclerotic glomeruli were excluded from the annotation .\n\nHowever, the level of injury is on a continuous scale, and there is a number of annotated glomeruli\nthat have defects. The figure (Fig 2) below indicates several examples of unhealthy glomeruli from\n[data.mendeley.com](https://data.mendeley.com/datasets/k7nvtgn2x6/3) with some of them being similar to annotated instances in the provided train data.\nIt is not clear what should be a fraction of fibrous tissue to stop considering a glomerulus to be healthy.\nAnother drastic example is annotation of FC glomeruli in *d488c759a* image from the test set and\nabsence of such annotation in other provided images. The uncertainty of including a particular\nglomerulus in the mask predicted by the model rather than detection of glomeruli (healthy + unhealthy)\nis the major factor degrading the model performance.\n","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 2: Several examples of unhealthy glomeruli taken from data.mendeley.com.\"\"\"\nplot_image(\n    \"../input/presentation-images/unhealthy_gloms.png\",\n    title,\n    (7, 7),\n    -0.1\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To mitigate the impact of unhealthy glomeruli detection on the model performance we have built a **2-\nclass model performing detection of both healthy and unhealthy glomeruli**. Given the large variety of\nglomeruli defects and insufficient expertise in the field, we decided to course-grain all the unhealthy glomeruli into a\nsingle class. We performed a hand annotation of the provided data to create masks for unhealthy\nglomeruli. In addition, we hand-labeled the external data from [data.mendeley.com](https://data.mendeley.com/datasets/k7nvtgn2x6/3) the same way. So, during training the model is less affected by the noise resulted by\nincluding/excluding a particular instance to the healthy glomeruli class annotated originally. The model\nemploys the information that an unhealthy glomerulus is also a glomerulus. Moreover, the ability of\nthe model to detect unhealthy glomeruli may be helpful for practical use cases. Meanwhile, the drastic\ndifference between the cross validation (CV) score of the model of 0.941 and 0.63 for healthy and\nunhealthy glomeruli, respectively, indicates that the task of detection of glomeruli with defects is\nsignificantly more complicated than the one considered in this challenge.\n\n\n## Missing annotation and AI guided labeling\n\nOne of the difficulties with hand annotation of data is that a human, despite having the ability to\ncorrectly treat difficult cases, sometimes may not pay enough attention and miss several instances.\nFor example, Figure 3 below indicates several glomeruli missed during annotation, but recognized\nby the model. Those missed masks negatively impact training, and, unfortunately, the model\nevaluation. More importantly, such mistakes can affect medical diagnostic and, therefore, human lives.","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 3: An example of missing annotation in 8242609fa image. Green color corresponds to correctly\npredicted glomeruli, while red color indicates glomeruli predicted by the model but not included\ninto the ground truth masks. Zoomed in regions with missed glomeruli are shown on the left.\"\"\"\nplot_image(\n    \"../input/presentation-images/missing_annotations.png\",\n    title,\n    (15, 15),\n    -0.1\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"One of the possible solutions is using a deep learning model working together with a human on a\nsample analysis and annotation. Then, the human can verify cases of disagreement between him and\nthe AI annotation to correct possible errors. Specifically, we used this strategy to correct several\nmistakes in the provided annotation of train data, as illustrated in Fig 3.","metadata":{}},{"cell_type":"markdown","source":"# Part 2: Solution Overview\n\nFor simplicity, we do not include the training code in the notebook. The code will be cleaned,\ndocumented, and made publicly available on GitHub: https://github.com/Optimox/HubMap\n\n## Augmentations & Data\n\n\n### Image sizes\n\n\nWe worked with several image resolutions and tile sizes (a tile is a cropped image from the original image used for training and inference for the model).\n\nInitial experiments were performed for tiles\nof size 256x256 extracted from images downsized by 4 times (resolution/4), which enabled fast training\nand rapid prototyping.\n\nThen we switched to 512x512 tiles size at resolution/4 to ensure that sufficient\nglomeruli surrounding is included into the input.\n\nIn addition, our final models were trained for tile size\nof 512x512 extracted at resolution/2 and 768x768 extracted at resolution/3.\n\nIn all cases the tiles were\ndynamically selected during training from preloaded downsized images into RAM.\n\n### Sampling strategy\n\nTo speed up the convergence, we used sampling strategies that automatically select interesting tiles\nbased on the tissue annotation or having visible glomeruli in them. Each of these methods outperformed\nrandom sampling. To make sure that the information about the areas not covered by the above\nsampling is also learnt, we only enforce the chosen strategy 90% of the time.\n\n### Augmentation strategy\n\nTo address the aforementioned data quality issues, we use aggressive data augmentation, which\nincludes:\n- Brightness and Contrast changes\n- RGB Shifting\n- Hue, Saturation and Value shifting\n- Color Jittering\n- Artificial blurring: Motion blur, Gaussian blur, and Defocus blur\n- CutMix and MixUp in some experiments, applied with 50% probability\n\nIn addition, we leverage invariance in the data: we randomly flip, rotate, shift, and scale the tiles.\nHowever, since rotating, scaling, and shifting an image introduces side effects at edges, we perform\naugmentation on 1.5x larger tiles and then take the center crop. Several examples of\naugmented images are illustrated below (Fig 4).","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 4: Examples of augmented images used during trianing (with corresponding annotations in red).\"\"\"\nplot_image(\n    \"../input/presentation-images/image_augmentations.png\",\n    title,\n    (20, 15),\n    -0.05\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## External data\n\nFor building our model in addition to the provided train data we utilized kidney histopathological images\nfrom several additional sources.\n\n- DATASET_A from [data.mendeley.com](https://data.mendeley.com/datasets/k7nvtgn2x6/3). It consists of 31 whole slide images that provides a significant variability and ensures better generalization. The size of the WSI range between 21651x10498 pixels and 49799 x 32359 pixels. We manually labeled this dataset with 2 class masks: healthy (detected in this challenge) and unhealthy glomeruli using QuPath program.\n\n- Kidney glomeruli-ROIs dataset from [zenodo](https://zenodo.org/record/4299694). The original annotation consists of two classes, similar to ones we used for DATASET_A. The annotations were missing a number of glomeruli (mostly at image edges), so we manually added them. Some of our models do not actually use this data.\n\n- 2 publicly available kidney PAS stained microscopy images from [the HubMAP portal](https://portal.hubmapconsortium.org) not included into train/test data and 5 images from public test set of this challenge. These images were annotated with pseudo labels generate by our model. In addition, we manually labeled unhealthy glomeruli in the images from the test set. Thus, the model could become familiar with artefacts precent in the test set.\n\n\n## Model architecture\n\nFor glomeruli segmentation we chose a U-Net like network architecture [[2]](https://arxiv.org/abs/1505.04597). This architecture consists of\nan encoder, creating a representation of extracted features at different levels, and a decoder which\ncombines the features and generates a prediction as a segmentation mask. The skip connections\nbetween the encoder and the decoder allow effective use of features from the intermediate convolutional\nlayers of the decoder, without a need for the information to go the full way through the entire model.\n\nIn our pipeline we use the commonly used EfficientNet encoders. They slightly outperformed networks from\nResNet family (e.g. ResNeXt50) in our experiments. We considered several options to improve the decoder:\n- Feature Pyramid Networks (FPN) skip connections [[3]](https://arxiv.org/abs/1612.03144)\n- Bottleneck Transformer central block [[4]](https://arxiv.org/abs/2101.11605) to expand the receptive field of the model\n\nHowever, since the problem considered in this competition is rather related to making a decision on\nwhether or not to include a particular glomerulus in the produced mask than to create of a highly\naccurate segmentation, the choice of the decoder architecture is not critical since the encoder takes\ncare about most of the decision task. In our experiments, we saw that even the naivest decoders\nachieve good performance, although they are slower to converge.\n\n## Training setup\nFor the training setup, we adopt practices acquired from previous competitions that we know work\nwell:\n- The cross-entropy loss is used as a loss function: it is simple but efficient. The Lovász loss [[5]](https://arxiv.org/abs/1705.08790) did not provide better performance in the considered task.\n- Our model predicts two glomeruli classes (healthy & unhealthy), we use a 0.2 weight for the unhealthy class.\n- Models are trained for at least 10000 iterations using the largest batch size that fits on a 2080Ti with half precision. Smaller models are trained for longer, especially if MixUp/CutMix augmentations are used.\n- The learning rate is linearly increased up to 0.001 during the first 500 iterations, and then decreased linearly to 0.\n\nOverall, we are able to reach a leaderboard score of 0.940 in 3 hours of training using a relatively\ncheap setup. We noticed that surprisingly, image resolution does not really play a noticeable role.\n\n# Part 3: Results\n\nThe validation scheme we used is a 5-fold split per image. The overall idea behind it is that we want\nthe models to generalize well to unseen images. We primary worked on optimization of our validation\ndice score: we assume this metric will correlate the best with the private leaderboard. Because one of\nthe images in the public dataset, *d488c759a*, has a number of Fibrous Crescent glomeruli annotated,\nwe quickly realized that this image made leaderboard unreliable and not representative. We observed a first\nleaderboard jump (on April 13) when our models started predicting fibrous crescent glomeruli.\nHowever, this jump was due to label ambiguity in the external data (the [publicly available labels](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/208972) for DATASET_A  include both healthy and unhealthy glomeruli as a single class), so we decided to focus\non optimizing the performance on the other test images excluding *d488c759a* (starting from April 30) as show in Fig 5.","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 5: Dice score evolution on CV and LB over time.\"\"\"\nplot_image(\n    \"../input/presentation-images/Cv_vs_LB.png\",\n    title,\n    (15, 15),\n    -0.05\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Performance assessment\n\nIn fact, the dice score is not ideal to assess how well our models are working. Segmentation labels\nare quite noisy and we believe it is more important to detect glomeruli than segment them. To this\nextent, we put emphasis on glomeruli level metrics. In our predictions, we characterize glomeruli using\nconnected components of the mask.\n\nAs an example, we report performances on *2f6ecfcdf* (see Fig 6):\n- The dice score is 0.9610\n- There are 160 glomeruli\n- 3 glomeruli are missed\n- There are 6 false positive\n- Detection score: 160 / (160 + 3 + 6) = 94.7%","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 6: Error cases on 2f6ecfcdf. Green = prediction, red = ground truth.\"\"\"\nplot_image(\n    \"../input/presentation-images/error_cases.png\",\n    title,\n    (7, 7),\n    -0.05\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"However, this metric is not robust to annotation noise. Among the few mistakes, it is sometimes hard for\nnon-expert to tell whether the model is wrong or if the label is missing, such as the *b2dc8411c* example (see Fig 7).\nOverall, to improve model performances, we spent a lot of time visualizing predictions since hand-crafted\nmetrics were not reliable enough.","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 7: b2dc8411c has 138 annotations, the model has no false negative but predicts 5 false positive.\"\"\"\nplot_image(\n    \"../input/presentation-images/false_positives.png\",\n    title,\n    (18, 5),\n    -0.1, \n    fontsize=18\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Limitations\n\nOne of the challenges of this competition is label noise, it is really difficult to evaluate models. As a\nresult, it is hard to robustly assess their performance, which in our opinion is a big issue for medical\nimaging tasks.\n\nStill, from the images above, we can identify a few choke points of our models:\n- Glomeruli predictions sometimes do not look round but have complex shapes (see images 2 and 4 in Fig 7)\n- Predictions have some artefacts\n- The model struggles with glomeruli with a large empty space in them\n- Despite our model being good at differentiating sclerotic from non-sclerotic glomeruli, it still predicts some sclerotic ones (as we mentioned earlier, the level of injury is on a continuous scale, and sometimes it may be difficult to distinguish the cases)\n\nWe believe post-processing can be the key to remove some of the flaws of the model, but could not\ndevelop a reliable strategy that consistently improves our validation score.\n\n## Confidence estimation\n\nAs mentioned above, glomeruli detection made more sense to use for assessment of model\nperformance. However, the model we used does not directly output a confidence score since we use\nsemantic segmentation instead of instance segmentation. As a proxy for this score, we use the\nmaximum pixel prediction over the glomeruli (see Fig 8). We noticed that this was a good estimation of how\nconfident the model was, and that low scores (<0.7) were often caused by hard cases.\nWe tried removing false positive using low confidence predictions, but it did not work consistently\nenough. However, this allowed us to be detect annotation mistakes more consistently.\n\nAlso, since our approach uses two classes, our model can also be used to provide a score for sclerotic\nglomeruli. This score can be used to assess the evolution of the disease, helping the practician to\ndifferentiate between an healthy and an unhealthy patient.\nOverall, a high confidence score indicates that the prediction is almost certainly correct. If a practician\nwere to manually verify all glomeruli, he/she would know that these ones are not worth spending a lot\nof time on. Then, the challenge is on the few (<10%) glomeruli with low scores, which can be visually\nchecked. This framework enables a speed-up of the tedious histology reviewing work, which is highly beneficial as we believe that the time of experts is very valuable.","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 8: Confidence scores example for b2dc8411c. Green = prediction, red = ground truth.\"\"\"\nplot_image(\n    \"../input/presentation-images/confidence_esimation.png\",\n    title,\n    (15, 15),\n    -0.1,\n    fontsize=18\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Part 4: Other applications and images\n\n## Histological slides of any kind\n\nThe method used by our team is suitable for any type of histological slide. It is straightforward to adapt\nit to any semantic segmentation task, provided there is data available. Then, achieving decent\nperformances is only a matter of tunning the hyperparameters. Furthermore, the pipeline also works\nwith multiclass problems, since we built the additional “unhealthy glomeruli” class.\n\n\n## Other imaging domains: Line-field confocal optical coherence tomography (LC-OCT)\n\n\nWith minor changes, our pipeline can be used with completely different images. In fact, only the data\naugmentation strategy is specific to histology. To illustrate this, we present results on the task of\nsegmenting nucleus cells in LC-OCT images. [LC-OCT](https://pubmed.ncbi.nlm.nih.gov/30353716/) is a new non-invasive medical imaging\ntechnology that allows to explore the skin in-vivo. The images are black and white which changes from\nthe histology, but it allows to retrieve almost the same information with the advantage of being non-invasive and allowing for 3D acquisitions (see Figure 9).","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 9: Visual comparison of LC-OCT and histology. Taken from Monier et al \"In vivo characterization of\nhealthy human skin with a novel, non-invasive imaging technique: line-field confocal optical\ncoherence tomography\".\"\"\"\nplot_image(\n    \"../input/presentation-images/monier_healthy_skin.png\",\n    title,\n    (15, 15),\n    -0.1\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Segmenting 2D LC-OCTs\n\nWe adapted our pipeline to the task of keratinocytes nuclei (KCs) segmentation. Keratinocytes contain a lot\nof information about the skin: some diseases and cancers affect keratinocytes. Segmenting them\nmanually is a long and tedious task, and with automated segmentation we hope to accelerate the\nscreening of patients and automatically provide the dermatologist with useful insights. In fact, once the\nsegmentations are generated (see Fig 10 and 11), one can generate quantitative metrics on top of them. For instance,\nassessing the number, size, and shape of the keratinocytes will definitely help diagnose diseased skin.\n\nThe problem differs a lot from the task of\nsegmenting glomeruli (more cells, smaller cells) but our pipeline adapts well to it.","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 10: 2D LC-OCT image example ('en coupe').\"\"\"\nplot_image(\n    \"../input/presentation-images/side_view_no_seg.png\",\n    title,\n    (20, 15),\n    -0.1\n)\n\ntitle = \"\"\"Fig 11: Segmentation of keratinocytes on 2D LC-OCT images ('en coupe').\"\"\"\nplot_image(\n    \"../input/presentation-images/side_view_seg.png\",\n    title,\n    (20, 15),\n    -0.1\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This pipeline also works with 'en face' LC-OCT images, we trained a second model for this other task (see Fig 12 and 13).","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 12: 2D LC-OCT image example ('en face').\"\"\"\nplot_image(\n    \"../input/presentation-images/top_view_no_seg.png\",\n    title,\n    (20, 15),\n    -0.1\n)\n\ntitle = \"\"\"Fig 13: Segmentation of keratinocytes on 2D LC-OCT images ('en face').\"\"\"\nplot_image(\n    \"../input/presentation-images/top_view_seg.png\",\n    title,\n    (20, 15),\n    -0.1\n)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Towards 3D segmentation of LC-OCT images\n\nAs mentioned above, LC-OCT allows to view 3D stacks, enabling a higher level of information than histology. Since keratinocytes are 3D as well, it makes sense to use our predictions at a 3D level. Using averaging of the predictions on the two axes (en coupe and en face), we are able to retrieve the 3D structure of cells (see Fig 14). ","metadata":{}},{"cell_type":"code","source":"title = \"\"\"Fig 14: Segmentation of keratinocytes on 3D LC-OCT image.\"\"\"\nplot_image(\n    \"../input/presentation-images/3D_KCs_seg.png\",\n    title,\n    (18, 15),\n    -0.1\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusions\n\nWe have presented our approach to the HuBMAP - Hacking the Kidney Competition. This approach is built on understanding the challenges behind the data:\n- It takes in consideration the link between healthy glomeruli and unhealthy ones (Sclerotic, Fibrous Crescent, etc.)  by predicting both into two different classes.\n- It is designed to be robust to noise, as we used aggressive augmentations.\n- We incorporate several external datasets in our pipeline and manually annotated them (using 2 classes of glomeruli) to ensure that our model is able to generalize well on new cases.\n\n\nOur model architecture is relatively simple, and the pipeline can be easily transferred to other very different segmentation tasks such as keratinocytes segmentation in LC-OCTs. Furthermore, it achieves competitive performances on the leaderboard, even though label noise makes it difficult to tell whether the results will translate well to the private leaderboard. \n\nWe also presented a framework to assess the model performance, using glomeruli level prediction. In fact, the image-level dice is too coarse to entirely capture the challenge of the problem. By leveraging true positive, false positive, and false negative rates at glomeruli level, we are able to understand the weaknesses of the model. Furthermore, our model is able to provide a confidence score estimation which could help practicians in their decision making.\n\nWe hope that our work will help to understand the relationships between glomeruli and the health of an individual. We also hope that a lot of people will benefit from the effort the hosts, the Kaggle team, the participants, and us put into the competition.","metadata":{}}]}