Hi there.
\nI have made 4 datasets and would like to share them. All datasets contain square, 256x256 images centred around an identified cell.
\nThe process to make the datasets is as follows:
\nStep 1: Run CellSegmentator tool (or use Tito's Public dataset of mask images, or use my public dataset containing updated train.csv with RLEs) to identify masks of all cells in all images.
\nStep 2: Identify the bounding box for each cell mask in all images
\nStep 3: Pad each bounding box to be square (use black space, pad content towards the centre)
\nStep 4: Resize each bounding box to be 256x256 (this is big… bigger than in the original image often)
\nStep 5: Open and stack RGBY channels into a single image and cut out the relevant squares surrounding all identified cells.
\nStep 6: Save these cellwise crops to a new folder and label accordingly.
Apologies in advance for the messed up folder structure of the datasets. It, at least, is consistent in its weirdness. If you need help parsing it please let me know and I will post a code snippet below.
\nCredit to @its7171 for his public mask dataset.
", "messages": [ { "id": 1186356, "postDate": "2021-02-04T19:18:46.600Z", "content": "Hi there.
\nI have made 4 datasets and would like to share them. All datasets contain square, 256x256 images centred around an identified cell.
\nThe process to make the datasets is as follows:
\nStep 1: Run CellSegmentator tool (or use Tito's Public dataset of mask images, or use my public dataset containing updated train.csv with RLEs) to identify masks of all cells in all images.
\nStep 2: Identify the bounding box for each cell mask in all images
\nStep 3: Pad each bounding box to be square (use black space, pad content towards the centre)
\nStep 4: Resize each bounding box to be 256x256 (this is big… bigger than in the original image often)
\nStep 5: Open and stack RGBY channels into a single image and cut out the relevant squares surrounding all identified cells.
\nStep 6: Save these cellwise crops to a new folder and label accordingly.
Apologies in advance for the messed up folder structure of the datasets. It, at least, is consistent in its weirdness. If you need help parsing it please let me know and I will post a code snippet below.
\nCredit to @its7171 for his public mask dataset.
", "rawMarkdown": "Hi there.\n\nI have made **4 datasets** and would like to share them. All datasets contain **square, 256x256 images centred around an identified cell**.\n- [**Red Channel (Microtubules)**](https://www.kaggle.com/dschettler8845/human-protein-atlas-red-cell-tile-dataset?rvi=1)\n- [**Green Channel (Protein of Interest)**](https://www.kaggle.com/dschettler8845/human-protein-atlas-green-cell-tile-dataset?rvi=1)\n- [**Blue Channel (Nucleus)**](https://www.kaggle.com/dschettler8845/human-protein-atlas-blue-cell-tile-dataset?rvi=1)\n- [**Yellow Channel (Endoplasmic Reticulum)**](https://www.kaggle.com/dschettler8845/human-protein-atlas-yellow-cell-tile-dataset?rvi=1)\n\nDid you only use the images with one label associated?
\nIf yes, you might have also some negatives wrongly categorized as a certain category?
You are 100% correct. This dataset is a first iteration of what will likely require many steps.
\nI recently took this dataset and applied manual heuristics to filter out the misclassified negative classes. It looks like between 30-50% of the cell-wise images are actually negative (assuming my manual heuristics are not garbage).
\nI'll be posting that dataset once I've had a chance to conduct more experimentation.
", "rawMarkdown": "You are 100% correct. This dataset is a first iteration of what will likely require many steps.\n\nI recently took this dataset and applied manual heuristics to filter out the misclassified negative classes. It looks like between 30-50% of the cell-wise images are actually negative (assuming my manual heuristics are not garbage).\n\nI'll be posting that dataset once I've had a chance to conduct more experimentation.", "votes": 1 } ] }, { "id": 1227386, "postDate": "2021-03-05T13:33:53.020Z", "content": "Hi. So for segmentation if you had used the HPA tool it would have given a binary output image right with 0 as the background and 1 as the foreground. Have you uploaded the output of the segmentation separately?
", "rawMarkdown": "Hi. So for segmentation if you had used the HPA tool it would have given a binary output image right with 0 as the background and 1 as the foreground. Have you uploaded the output of the segmentation separately?", "replies": [ { "id": 1227545, "postDate": "2021-03-05T16:23:39.803Z", "content": "No. The mask dataset that correlates to these datasets can be found here and was created by Tito.
\nIf you'd like to see an updated dataset (and corresponding mask information), please find the two links below. These datasets were generated using the Even Faster Cell Segmentation version of the CellSegmentator tool.
\n", "rawMarkdown": "No. The mask dataset that correlates to these datasets can be found [**here**](https://www.kaggle.com/its7171/hpa-mask) and was created by [**Tito**](https://www.kaggle.com/its7171).\n\n---\n\nIf you'd like to see an updated dataset (and corresponding mask information), please find the two links below. These datasets were generated using the [**Even Faster Cell Segmentation**](https://www.kaggle.com/samusram/even-faster-hpa-cell-segmentation) version of the CellSegmentator tool.\n- [**Link to RGB Dataset**](https://www.kaggle.com/dschettler8845/hpa-rgb-tilewise-dataset-v20)\n- [**Link to updated CSV dataset containing mask and bounding box information**](https://www.kaggle.com/dschettler8845/hpa-train-data-with-additional-metadata)", "votes": 1 }, { "id": 1227547, "postDate": "2021-03-05T16:28:27.813Z", "content": "Also, I believe that the HPA tool does not output a binary mask. It outputs a mask where each cell is masked by a single value. i.e. Cell 1 is masked by 1s, Cell 2 is masked by 2s, etc. This allows us to identify overlapping cells with more ease.
\nTo go from this to individual binary cell maks, you simply use something like this…
\n\n
# `all_masks` is the output I described above\n# `individual_masks` will be a list containing a binary mask for each cell\n\nindividual_masks = [\n np.where(all_masks==i, 1, 0) \\\n for i in range(1, all_masks.max()+1)\n]\nDid you only use the images with one label associated?
\nIf yes, you might have also some negatives wrongly categorized as a certain category?
You are 100% correct. This dataset is a first iteration of what will likely require many steps.
\nI recently took this dataset and applied manual heuristics to filter out the misclassified negative classes. It looks like between 30-50% of the cell-wise images are actually negative (assuming my manual heuristics are not garbage).
\nI'll be posting that dataset once I've had a chance to conduct more experimentation.
", "votes": 1, "replies": [] } ] }, { "id": 1227386, "author_name": "RSASHWIN", "author_url": "", "post_date": "2021-03-05T13:33:53.020000", "content": "Hi. So for segmentation if you had used the HPA tool it would have given a binary output image right with 0 as the background and 1 as the foreground. Have you uploaded the output of the segmentation separately?
", "votes": 0, "replies": [ { "id": 1227545, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-03-05T16:23:39.803000", "content": "No. The mask dataset that correlates to these datasets can be found here and was created by Tito.
\nIf you'd like to see an updated dataset (and corresponding mask information), please find the two links below. These datasets were generated using the Even Faster Cell Segmentation version of the CellSegmentator tool.
\n", "votes": 1, "replies": [] }, { "id": 1227547, "author_name": "Darien Schettler", "author_url": "", "post_date": "2021-03-05T16:28:27.813000", "content": "Also, I believe that the HPA tool does not output a binary mask. It outputs a mask where each cell is masked by a single value. i.e. Cell 1 is masked by 1s, Cell 2 is masked by 2s, etc. This allows us to identify overlapping cells with more ease.
\nTo go from this to individual binary cell maks, you simply use something like this…
\n\n
# `all_masks` is the output I described above\n# `individual_masks` will be a list containing a binary mask for each cell\n\nindividual_masks = [\n np.where(all_masks==i, 1, 0) \\\n for i in range(1, all_masks.max()+1)\n]\n