![\"https://i.ibb.co/BczvV19/Selection-059.png\"](\"https://i.ibb.co/BczvV19/Selection-059.png\")
if you have good style transfer model, please put the link here!
\ne.g. the above is generated by
\n[paper] StyTr^2 : Image Style Transfer with Transformers（CVPR2022）
\nhttps://github.com/diyiiyiii/StyTR-2
i want to create a thread to benchmark style transfer augmentation results. e.g.
\n| style-transfer method | \nlocal CV | \nLB-hubmap | \n
|---|---|---|
| xxx | \n0.777 | \n0.5x | \n
if you have good style transfer model, please put the link here!
\ne.g. the above is generated by
\n[paper] StyTr^2 : Image Style Transfer with Transformers（CVPR2022）
\nhttps://github.com/diyiiyiii/StyTR-2
i want to create a thread to benchmark style transfer augmentation results. e.g.
\n| style-transfer method | \nlocal CV | \nLB-hubmap | \n
|---|---|---|
| xxx | \n0.777 | \n0.5x | \n
Thanks for sharing again. I just started researching adapting stains as well. I think the proper way of doing this is selecting a target image randomly from a pool of target images and augment domain image on the fly. It would probably slow down the training process though. What's your plan?
", "rawMarkdown": "Thanks for sharing again. I just started researching adapting stains as well. I think the proper way of doing this is selecting a target image randomly from a pool of target images and augment domain image on the fly. It would probably slow down the training process though. What's your plan?", "votes": null }, { "id": "1890838", "postDate": "08/09/2022 04:57:31", "content": "i would do by organ.
\ni think i will focus on spleen first and then large-intestine.
\nthere will be a lot of probing.
\" proper way of doing this is selecting a target image \"
\nyou can make sure your model do well for \"all\" your traget images first. then it is one time training
", "rawMarkdown": "i would do by organ.\ni think i will focus on spleen first and then large-intestine.\nthere will be a lot of probing.\n\n---\n\" proper way of doing this is selecting a target image \" \n\nyou can make sure your model do well for \"all\" your traget images first. then it is one time training", "votes": null }, { "id": "1891808", "postDate": "08/09/2022 16:54:24", "content": "there is some problem for me to upload large images to kaggle dataset. I will solve that later.
\nmeanwhile I put some spleen whole slide images at my public google drive as a temporary solution
\nhttps://drive.google.com/drive/folders/102b93H904DjrCQmTEu3ZnuJuS0Qnzllr?usp=sharing
License
\n[1] GTEX:
[2] HPA:
\n[3] histologyslides
\neach jpg file has a um=xx stage in naming, which is the pixel size in um. \"est-um\" means estimated.
\n![\"https://i.ibb.co/GVW7Z34/Selection-091.png\"](\"https://i.ibb.co/GVW7Z34/Selection-091.png\")
\n(shown above are slides at same common scale)
not all slides can be use for your solution because of the licenses.
\nplease check it yourselves.
nevertheless, you are good for your internal testing, etc
", "rawMarkdown": "there is some problem for me to upload large images to kaggle dataset. I will solve that later.\nmeanwhile I put some spleen whole slide images at my public google drive as a temporary solution \nhttps://drive.google.com/drive/folders/102b93H904DjrCQmTEu3ZnuJuS0Qnzllr?usp=sharing\n\nLicense\n[1] GTEX: \n- not check\n\n[2] HPA: \n- not check\n\n[3] histologyslides\n- https://histology.medicine.umich.edu/full-slide-list\nCreative Commons Attribution-Noncommercial-Share Alike 3.0 License \n\n\neach jpg file has a um=xx stage in naming, which is the pixel size in um. \"est-um\" means estimated.\n\n\n\n(shown above are slides at same common scale)\n\nnot all slides can be use for your solution because of the licenses. \nplease check it yourselves.\n\nnevertheless, you are good for your internal testing, etc", "votes": null }, { "id": "1895906", "postDate": "08/12/2022 12:32:30", "content": "i have updated the goggle drive access
\n![\"https://i.ibb.co/rvkZpR9/Selection-076.png\"](\"https://i.ibb.co/rvkZpR9/Selection-076.png\")
@hengck23 please let me know if you continue to hit errors while uploading data.
", "rawMarkdown": "hengck23 please let me know if you continue to hit errors while uploading data.", "votes": null }, { "id": "1896810", "postDate": "08/13/2022 07:02:51", "content": "Here are some images that I generated with DALL-E 2. I probably should have queried this in a better way. Adrenal glands aren't generated on those images.
\n![\"gen\"](\"https://i.ibb.co/SQ2QTdv/Screenshot-from-2022-08-13-09-58-57.png\")
Thanks for your sharing.
", "rawMarkdown": "Thanks for your sharing.", "votes": null }, { "id": "1900751", "postDate": "08/16/2022 08:24:22", "content": "the dataset is now out at kaggle link: https://www.kaggle.com/datasets/hengck23/kaggle-hubmap-hpa-competition-external-v1/settings
", "rawMarkdown": "the dataset is now out at kaggle link: https://www.kaggle.com/datasets/hengck23/kaggle-hubmap-hpa-competition-external-v1/settings", "votes": null }, { "id": "1900755", "postDate": "08/16/2022 08:25:43", "content": "@sohier thanks for the message. it is due to poor wifi connection and connection timeout at my side. I have resolved the issue and made the upload at kaggle
\nThanks again!
", "rawMarkdown": "sohier thanks for the message. it is due to poor wifi connection and connection timeout at my side. I have resolved the issue and made the upload at kaggle\n\nThanks again!", "votes": null }, { "id": "1903266", "postDate": "08/17/2022 08:46:24", "content": "how to check domain shift
\n![\"https://i.ibb.co/CmwXFmx/Selection-068.png\"](\"https://i.ibb.co/CmwXFmx/Selection-068.png\")
![\"https://i.ibb.co/jrQvFxD/Selection-002.png\"](\"https://i.ibb.co/jrQvFxD/Selection-002.png\")
many kagglers are using stain tools to color color of tissue images for augmentation.
\nthey are missing out on the other more important aspect : texture (i.e. morphology)
the above image is created using Top Hat Transform
\nfrom staintools.stain_extraction.macenko_stain_extractor import MacenkoStainExtractor\nfrom staintools.stain_extraction.vahadane_stain_extractor import VahadaneStainExtractor\nfrom staintools.tissue_masks.luminosity_threshold_tissue_locator import LuminosityThresholdTissueLocator\nfrom staintools.utils.get_concentrations import get_concentrations\n\n\nstain_matrix = extractor.get_stain_matrix(image)\nsource_concentration = get_concentrations(image, stain_matrix)\ns0 = source_concentration[:,0].reshape(h,w)\ns1 = source_concentration[:,1].reshape(h,w)\n\n# Top Hat Transform\nkernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(15,15))\ntop_hat = cv2.morphologyEx(s0, cv2.MORPH_TOPHAT, kernel)# Black Hat Transform\nblack_hat = cv2.morphologyEx(s0, cv2.MORPH_BLACKHAT, kernel)\ns0_aug = s0 + top_hat - black_hat\n\n\ns0s1 = np.dstack([s0,s1_aug])\ns0s1_flat = s0s1.reshape(-1,2)\n\naugment_flat = 255 * np.exp(-1 * np.dot(s0s1_flat, stain_matrix))\naugment = augment_flat.reshape((h,w,3))\nbetter solution:
\ncreate man many augmented images. (manual generator)
\ntrain a discriminator the discriminate augmented images from the target H&E collected images.
\n(this can be done online on hidden test or offline).
then you know which augmentation paramters to use.
\nthere are also differential color transform, or differential morphology deep network.
\nthere are also GAN that disentangle color and shape/texture.
\nASK google for the link :)
", "rawMarkdown": "better solution:\n\ncreate man many augmented images. (manual generator)\ntrain a discriminator the discriminate augmented images from the target H&E collected images.\n(this can be done online on hidden test or offline).\n\nthen you know which augmentation paramters to use.\n\nthere are also differential color transform, or differential morphology deep network.\n\nthere are also GAN that disentangle color and shape/texture.\n\nASK google for the link :)", "votes": null }, { "id": "1908781", "postDate": "08/22/2022 02:28:08", "content": "[paper] Enhanced Cycle-Consistent Generative Adversarial Network for Color Normalization of H&E Stained Images
", "rawMarkdown": "[paper] Enhanced Cycle-Consistent Generative Adversarial Network for Color Normalization of H&E Stained Images", "votes": null }, { "id": "1916451", "postDate": "08/27/2022 22:46:01", "content": "s1_aug ?
", "rawMarkdown": "# s0s1 = np.dstack([s0,s1_aug])\ns1_aug ?", "votes": null } ], "comments": [ { "id": 1890830, "author_name": "gunesevitan", "author_url": "", "post_date": "08/09/2022 04:49:40", "content": "Thanks for sharing again. I just started researching adapting stains as well. I think the proper way of doing this is selecting a target image randomly from a pool of target images and augment domain image on the fly. It would probably slow down the training process though. What's your plan?
", "votes": null, "replies": [ { "id": 1890838, "author_name": "hengck23", "author_url": "", "post_date": "08/09/2022 04:57:31", "content": "i would do by organ.
\ni think i will focus on spleen first and then large-intestine.
\nthere will be a lot of probing.
\" proper way of doing this is selecting a target image \"
\nyou can make sure your model do well for \"all\" your traget images first. then it is one time training
", "votes": null, "replies": [] } ] }, { "id": 1891808, "author_name": "hengck23", "author_url": "", "post_date": "08/09/2022 16:54:24", "content": "there is some problem for me to upload large images to kaggle dataset. I will solve that later.
\nmeanwhile I put some spleen whole slide images at my public google drive as a temporary solution
\nhttps://drive.google.com/drive/folders/102b93H904DjrCQmTEu3ZnuJuS0Qnzllr?usp=sharing
License
\n[1] GTEX:
[2] HPA:
\n[3] histologyslides
\neach jpg file has a um=xx stage in naming, which is the pixel size in um. \"est-um\" means estimated.
\n
\n(shown above are slides at same common scale)
not all slides can be use for your solution because of the licenses.
\nplease check it yourselves.
nevertheless, you are good for your internal testing, etc
", "votes": null, "replies": [ { "id": 1895906, "author_name": "hengck23", "author_url": "", "post_date": "08/12/2022 12:32:30", "content": "i have updated the goggle drive access
\n
@hengck23 please let me know if you continue to hit errors while uploading data.
", "votes": null, "replies": [] }, { "id": 1900751, "author_name": "hengck23", "author_url": "", "post_date": "08/16/2022 08:24:22", "content": "the dataset is now out at kaggle link: https://www.kaggle.com/datasets/hengck23/kaggle-hubmap-hpa-competition-external-v1/settings
", "votes": null, "replies": [] }, { "id": 1900755, "author_name": "hengck23", "author_url": "", "post_date": "08/16/2022 08:25:43", "content": "@sohier thanks for the message. it is due to poor wifi connection and connection timeout at my side. I have resolved the issue and made the upload at kaggle
\nThanks again!
", "votes": null, "replies": [] } ] }, { "id": 1896810, "author_name": "gunesevitan", "author_url": "", "post_date": "08/13/2022 07:02:51", "content": "Here are some images that I generated with DALL-E 2. I probably should have queried this in a better way. Adrenal glands aren't generated on those images.
\n
Thanks for your sharing.
", "votes": null, "replies": [] }, { "id": 1903266, "author_name": "hengck23", "author_url": "", "post_date": "08/17/2022 08:46:24", "content": "how to check domain shift
\n
many kagglers are using stain tools to color color of tissue images for augmentation.
\nthey are missing out on the other more important aspect : texture (i.e. morphology)
the above image is created using Top Hat Transform
\nfrom staintools.stain_extraction.macenko_stain_extractor import MacenkoStainExtractor\nfrom staintools.stain_extraction.vahadane_stain_extractor import VahadaneStainExtractor\nfrom staintools.tissue_masks.luminosity_threshold_tissue_locator import LuminosityThresholdTissueLocator\nfrom staintools.utils.get_concentrations import get_concentrations\n\n\nstain_matrix = extractor.get_stain_matrix(image)\nsource_concentration = get_concentrations(image, stain_matrix)\ns0 = source_concentration[:,0].reshape(h,w)\ns1 = source_concentration[:,1].reshape(h,w)\n\n# Top Hat Transform\nkernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(15,15))\ntop_hat = cv2.morphologyEx(s0, cv2.MORPH_TOPHAT, kernel)# Black Hat Transform\nblack_hat = cv2.morphologyEx(s0, cv2.MORPH_BLACKHAT, kernel)\ns0_aug = s0 + top_hat - black_hat\n\n\ns0s1 = np.dstack([s0,s1_aug])\ns0s1_flat = s0s1.reshape(-1,2)\n\naugment_flat = 255 * np.exp(-1 * np.dot(s0s1_flat, stain_matrix))\naugment = augment_flat.reshape((h,w,3))\nbetter solution:
\ncreate man many augmented images. (manual generator)
\ntrain a discriminator the discriminate augmented images from the target H&E collected images.
\n(this can be done online on hidden test or offline).
then you know which augmentation paramters to use.
\nthere are also differential color transform, or differential morphology deep network.
\nthere are also GAN that disentangle color and shape/texture.
\nASK google for the link :)
", "votes": null, "replies": [] }, { "id": 1916451, "author_name": "rarun2596", "author_url": "", "post_date": "08/27/2022 22:46:01", "content": "s1_aug ?
", "votes": null, "replies": [] } ] }, { "id": 1908781, "author_name": "gray98", "author_url": "", "post_date": "08/22/2022 02:28:08", "content": "[paper] Enhanced Cycle-Consistent Generative Adversarial Network for Color Normalization of H&E Stained Images
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1890811": "\n\nif you have good style transfer model, please put the link here!\ne.g. the above is generated by \n[paper] StyTr^2 : Image Style Transfer with Transformers（CVPR2022）\nhttps://github.com/diyiiyiii/StyTR-2 \n\ni want to create a thread to benchmark style transfer augmentation results. e.g.\n| style-transfer method |local CV |LB-hubmap |\n| --- | --- |--- |\n| xxx |0.777 |0.5x|", "1890830": "Thanks for sharing again. I just started researching adapting stains as well. I think the proper way of doing this is selecting a target image randomly from a pool of target images and augment domain image on the fly. It would probably slow down the training process though. What's your plan?", "1890838": "i would do by organ.\ni think i will focus on spleen first and then large-intestine.\nthere will be a lot of probing.\n\n---\n\" proper way of doing this is selecting a target image \" \n\nyou can make sure your model do well for \"all\" your traget images first. then it is one time training", "1891808": "there is some problem for me to upload large images to kaggle dataset. I will solve that later.\nmeanwhile I put some spleen whole slide images at my public google drive as a temporary solution \nhttps://drive.google.com/drive/folders/102b93H904DjrCQmTEu3ZnuJuS0Qnzllr?usp=sharing\n\nLicense\n[1] GTEX: \n- not check\n\n[2] HPA: \n- not check\n\n[3] histologyslides\n- https://histology.medicine.umich.edu/full-slide-list\nCreative Commons Attribution-Noncommercial-Share Alike 3.0 License \n\n\neach jpg file has a um=xx stage in naming, which is the pixel size in um. \"est-um\" means estimated.\n\n\n\n(shown above are slides at same common scale)\n\nnot all slides can be use for your solution because of the licenses. \nplease check it yourselves.\n\nnevertheless, you are good for your internal testing, etc", "1895906": "i have updated the goggle drive access\n", "1896398": "hengck23 please let me know if you continue to hit errors while uploading data.", "1896810": "Here are some images that I generated with DALL-E 2. I probably should have queried this in a better way. Adrenal glands aren't generated on those images.\n\n", "1900410": "Thanks for your sharing.", "1900751": "the dataset is now out at kaggle link: https://www.kaggle.com/datasets/hengck23/kaggle-hubmap-hpa-competition-external-v1/settings", "1900755": "sohier thanks for the message. it is due to poor wifi connection and connection timeout at my side. I have resolved the issue and made the upload at kaggle\n\nThanks again!", "1903266": "how to check domain shift\n", "1905422": "\n\nmany kagglers are using stain tools to color color of tissue images for augmentation.\nthey are missing out on the other more important aspect : texture (i.e. morphology)\n\nthe above image is created using Top Hat Transform\n\n```\nfrom staintools.stain_extraction.macenko_stain_extractor import MacenkoStainExtractor\nfrom staintools.stain_extraction.vahadane_stain_extractor import VahadaneStainExtractor\nfrom staintools.tissue_masks.luminosity_threshold_tissue_locator import LuminosityThresholdTissueLocator\nfrom staintools.utils.get_concentrations import get_concentrations\n\n\nstain_matrix = extractor.get_stain_matrix(image)\nsource_concentration = get_concentrations(image, stain_matrix)\ns0 = source_concentration[:,0].reshape(h,w)\ns1 = source_concentration[:,1].reshape(h,w)\n\n# Top Hat Transform\nkernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(15,15))\ntop_hat = cv2.morphologyEx(s0, cv2.MORPH_TOPHAT, kernel)# Black Hat Transform\nblack_hat = cv2.morphologyEx(s0, cv2.MORPH_BLACKHAT, kernel)\ns0_aug = s0 + top_hat - black_hat\n\n\ns0s1 = np.dstack([s0,s1_aug])\ns0s1_flat = s0s1.reshape(-1,2)\n\naugment_flat = 255 * np.exp(-1 * np.dot(s0s1_flat, stain_matrix))\naugment = augment_flat.reshape((h,w,3))\n\n```", "1905425": "better solution:\n\ncreate man many augmented images. (manual generator)\ntrain a discriminator the discriminate augmented images from the target H&E collected images.\n(this can be done online on hidden test or offline).\n\nthen you know which augmentation paramters to use.\n\nthere are also differential color transform, or differential morphology deep network.\n\nthere are also GAN that disentangle color and shape/texture.\n\nASK google for the link :)", "1908781": "[paper] Enhanced Cycle-Consistent Generative Adversarial Network for Color Normalization of H&E Stained Images", "1916451": "# s0s1 = np.dstack([s0,s1_aug])\ns1_aug ?" }, "source": "meta" }