![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F104d4bb9442a76c1fdc74102c68c8f90%2FSelection_027.png?generation=1607024236601748&alt=media\")
we can get kidney slide images from HuBMAP website as shown below.
\nI think current deep learning is matured enough for self / semi-supervised learning.
\ni.e. i want to design a model that can learn labels automatically.
Are we allowed to use such data as external data for making the model in normal Kaggle prize?
If we are not allowed to use the data for training, how about using this data only for analysis of our model for Judge Prize. i.e. further visualization, evaluation, etc …
we can get kidney slide images from HuBMAP website as shown below.
\nI think current deep learning is matured enough for self / semi-supervised learning.
\ni.e. i want to design a model that can learn labels automatically.
Are we allowed to use such data as external data for making the model in normal Kaggle prize?
If we are not allowed to use the data for training, how about using this data only for analysis of our model for Judge Prize. i.e. further visualization, evaluation, etc …
I think those images are in the private test set
\nYou can probe the private lb with this code:
If the score is 0.0, then the image is in the private test set.
\nIf the result is an error, then the image is not found.
DIR_INPUT = '/kaggle/input/hubmap-kidney-segmentation'\nPROBE = np.array([\n [184, 177, ... , 113, 90],\n [rotated 90],\n [...]\n])\n\nsub_df = pd.read_csv(f'{DIR_INPUT}/sample_submission.csv')\n\nif sub_df.shape[0] > 5:\n\n found = False\n\n for i, row in sub_df.iterrows():\n image = skimage.io.imread(f\"{DIR_INPUT}/test/{row['id']}.tiff\")\n\n if len(image.shape) == 5:\n image = image.squeeze()\n\n # I cut from this position\n for probe in PROBE:\n if np.array_equal(image[20000:20050, 20000:20001, 0].flatten(), probe):\n found = True\n break\n\n # No luck, it seems there is no leak.\n if not found:\n # Force exit\n sys.exit()\n else:\n # If we found the sample in one of the prvate test image\n # Then our score should be 0 on the public LB\n sub_df.to_csv('submission.csv', index=False)\n\nelse:\n # This is for committing only.\n sub_df.to_csv('submission.csv', index=False)\nSo much for the hidden test set I guess, that's unfortunate. Thanks for pointing this out though
", "rawMarkdown": "So much for the hidden test set I guess, that's unfortunate. Thanks for pointing this out though", "votes": null }, { "id": "1101833", "postDate": "12/04/2020 10:04:42", "content": "This is a huge deal, because as there are 7 images on the private test set ( 7 / (5 + 7) = 0.58 which is the private proportion) , the full private can easily be recovered.
\nThen inference can be made 100% offline.
", "rawMarkdown": "This is a huge deal, because as there are 7 images on the private test set ( 7 / (5 + 7) = 0.58 which is the private proportion) , the full private can easily be recovered.\n\nThen inference can be made 100% offline.", "votes": null }, { "id": "1101836", "postDate": "12/04/2020 10:07:38", "content": "Poke @philculliton @leahscherschel @inversion @addisonhoward
", "rawMarkdown": "Poke @philculliton @leahscherschel @inversion @addisonhoward", "votes": null }, { "id": "1101871", "postDate": "12/04/2020 10:55:01", "content": "I agree, it is a huge deal. I think I've found 9 additional images, but I only tested one. I am not sure about the others. I sent a message to kaggle a weak ago, but they did not respond. Probably because of the thanksgiving (I sent it last Thursday)
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F864684%2Feb437ccafdd12c5d3eb498029d80f4d9%2Fprobe.png?generation=1607079215999839&alt=media\")
from https://www.kaggle.com/c/hubmap-kidney-segmentation/overview/judges-prize, it is mentioned,
\n\"Did the team provide insights that would be useful for generating reference glomeruli for inclusion into a Human Reference Atlas?\"
\nthat is why I am thinking for point (2) to use HuBMAP images to show more analysis results for the judges-prize. HuBMAP images may not (or may) to be used for training at all.
", "rawMarkdown": "from https://www.kaggle.com/c/hubmap-kidney-segmentation/overview/judges-prize, it is mentioned, \n\n\"Did the team provide insights that would be useful for generating reference glomeruli for inclusion into a Human Reference Atlas?\"\n\nthat is why I am thinking for point (2) to use HuBMAP images to show more analysis results for the judges-prize. HuBMAP images may not (or may) to be used for training at all.", "votes": null }, { "id": "1102192", "postDate": "12/04/2020 17:15:46", "content": "Hi all,
\nThanks for raising this. Please know we're investigating further.
", "rawMarkdown": "Hi all,\n\nThanks for raising this. Please know we're investigating further.", "votes": null }, { "id": "1102816", "postDate": "12/05/2020 10:49:42", "content": "It's time kaggle team to completely disable using private data during competition. And use only 2 stage to predict on private data. An addition reduce inference time to 1h or maybe 30m
", "rawMarkdown": "It's time kaggle team to completely disable using private data during competition. And use only 2 stage to predict on private data. An addition reduce inference time to 1h or maybe 30m", "votes": null }, { "id": "1103012", "postDate": "12/05/2020 15:06:57", "content": "Yes, as noted, this site does include the private test images. We are assessing impact for the competition and will let the community know how we choose to proceed with the test data.
", "rawMarkdown": "Yes, as noted, this site does include the private test images. We are assessing impact for the competition and will let the community know how we choose to proceed with the test data.", "votes": null }, { "id": "1106467", "postDate": "12/08/2020 21:26:47", "content": "One oddity is the afa5e8098.tiff file in the public dataset.
I found the original file on the HuBMAP portal site, but it was reasonably clear, without the black shadows that @hengck23 had pointed out in this thread (And also think a little bit differently).
\nhttps://portal.hubmapconsortium.org/browse/dataset/65dffa77af3430412ceb873e27660d40
original file
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F479538%2F8c340398107edbea4537638b778cc965%2FHBM339.XBLJ.842.png?generation=1607462331528251&alt=media\")
kaggle public dataset
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F479538%2F859e94c2b45d0332db3a2d846918906e%2Fafa5e8098.jpg?generation=1607462453018747&alt=media\")
BTW, if we know which private datasets are and are allowed to do hand labeling of the external data (above HuBMAP datasets), we could easily get a higher score by training the model on precisely hand labeled private datasets.
\nIf that happens, I think this competition would be divorced from its original purpose and would be close to meaningless.
", "rawMarkdown": "One oddity is the `afa5e8098.tiff` file in the public dataset.\n\nI found the original file on the HuBMAP portal site, but it was reasonably clear, without the black shadows that @hengck23 had pointed out in [this thread](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200955#1104643) (And also think a little bit differently).\nhttps://portal.hubmapconsortium.org/browse/dataset/65dffa77af3430412ceb873e27660d40\n\n**original file**\n\n\n\n**kaggle public dataset**\n\n\n\n\nBTW, if we know which private datasets are and are allowed to do hand labeling of the external data (above HuBMAP datasets), we could easily get a higher score by training the model on precisely hand labeled private datasets.\n\nIf that happens, I think this competition would be divorced from its original purpose and would be close to meaningless.", "votes": null }, { "id": "1106679", "postDate": "12/09/2020 03:15:17", "content": "Thanks for sharing. Do you think the images were augmented ?
\nTraining on hand-labeled test set, I guess, is not allowed and the winning teams will be asked to reproduce the results.
\nBut, you are right - those who are with doctors by their side having labeled the test set will get an unfair (or fair) advantage.
\n\nDo you think the images were augmented ?
\n
I don't know, but it looks to me like the above Kaggle dataset has been given some kind of distortion. Even with private data, Kaggle may dare to do some data augmentation. But this is my guess.
\n\n\nTraining on hand-labeled test set, I guess, is not allowed and the winning teams will be asked to reproduce the results.
\n
\n But, you are right - those who are with doctors by their side having labeled the test set will get an unfair (or fair) advantage.
Some people said that hand labeling of external data is possible, so in this case, as long as the private data is external data, it is possible to hand label it.
\nAs you said, I hope that the host will make sure the rules to make the competition fair and meaningful.
Supplement
\nYou can easily find the training images and public data images used in this competition by checking the checkboxes below at the site below.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F479538%2F8358f420b04a0a1b37acdfbfef6d9fd7%2FHuBMAP%20datasets.png?generation=1607486741182643&alt=media\")
hmm … this looks familiar
\njson annotations must follow QuPath's json export format:
\nhttps://qupath.github.io/
https://github.com/pjl54/WSI_handling
\n[\n {\n \"type\": \"Feature\",\n \"id\": \"PathAnnotationObject\",\n \"geometry\": {\n \"type\": \"Polygon\",\n \"coordinates\": [\n [\n [76793.51, 4613.02],\n [76651.56, 4684],\n [76580.59, 4684],\n [76580.59, 4754.97] \n ]\n ]\n },\n \"properties\": {\n \"classification\": {\n \"name\": \"Tumor\",\n \"colorRGB\": -3670016\n },\n \"isLocked\": true,\n \"measurements\": []\n }\n]\nhello, frog brother, can we use the external data for pseudo label training?
", "rawMarkdown": "hello, frog brother, can we use the external data for pseudo label training?", "votes": null }, { "id": "1122314", "postDate": "12/22/2020 10:54:12", "content": "Hubmap uses cytokit (https://github.com/hammerlab/cytokit) to preprocess the data.
\nIt is possible that some training data are unprocessed images or partially processed ones. The pipeline to create the processed images that can be downloaded on the hubmap website can also be found on https://github.com/hubmapconsortium/codex-pipeline/blob/master/bin/create_cytokit_config.py
", "rawMarkdown": "Hubmap uses cytokit (https://github.com/hammerlab/cytokit) to preprocess the data. \n\nIt is possible that some training data are unprocessed images or partially processed ones. The pipeline to create the processed images that can be downloaded on the hubmap website can also be found on https://github.com/hubmapconsortium/codex-pipeline/blob/master/bin/create_cytokit_config.py", "votes": null }, { "id": "1130369", "postDate": "12/29/2020 01:09:08", "content": "Any updates on this and the annotation issues?
", "rawMarkdown": "Any updates on this and the annotation issues?", "votes": null }, { "id": "1131558", "postDate": "12/29/2020 20:04:43", "content": "This is a comparing table with webpage content
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4367831%2F3c3e111ff7e6b98132a0aaba0db8ffae%2FSelection_044.png?generation=1609272193419693&alt=media\")
@bessenyeiszilrd what does it mean by <<< tricky?
It means that it has some unusual content such as VAN0003-LK-32-21-PAS_registered has unusual white square in the middle:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4367831%2F28c04cf5c3c665cf75a58ca175fb79b0%2FSelection_045.png?generation=1609274301650138&alt=media\")
thanks for the explanation
", "rawMarkdown": "thanks for the explanation", "votes": null } ], "comments": [ { "id": 1101458, "author_name": "pestipeti", "author_url": "", "post_date": "12/03/2020 23:17:36", "content": "I think those images are in the private test set
\nYou can probe the private lb with this code:
If the score is 0.0, then the image is in the private test set.
\nIf the result is an error, then the image is not found.
DIR_INPUT = '/kaggle/input/hubmap-kidney-segmentation'\nPROBE = np.array([\n [184, 177, ... , 113, 90],\n [rotated 90],\n [...]\n])\n\nsub_df = pd.read_csv(f'{DIR_INPUT}/sample_submission.csv')\n\nif sub_df.shape[0] > 5:\n\n found = False\n\n for i, row in sub_df.iterrows():\n image = skimage.io.imread(f\"{DIR_INPUT}/test/{row['id']}.tiff\")\n\n if len(image.shape) == 5:\n image = image.squeeze()\n\n # I cut from this position\n for probe in PROBE:\n if np.array_equal(image[20000:20050, 20000:20001, 0].flatten(), probe):\n found = True\n break\n\n # No luck, it seems there is no leak.\n if not found:\n # Force exit\n sys.exit()\n else:\n # If we found the sample in one of the prvate test image\n # Then our score should be 0 on the public LB\n sub_df.to_csv('submission.csv', index=False)\n\nelse:\n # This is for committing only.\n sub_df.to_csv('submission.csv', index=False)\nSo much for the hidden test set I guess, that's unfortunate. Thanks for pointing this out though
", "votes": null, "replies": [] }, { "id": 1101833, "author_name": "theoviel", "author_url": "", "post_date": "12/04/2020 10:04:42", "content": "This is a huge deal, because as there are 7 images on the private test set ( 7 / (5 + 7) = 0.58 which is the private proportion) , the full private can easily be recovered.
\nThen inference can be made 100% offline.
", "votes": null, "replies": [] }, { "id": 1101871, "author_name": "pestipeti", "author_url": "", "post_date": "12/04/2020 10:55:01", "content": "I agree, it is a huge deal. I think I've found 9 additional images, but I only tested one. I am not sure about the others. I sent a message to kaggle a weak ago, but they did not respond. Probably because of the thanksgiving (I sent it last Thursday)
\n
Poke @philculliton @leahscherschel @inversion @addisonhoward
", "votes": null, "replies": [] }, { "id": 1102020, "author_name": "hengck23", "author_url": "", "post_date": "12/04/2020 13:43:58", "content": "from https://www.kaggle.com/c/hubmap-kidney-segmentation/overview/judges-prize, it is mentioned,
\n\"Did the team provide insights that would be useful for generating reference glomeruli for inclusion into a Human Reference Atlas?\"
\nthat is why I am thinking for point (2) to use HuBMAP images to show more analysis results for the judges-prize. HuBMAP images may not (or may) to be used for training at all.
", "votes": null, "replies": [] }, { "id": 1102192, "author_name": "addisonhoward", "author_url": "", "post_date": "12/04/2020 17:15:46", "content": "Hi all,
\nThanks for raising this. Please know we're investigating further.
", "votes": null, "replies": [ { "id": 1103012, "author_name": "leahscherschel", "author_url": "", "post_date": "12/05/2020 15:06:57", "content": "Yes, as noted, this site does include the private test images. We are assessing impact for the competition and will let the community know how we choose to proceed with the test data.
", "votes": null, "replies": [] }, { "id": 1130369, "author_name": "jihangz", "author_url": "", "post_date": "12/29/2020 01:09:08", "content": "Any updates on this and the annotation issues?
", "votes": null, "replies": [] } ] }, { "id": 1102816, "author_name": "donchuk", "author_url": "", "post_date": "12/05/2020 10:49:42", "content": "It's time kaggle team to completely disable using private data during competition. And use only 2 stage to predict on private data. An addition reduce inference time to 1h or maybe 30m
", "votes": null, "replies": [] }, { "id": 1106467, "author_name": "maxwell110", "author_url": "", "post_date": "12/08/2020 21:26:47", "content": "One oddity is the afa5e8098.tiff file in the public dataset.
I found the original file on the HuBMAP portal site, but it was reasonably clear, without the black shadows that @hengck23 had pointed out in this thread (And also think a little bit differently).
\nhttps://portal.hubmapconsortium.org/browse/dataset/65dffa77af3430412ceb873e27660d40
original file
\n
kaggle public dataset
\n
BTW, if we know which private datasets are and are allowed to do hand labeling of the external data (above HuBMAP datasets), we could easily get a higher score by training the model on precisely hand labeled private datasets.
\nIf that happens, I think this competition would be divorced from its original purpose and would be close to meaningless.
", "votes": null, "replies": [ { "id": 1106679, "author_name": "isakev", "author_url": "", "post_date": "12/09/2020 03:15:17", "content": "Thanks for sharing. Do you think the images were augmented ?
\nTraining on hand-labeled test set, I guess, is not allowed and the winning teams will be asked to reproduce the results.
\nBut, you are right - those who are with doctors by their side having labeled the test set will get an unfair (or fair) advantage.
\n\nDo you think the images were augmented ?
\n
I don't know, but it looks to me like the above Kaggle dataset has been given some kind of distortion. Even with private data, Kaggle may dare to do some data augmentation. But this is my guess.
\n\n\nTraining on hand-labeled test set, I guess, is not allowed and the winning teams will be asked to reproduce the results.
\n
\n But, you are right - those who are with doctors by their side having labeled the test set will get an unfair (or fair) advantage.
Some people said that hand labeling of external data is possible, so in this case, as long as the private data is external data, it is possible to hand label it.
\nAs you said, I hope that the host will make sure the rules to make the competition fair and meaningful.
Supplement
\nYou can easily find the training images and public data images used in this competition by checking the checkboxes below at the site below.
Hubmap uses cytokit (https://github.com/hammerlab/cytokit) to preprocess the data.
\nIt is possible that some training data are unprocessed images or partially processed ones. The pipeline to create the processed images that can be downloaded on the hubmap website can also be found on https://github.com/hubmapconsortium/codex-pipeline/blob/master/bin/create_cytokit_config.py
", "votes": null, "replies": [] }, { "id": 1131558, "author_name": "bessenyeiszilrd", "author_url": "", "post_date": "12/29/2020 20:04:43", "content": "This is a comparing table with webpage content
\n
@bessenyeiszilrd what does it mean by <<< tricky?
It means that it has some unusual content such as VAN0003-LK-32-21-PAS_registered has unusual white square in the middle:
\n
thanks for the explanation
", "votes": null, "replies": [] } ] }, { "id": 1107044, "author_name": "hengck23", "author_url": "", "post_date": "12/09/2020 10:42:05", "content": "hmm … this looks familiar
\njson annotations must follow QuPath's json export format:
\nhttps://qupath.github.io/
https://github.com/pjl54/WSI_handling
\n[\n {\n \"type\": \"Feature\",\n \"id\": \"PathAnnotationObject\",\n \"geometry\": {\n \"type\": \"Polygon\",\n \"coordinates\": [\n [\n [76793.51, 4613.02],\n [76651.56, 4684],\n [76580.59, 4684],\n [76580.59, 4754.97] \n ]\n ]\n },\n \"properties\": {\n \"classification\": {\n \"name\": \"Tumor\",\n \"colorRGB\": -3670016\n },\n \"isLocked\": true,\n \"measurements\": []\n }\n]\nhello, frog brother, can we use the external data for pseudo label training?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1101322": "we can get kidney slide images from HuBMAP website as shown below.\nI think current deep learning is matured enough for self / semi-supervised learning.\ni.e. i want to design a model that can learn labels automatically.\n\n1. Are we allowed to use such data as external data for making the model in normal Kaggle prize?\n\n2. If we are not allowed to use the data for training, how about using this data only for analysis of our model for Judge Prize. i.e. further visualization, evaluation, etc ...\n\n", "1101458": "I think those images are in the private test set\nYou can probe the private lb with this code:\n\n- Download an image (other than the train, public test) form HuBMAP\n- Cut a small piece from the middle (I used [20000:20050, 20000:20001].flatten())\n- Repeat for rotated and flipped image as well (I am not sure this is necessary)\n- Put the results into the PROBE array\n- Commit, submit\n\nIf the score is 0.0, then the image is **in** the private test set.\nIf the result is an error, then the image is not found.\n\n```\nDIR_INPUT = '/kaggle/input/hubmap-kidney-segmentation'\nPROBE = np.array([\n [184, 177, ... , 113, 90],\n [rotated 90],\n [...]\n])\n\nsub_df = pd.read_csv(f'{DIR_INPUT}/sample_submission.csv')\n\nif sub_df.shape[0] > 5:\n \n found = False\n \n for i, row in sub_df.iterrows():\n image = skimage.io.imread(f\"{DIR_INPUT}/test/{row['id']}.tiff\")\n\n if len(image.shape) == 5:\n image = image.squeeze()\n\n # I cut from this position\n for probe in PROBE:\n if np.array_equal(image[20000:20050, 20000:20001, 0].flatten(), probe):\n found = True\n break\n \n # No luck, it seems there is no leak.\n if not found:\n # Force exit\n sys.exit()\n else:\n # If we found the sample in one of the prvate test image\n # Then our score should be 0 on the public LB\n sub_df.to_csv('submission.csv', index=False)\n \nelse:\n # This is for committing only.\n sub_df.to_csv('submission.csv', index=False)\n```", "1101569": "So much for the hidden test set I guess, that's unfortunate. Thanks for pointing this out though", "1101833": "This is a huge deal, because as there are 7 images on the private test set ( 7 / (5 + 7) = 0.58 which is the private proportion) , the full private can easily be recovered.\n\nThen inference can be made 100% offline.", "1101836": "Poke @philculliton @leahscherschel @inversion @addisonhoward", "1101871": "I agree, it is a huge deal. I think I've found 9 additional images, but I only tested one. I am not sure about the others. I sent a message to kaggle a weak ago, but they did not respond. Probably because of the thanksgiving (I sent it last Thursday)\n\n", "1102020": "from https://www.kaggle.com/c/hubmap-kidney-segmentation/overview/judges-prize, it is mentioned, \n\n\"Did the team provide insights that would be useful for generating reference glomeruli for inclusion into a Human Reference Atlas?\"\n\nthat is why I am thinking for point (2) to use HuBMAP images to show more analysis results for the judges-prize. HuBMAP images may not (or may) to be used for training at all.", "1102192": "Hi all,\n\nThanks for raising this. Please know we're investigating further.", "1102816": "It's time kaggle team to completely disable using private data during competition. And use only 2 stage to predict on private data. An addition reduce inference time to 1h or maybe 30m", "1103012": "Yes, as noted, this site does include the private test images. We are assessing impact for the competition and will let the community know how we choose to proceed with the test data.", "1106467": "One oddity is the `afa5e8098.tiff` file in the public dataset.\n\nI found the original file on the HuBMAP portal site, but it was reasonably clear, without the black shadows that @hengck23 had pointed out in [this thread](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200955#1104643) (And also think a little bit differently).\nhttps://portal.hubmapconsortium.org/browse/dataset/65dffa77af3430412ceb873e27660d40\n\n**original file**\n\n\n\n**kaggle public dataset**\n\n\n\n\nBTW, if we know which private datasets are and are allowed to do hand labeling of the external data (above HuBMAP datasets), we could easily get a higher score by training the model on precisely hand labeled private datasets.\n\nIf that happens, I think this competition would be divorced from its original purpose and would be close to meaningless.", "1106679": "Thanks for sharing. Do you think the images were augmented ?\nTraining on hand-labeled test set, I guess, is not allowed and the winning teams will be asked to reproduce the results.\nBut, you are right - those who are with doctors by their side having labeled the test set will get an unfair (or fair) advantage.", "1106699": "isakev \n> Do you think the images were augmented ?\n\nI don't know, but it looks to me like the above Kaggle dataset has been given some kind of distortion. Even with private data, Kaggle may dare to do some data augmentation. But this is my guess.\n\n> Training on hand-labeled test set, I guess, is not allowed and the winning teams will be asked to reproduce the results.\nBut, you are right - those who are with doctors by their side having labeled the test set will get an unfair (or fair) advantage.\n\nSome people said that hand labeling of external data is possible, so in this case, as long as the private data is external data, it is possible to hand label it. \nAs you said, I hope that the host will make sure the rules to make the competition fair and meaningful.\n\n\n**Supplement**\nYou can easily find the training images and public data images used in this competition by checking the checkboxes below at the site below.\n\nhttps://portal.hubmapconsortium.org/search?mapped_data_types[0]=PAS%20Stained%20Microscopy%20%5BImage%20Pyramid%5D&origin_sample.mapped_organ[0]=Kidney%20%28Left%29&origin_sample.mapped_organ[1]=Kidney%20%28Right%29&entity_type[0]=Dataset\n\n", "1107044": "hmm ... this looks familiar\njson annotations must follow QuPath's json export format:\nhttps://qupath.github.io/\n\nhttps://github.com/pjl54/WSI_handling\n```\n[\n {\n \"type\": \"Feature\",\n \"id\": \"PathAnnotationObject\",\n \"geometry\": {\n \"type\": \"Polygon\",\n \"coordinates\": [\n [\n [76793.51, 4613.02],\n [76651.56, 4684],\n [76580.59, 4684],\n [76580.59, 4754.97] \n ]\n ]\n },\n \"properties\": {\n \"classification\": {\n \"name\": \"Tumor\",\n \"colorRGB\": -3670016\n },\n \"isLocked\": true,\n \"measurements\": []\n }\n]\n\n```", "1111777": "hello, frog brother, can we use the external data for pseudo label training?", "1122314": "Hubmap uses cytokit (https://github.com/hammerlab/cytokit) to preprocess the data. \n\nIt is possible that some training data are unprocessed images or partially processed ones. The pipeline to create the processed images that can be downloaded on the hubmap website can also be found on https://github.com/hubmapconsortium/codex-pipeline/blob/master/bin/create_cytokit_config.py", "1130369": "Any updates on this and the annotation issues?", "1131558": "This is a comparing table with [webpage](https://portal.hubmapconsortium.org/search?mapped_data_types[0]=PAS%20Stained%20Microscopy%20%5BImage%20Pyramid%5D&origin_sample.mapped_organ[0]=Kidney%20%28Left%29&origin_sample.mapped_organ[1]=Kidney%20%28Right%29&entity_type[0]=Dataset) content\n\n", "1131588": "bessenyeiszilrd what does it mean by `<<< tricky`?", "1131598": "It means that it has some unusual content such as `VAN0003-LK-32-21-PAS_registered` has unusual white square in the middle:\n", "1131600": "thanks for the explanation" }, "source": "meta" }