Hi, I made a dataset with 1024x1024 png files: https://www.kaggle.com/baesiann/glomeruli-hubmap-external-1024x1024
\nMore information about the data provided in the dataset description.
\nStarter notebook for checking some images/masks also provided https://www.kaggle.com/baesiann/starter-glomeruli-hubmap-external
\nI read the point about external data (cited below) and think, that its acceptable to take publicly available data and label it, correct if I'm wrong
\n\"C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline.\"
", "messages": [ { "id": 1140065, "postDate": "2021-01-05T19:34:34.707Z", "content": "Hi, I made a dataset with 1024x1024 png files: https://www.kaggle.com/baesiann/glomeruli-hubmap-external-1024x1024
\nMore information about the data provided in the dataset description.
\nStarter notebook for checking some images/masks also provided https://www.kaggle.com/baesiann/starter-glomeruli-hubmap-external
\nI read the point about external data (cited below) and think, that its acceptable to take publicly available data and label it, correct if I'm wrong
\n\"C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline.\"
", "rawMarkdown": "Hi, I made a dataset with 1024x1024 png files: https://www.kaggle.com/baesiann/glomeruli-hubmap-external-1024x1024\n\nMore information about the data provided in the dataset description.\n\nStarter notebook for checking some images/masks also provided https://www.kaggle.com/baesiann/starter-glomeruli-hubmap-external\n\nI read the point about external data (cited below) and think, that its acceptable to take publicly available data and label it, correct if I'm wrong\n\n\"C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline.\"", "votes": 30 }, { "id": 1274844, "postDate": "2021-04-15T16:56:40.430Z", "content": "@baesiann Hi there thanx for the dataset , i have a question > does your dataset contains both scelrotic and non - scelrotic glomeruli masks ?
", "rawMarkdown": "@baesiann Hi there thanx for the dataset , i have a question > does your dataset contains both scelrotic and non - scelrotic glomeruli masks ? " }, { "id": 1264913, "postDate": "2021-04-06T13:54:12.117Z", "content": "this is great, thank you!!!
", "rawMarkdown": "this is great, thank you!!! ", "replies": [ { "id": 1264929, "postDate": "2021-04-06T14:01:47.930Z", "content": "Hello. I indicated the origin in the dataset description. The data is taken from here https://data.mendeley.com/datasets/k7nvtgn2x6/3.
\nThey were originally used in the article “Glomerulosclerosis Identification in Whole Slide Images using Semantic Segmentation”, published in Computer Methods and Programs in Biomedicine Journal.
@baesiann Interesting! How did you build your grid to generate the 1024x1024 tiles? Did you use any overlap?
\nAlso, on which criteria did you drop some tiles? (full gray?)
I was able to rebuild the full mask images from your dataset, thanks for such annotations! But after comparing to the second dataset B (the detected Glomeruli - 227x227 as PNG format) I noticed many differencies. https://data.mendeley.com/datasets/k7nvtgn2x6/3
\nDepending on images, some annotations look very good, and some are missing.
Hi, I dropped full gray tiles based on saturation check. Tiles were generated without overlap.
\nIndeed, I did not process all the raw images from dataset A. There are two reasons for this, firstly, unfortunately, I have little time and a lot of other work and secondly, because there are many sclerotic glomeruli. I did not consider in detail all the glomeruli from the competition dataset, but those that I saw are mostly normal. It seemed to me that the appearance of many sclerotic glomeruli in the data could negatively affect predictions.
", "rawMarkdown": "Hi, I dropped full gray tiles based on saturation check. Tiles were generated without overlap. \n\nIndeed, I did not process all the raw images from dataset A. There are two reasons for this, firstly, unfortunately, I have little time and a lot of other work and secondly, because there are many sclerotic glomeruli. I did not consider in detail all the glomeruli from the competition dataset, but those that I saw are mostly normal. It seemed to me that the appearance of many sclerotic glomeruli in the data could negatively affect predictions.", "votes": 1 }, { "id": 1198758, "postDate": "2021-02-13T09:21:42.503Z", "content": "Thanks for the details, it's clear to me now.
", "rawMarkdown": "Thanks for the details, it's clear to me now." }, { "id": 1258265, "postDate": "2021-03-31T13:47:49.117Z", "content": "@mpware hi, i have one question about the preprocess of this external dataset, i want to know if it was scaled and then crop without overlap? what's the scale params if it was scaled
", "rawMarkdown": "@mpware hi, i have one question about the preprocess of this external dataset, i want to know if it was scaled and then crop without overlap? what's the scale params if it was scaled", "votes": 1 }, { "id": 1258297, "postDate": "2021-03-31T14:13:47.497Z", "content": "Hi, i didn't scale the picture. The huge pictures contained several slices of tissue, I cut out individual samples from them, and then just cropped them to tiles without overlap. So I didn't change the resolution.
", "rawMarkdown": "Hi, i didn't scale the picture. The huge pictures contained several slices of tissue, I cut out individual samples from them, and then just cropped them to tiles without overlap. So I didn't change the resolution.", "votes": 1 }, { "id": 1258305, "postDate": "2021-03-31T14:17:10.430Z", "content": "@baesiann thanks for reply and sharing.
", "rawMarkdown": "@baesiann thanks for reply and sharing." }, { "id": 1258358, "postDate": "2021-03-31T14:50:44.067Z", "content": "@cswwp347724 I've almost extracted the masks (from the tiles) and there is no overlap and no scale. The x,y offset can be found for extract match for around 70% of images. For some reason I did not find the missing 30%.
", "rawMarkdown": "@cswwp347724 I've almost extracted the masks (from the tiles) and there is no overlap and no scale. The x,y offset can be found for extract match for around 70% of images. For some reason I did not find the missing 30%." }, { "id": 1259127, "postDate": "2021-04-01T06:58:28.430Z", "content": "thank you, maybe the 30% drooped because of saturation is too low
", "rawMarkdown": "thank you, maybe the 30% drooped because of saturation is too low" }, { "id": 1268466, "postDate": "2021-04-09T12:27:17.037Z", "content": "@mpware does this external dataset help?
", "rawMarkdown": "@mpware does this external dataset help?", "votes": 1 }, { "id": 1268713, "postDate": "2021-04-09T17:10:54.970Z", "content": "It helped for initial data (before the updated data) and only partially (not all images) as some masks are missing. I did not try them yet with new data. I will try soon.
", "rawMarkdown": "It helped for initial data (before the updated data) and only partially (not all images) as some masks are missing. I did not try them yet with new data. I will try soon.", "votes": 1 }, { "id": 1273497, "postDate": "2021-04-14T12:08:54.183Z", "content": "@baesiann could u help understand the overlap here . please.
", "rawMarkdown": "@baesiann could u help understand the overlap here . please." }, { "id": 1281770, "postDate": "2021-04-23T09:35:25.977Z", "content": "@mpware what does terminology overlap means .. i see often come across it in discussions :)
", "rawMarkdown": "@mpware what does terminology overlap means .. i see often come across it in discussions :)" }, { "id": 1281843, "postDate": "2021-04-23T11:07:12.783Z", "content": "Overlap between tiles. Partial common part from 2 consecutive tiles.
", "rawMarkdown": "Overlap between tiles. Partial common part from 2 consecutive tiles." }, { "id": 1281969, "postDate": "2021-04-23T13:29:34.213Z", "content": "@mpware does this external dataset help ? i mean it contains both scelrotic and non sclerotic if i am not wrong
", "rawMarkdown": "@mpware does this external dataset help ? i mean it contains both scelrotic and non sclerotic if i am not wrong " } ] }, { "id": 1144104, "postDate": "2021-01-08T08:30:27.620Z", "content": "@baesiann Yes, I believe you are correct. BTW nice work there!
", "rawMarkdown": "@baesiann Yes, I believe you are correct. BTW nice work there!" }, { "id": 1272400, "postDate": "2021-04-13T12:54:13.293Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1249168, "postDate": "2021-03-23T06:39:16.717Z", "content": "Really good works. Thanks.
", "rawMarkdown": "Really good works. Thanks. " } ], "comments": [ { "id": 1274844, "author_name": "Shubham Thapa", "author_url": "", "post_date": "2021-04-15T16:56:40.430000", "content": "@baesiann Hi there thanx for the dataset , i have a question > does your dataset contains both scelrotic and non - scelrotic glomeruli masks ?
", "votes": 0, "replies": [] }, { "id": 1264913, "author_name": "andras", "author_url": "", "post_date": "2021-04-06T13:54:12.117000", "content": "this is great, thank you!!!
", "votes": 0, "replies": [ { "id": 1264929, "author_name": "Alexey Gavrikov", "author_url": "", "post_date": "2021-04-06T14:01:47.930000", "content": "Hello. I indicated the origin in the dataset description. The data is taken from here https://data.mendeley.com/datasets/k7nvtgn2x6/3.
\nThey were originally used in the article “Glomerulosclerosis Identification in Whole Slide Images using Semantic Segmentation”, published in Computer Methods and Programs in Biomedicine Journal.
@baesiann Interesting! How did you build your grid to generate the 1024x1024 tiles? Did you use any overlap?
\nAlso, on which criteria did you drop some tiles? (full gray?)
I was able to rebuild the full mask images from your dataset, thanks for such annotations! But after comparing to the second dataset B (the detected Glomeruli - 227x227 as PNG format) I noticed many differencies. https://data.mendeley.com/datasets/k7nvtgn2x6/3
\nDepending on images, some annotations look very good, and some are missing.
Hi, I dropped full gray tiles based on saturation check. Tiles were generated without overlap.
\nIndeed, I did not process all the raw images from dataset A. There are two reasons for this, firstly, unfortunately, I have little time and a lot of other work and secondly, because there are many sclerotic glomeruli. I did not consider in detail all the glomeruli from the competition dataset, but those that I saw are mostly normal. It seemed to me that the appearance of many sclerotic glomeruli in the data could negatively affect predictions.
", "votes": 1, "replies": [] }, { "id": 1198758, "author_name": "MPWARE", "author_url": "", "post_date": "2021-02-13T09:21:42.503000", "content": "Thanks for the details, it's clear to me now.
", "votes": 0, "replies": [] }, { "id": 1258265, "author_name": "cswwp", "author_url": "", "post_date": "2021-03-31T13:47:49.117000", "content": "@mpware hi, i have one question about the preprocess of this external dataset, i want to know if it was scaled and then crop without overlap? what's the scale params if it was scaled
", "votes": 1, "replies": [] }, { "id": 1258297, "author_name": "Alexey Gavrikov", "author_url": "", "post_date": "2021-03-31T14:13:47.497000", "content": "Hi, i didn't scale the picture. The huge pictures contained several slices of tissue, I cut out individual samples from them, and then just cropped them to tiles without overlap. So I didn't change the resolution.
", "votes": 1, "replies": [] }, { "id": 1258305, "author_name": "cswwp", "author_url": "", "post_date": "2021-03-31T14:17:10.430000", "content": "@baesiann thanks for reply and sharing.
", "votes": 0, "replies": [] }, { "id": 1258358, "author_name": "MPWARE", "author_url": "", "post_date": "2021-03-31T14:50:44.067000", "content": "@cswwp347724 I've almost extracted the masks (from the tiles) and there is no overlap and no scale. The x,y offset can be found for extract match for around 70% of images. For some reason I did not find the missing 30%.
", "votes": 0, "replies": [] }, { "id": 1259127, "author_name": "cswwp", "author_url": "", "post_date": "2021-04-01T06:58:28.430000", "content": "thank you, maybe the 30% drooped because of saturation is too low
", "votes": 0, "replies": [] }, { "id": 1268466, "author_name": "tugstugi", "author_url": "", "post_date": "2021-04-09T12:27:17.037000", "content": "@mpware does this external dataset help?
", "votes": 1, "replies": [] }, { "id": 1268713, "author_name": "MPWARE", "author_url": "", "post_date": "2021-04-09T17:10:54.970000", "content": "It helped for initial data (before the updated data) and only partially (not all images) as some masks are missing. I did not try them yet with new data. I will try soon.
", "votes": 1, "replies": [] }, { "id": 1273497, "author_name": "Jaideep", "author_url": "", "post_date": "2021-04-14T12:08:54.183000", "content": "@baesiann could u help understand the overlap here . please.
", "votes": 0, "replies": [] }, { "id": 1281770, "author_name": "Jaideep", "author_url": "", "post_date": "2021-04-23T09:35:25.977000", "content": "@mpware what does terminology overlap means .. i see often come across it in discussions :)
", "votes": 0, "replies": [] }, { "id": 1281843, "author_name": "MPWARE", "author_url": "", "post_date": "2021-04-23T11:07:12.783000", "content": "Overlap between tiles. Partial common part from 2 consecutive tiles.
", "votes": 0, "replies": [] }, { "id": 1281969, "author_name": "Shubham Thapa", "author_url": "", "post_date": "2021-04-23T13:29:34.213000", "content": "@mpware does this external dataset help ? i mean it contains both scelrotic and non sclerotic if i am not wrong
", "votes": 0, "replies": [] } ] }, { "id": 1144104, "author_name": "Izzy Adesanya", "author_url": "", "post_date": "2021-01-08T08:30:27.620000", "content": "@baesiann Yes, I believe you are correct. BTW nice work there!
", "votes": 0, "replies": [] }, { "id": 1272400, "author_name": "", "author_url": "", "post_date": "2021-04-13T12:54:13.293000", "content": "", "votes": 0, "replies": [] }, { "id": 1249168, "author_name": "Md. Masud Rana", "author_url": "", "post_date": "2021-03-23T06:39:16.717000", "content": "Really good works. Thanks.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1140065": "Hi, I made a dataset with 1024x1024 png files: https://www.kaggle.com/baesiann/glomeruli-hubmap-external-1024x1024\n\nMore information about the data provided in the dataset description.\n\nStarter notebook for checking some images/masks also provided https://www.kaggle.com/baesiann/starter-glomeruli-hubmap-external\n\nI read the point about external data (cited below) and think, that its acceptable to take publicly available data and label it, correct if I'm wrong\n\n\"C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline.\"", "1274844": "@baesiann Hi there thanx for the dataset , i have a question > does your dataset contains both scelrotic and non - scelrotic glomeruli masks ? ", "1264913": "this is great, thank you!!! ", "1195560": "@baesiann Interesting! How did you build your grid to generate the 1024x1024 tiles? Did you use any overlap?\nAlso, on which criteria did you drop some tiles? (full gray?)", "1144104": "@baesiann Yes, I believe you are correct. BTW nice work there!", "1272400": "", "1249168": "Really good works. Thanks. " } }