{
  "id": 224826,
  "title": "Data updated, timeline extended",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/224826",
  "author_name": "",
  "post_date": "2021-03-09T22:37:53.241645900Z",
  "votes": 42,
  "comment_count": 23,
  "views": 0,
  "content": "<p>Hi all,</p>\n<p>Earlier in the competition, <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/207884\" target=\"_blank\">it was noted</a> that the dataset had a number of issues and needed a refresh. </p>\n<p>We appreciate your patience, and we have finally completed the refresh. We've extended the competition deadline by two months (to end on May 10, 11:59pm UTC), and adjusted other timelines accordingly. The judges review timeline may still be subject to change given the new schedule.</p>\n<p>The data update is complete and submissions have been re-enabled, but it may take some time for us to finish re-running all existing submissions. Any new submissions will be scored as they're received. </p>\n<p>Thanks again for your patience and best of luck modeling!</p>\n<p>Kaggle and HuBMAP Team</p>",
  "messages": [
    {
      "id": "1232650",
      "postDate": "03/09/2021 22:37:53",
      "content": "<p>Hi all,</p>\n<p>Earlier in the competition, <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/207884\" target=\"_blank\">it was noted</a> that the dataset had a number of issues and needed a refresh. </p>\n<p>We appreciate your patience, and we have finally completed the refresh. We've extended the competition deadline by two months (to end on May 10, 11:59pm UTC), and adjusted other timelines accordingly. The judges review timeline may still be subject to change given the new schedule.</p>\n<p>The data update is complete and submissions have been re-enabled, but it may take some time for us to finish re-running all existing submissions. Any new submissions will be scored as they're received. </p>\n<p>Thanks again for your patience and best of luck modeling!</p>\n<p>Kaggle and HuBMAP Team</p>",
      "rawMarkdown": "Hi all,\n\nEarlier in the competition, [it was noted](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/207884) that the dataset had a number of issues and needed a refresh. \n\nWe appreciate your patience, and we have finally completed the refresh. We've extended the competition deadline by two months (to end on May 10, 11:59pm UTC), and adjusted other timelines accordingly. The judges review timeline may still be subject to change given the new schedule.\n\nThe data update is complete and submissions have been re-enabled, but it may take some time for us to finish re-running all existing submissions. Any new submissions will be scored as they're received. \n\nThanks again for your patience and best of luck modeling!\n\nKaggle and HuBMAP Team",
      "votes": null
    },
    {
      "id": "1232691",
      "postDate": "03/09/2021 23:53:39",
      "content": "<p>Thanks for the updating. I had given up, but now I can try this endless competition again. :)</p>",
      "rawMarkdown": "Thanks for the updating. I had given up, but now I can try this endless competition again. :)",
      "votes": null
    },
    {
      "id": "1232897",
      "postDate": "03/10/2021 03:16:44",
      "content": "<p>Do we need to download training data again?</p>",
      "rawMarkdown": "Do we need to download training data again?",
      "votes": null
    },
    {
      "id": "1232957",
      "postDate": "03/10/2021 04:05:28",
      "content": "<p>Yes, they modified the files in the training and testing datasets.</p>",
      "rawMarkdown": "Yes, they modified the files in the training and testing datasets.",
      "votes": null
    },
    {
      "id": "1233074",
      "postDate": "03/10/2021 05:44:52",
      "content": "<p>So cool! We will try a larger model on a larger dataset.</p>",
      "rawMarkdown": "So cool! We will try a larger model on a larger dataset.",
      "votes": null
    },
    {
      "id": "1233159",
      "postDate": "03/10/2021 07:42:53",
      "content": "<p>The private set will have .tiff files larger than 4GB?  or 5GB? Thankyou</p>",
      "rawMarkdown": "The private set will have .tiff files larger than 4GB?  or 5GB? Thankyou",
      "votes": null
    },
    {
      "id": "1233705",
      "postDate": "03/10/2021 16:22:30",
      "content": "<p>I'm having a problem trying to run my tiling program on the new data:</p>\n<p>102.0s</p>\n<h2>157</h2>\n<p>102.0s<br>\n158<br>\nException encountered at \"In [40]\":<br>\n102.0s</p>\n<h2>159</h2>\n<p>102.0s<br>\n160<br>\nerror                                     Traceback (most recent call last)<br>\n102.0s<br>\n161<br>\n in <br>\n102.0s<br>\n162<br>\n      2 input_dir = \"/kaggle/input/hubmap-kidney-segmentation/train/\"<br>\n102.0s<br>\n163<br>\n      3 output_dir = '/kaggle/working/train/'<br>\n102.0s<br>\n164<br>\n----&gt; 4 tile_and_build_tfr( input_dir, output_dir, do_mask = True, do_normalize = P['DO_NORMALIZE'] )<br>\n102.0s<br>\n165<br>\n      5 get_ipython().system('ls -l /kaggle/working/train')<br>\n102.0s<br>\n166</p>\n<p>102.0s<br>\n167<br>\n in tile_and_build_tfr(input_dir, output_dir, do_mask, do_normalize, tile_size, tile_overlap)<br>\n102.0s<br>\n168<br>\n     45         if do_mask:<br>\n102.0s<br>\n169<br>\n     46             mask_shape = (baseimage.shape[0], baseimage.shape[1])<br>\n102.0s<br>\n170<br>\n---&gt; 47             mask = read_mask( mask_file_name, mask_shape)<br>\n102.0s<br>\n171<br>\n     48         else:<br>\n102.0s<br>\n172<br>\n     49             mask = None<br>\n102.0s<br>\n173</p>\n<p>102.0s<br>\n174<br>\n in read_mask(mask_file, mask_shape)<br>\n102.0s<br>\n175<br>\n      8 <br>\n102.0s<br>\n176<br>\n      9     mask = np.zeros(mask_shape, dtype = np.uint8 )<br>\n102.0s<br>\n177<br>\n---&gt; 10     cv2.fillPoly(mask, polys, 1)<br>\n102.0s<br>\n178<br>\n     11     mask = mask.astype(bool)<br>\n102.0s<br>\n179<br>\n     12     return mask<br>\n102.0s<br>\n180</p>\n<p>102.0s<br>\n181<br>\nerror: OpenCV(4.4.0) /tmp/pip-req-build-2mquh264/opencv/modules/imgproc/src/drawing.cpp:2395: error: (-215:Assertion failed) p.checkVector(2, CV_32S) &gt;= 0 in function 'fillPoly'</p>",
      "rawMarkdown": "I'm having a problem trying to run my tiling program on the new data:\n\n102.0s\n157\n---------------------------------------------------------------------------\n102.0s\n158\nException encountered at \"In [40]\":\n102.0s\n159\n---------------------------------------------------------------------------\n102.0s\n160\nerror                                     Traceback (most recent call last)\n102.0s\n161\n<ipython-input-40-e06cdd71d685> in <module>\n102.0s\n162\n      2 input_dir = \"/kaggle/input/hubmap-kidney-segmentation/train/\"\n102.0s\n163\n      3 output_dir = '/kaggle/working/train/'\n102.0s\n164\n----> 4 tile_and_build_tfr( input_dir, output_dir, do_mask = True, do_normalize = P['DO_NORMALIZE'] )\n102.0s\n165\n      5 get_ipython().system('ls -l /kaggle/working/train')\n102.0s\n166\n\n102.0s\n167\n<ipython-input-39-9005e41d64ae> in tile_and_build_tfr(input_dir, output_dir, do_mask, do_normalize, tile_size, tile_overlap)\n102.0s\n168\n     45         if do_mask:\n102.0s\n169\n     46             mask_shape = (baseimage.shape[0], baseimage.shape[1])\n102.0s\n170\n---> 47             mask = read_mask( mask_file_name, mask_shape)\n102.0s\n171\n     48         else:\n102.0s\n172\n     49             mask = None\n102.0s\n173\n\n102.0s\n174\n<ipython-input-15-5a248ee894a5> in read_mask(mask_file, mask_shape)\n102.0s\n175\n      8 \n102.0s\n176\n      9     mask = np.zeros(mask_shape, dtype = np.uint8 )\n102.0s\n177\n---> 10     cv2.fillPoly(mask, polys, 1)\n102.0s\n178\n     11     mask = mask.astype(bool)\n102.0s\n179\n     12     return mask\n102.0s\n180\n\n102.0s\n181\nerror: OpenCV(4.4.0) /tmp/pip-req-build-2mquh264/opencv/modules/imgproc/src/drawing.cpp:2395: error: (-215:Assertion failed) p.checkVector(2, CV_32S) >= 0 in function 'fillPoly'",
      "votes": null
    },
    {
      "id": "1233871",
      "postDate": "03/10/2021 18:38:34",
      "content": "<p>Have u noticed that the new data has some \"new features\"?  The format/encoding is  bit different from the older ones. </p>",
      "rawMarkdown": "Have u noticed that the new data has some \"new features\"?  The format/encoding is  bit different from the older ones.",
      "votes": null
    },
    {
      "id": "1233889",
      "postDate": "03/10/2021 18:52:18",
      "content": "<p>Oh, drat!   Is this documented anywhere?    In any case, I have solved my problem (bad .json format?) by switching to use the RLE representation of the glom masks in <code>train.csv</code>.</p>",
      "rawMarkdown": "Oh, drat!   Is this documented anywhere?    In any case, I have solved my problem (bad .json format?) by switching to use the RLE representation of the glom masks in ```train.csv```.",
      "votes": null
    },
    {
      "id": "1235557",
      "postDate": "03/12/2021 08:41:07",
      "content": "<p>I'm a bit confused about data availability:<br>\nin \"Data\" tab you can find \"Glomeruli FTU annotations exist for all 20 tissue samples\"; files \"2ec3f1bb9.json\", \"3589adb90.json\", \"57512b7f1.json\", \"aa05346ff.json\", \"d488c759a.json\", that correspond to 5 tiff files in test folder are mentioned in \"HuBMAP-20-dataset_information.csv\", but there's only 15 lines in \"train.csv\" and only anatomical jsons in test foilder. Do we have 15 files with annotation or 20?</p>",
      "rawMarkdown": "I'm a bit confused about data availability:\nin \"Data\" tab you can find \"Glomeruli FTU annotations exist for all 20 tissue samples\"; files \"2ec3f1bb9.json\", \"3589adb90.json\", \"57512b7f1.json\", \"aa05346ff.json\", \"d488c759a.json\", that correspond to 5 tiff files in test folder are mentioned in \"HuBMAP-20-dataset_information.csv\", but there's only 15 lines in \"train.csv\" and only anatomical jsons in test foilder. Do we have 15 files with annotation or 20?",
      "votes": null
    },
    {
      "id": "1235673",
      "postDate": "03/12/2021 11:23:49",
      "content": "<p>Seems like a couple of the tiff files might have a different compression method or color space. I'm not sure as the cloud environment I get assigned changes.</p>\n<p>Here's how I'm loading the files:</p>\n<pre><code>from skimage import transform, io\nimport PIL.Image\nPIL.Image.MAX_IMAGE_PIXELS = 100000000000\n</code></pre>\n<pre><code>try:\n  img = io.imread(img_path).squeeze()\nexcept:\n  img = io.imread(img_path, plugin='pil').squeeze()\n</code></pre>",
      "rawMarkdown": "Seems like a couple of the tiff files might have a different compression method or color space. I'm not sure as the cloud environment I get assigned changes.\n\nHere's how I'm loading the files:\n\n```\nfrom skimage import transform, io\nimport PIL.Image\nPIL.Image.MAX_IMAGE_PIXELS = 100000000000\n\n```\n```\ntry:\n  img = io.imread(img_path).squeeze()\nexcept:\n  img = io.imread(img_path, plugin='pil').squeeze()\n```",
      "votes": null
    },
    {
      "id": "1235772",
      "postDate": "03/12/2021 13:13:43",
      "content": "<p></p>",
      "rawMarkdown": "~~Some of the images are greyscale and only have one channel. You need to account for this when you read them~~",
      "votes": null
    },
    {
      "id": "1235869",
      "postDate": "03/12/2021 14:34:53",
      "content": "<p>Which one is BW? Are you sure?</p>",
      "rawMarkdown": "Which one is BW? Are you sure?",
      "votes": null
    },
    {
      "id": "1235924",
      "postDate": "03/12/2021 15:44:27",
      "content": "<p>Hmm, I'm using <code>rasterio</code> and am only seeing 1 channel in 095bf7a1f, 4ef6695ce, 26dc41664, c68fe75ea, 1e2425f28.</p>\n<p>I'll try opening them in GIMP and double check…</p>\n<p>Edit: 4ef6695ce has 3 layers in GIMP, not sure why <code>rasterio</code> isn't seeing them. Thanks for making me check and apologies for the confusion! :)</p>",
      "rawMarkdown": "Hmm, I'm using `rasterio` and am only seeing 1 channel in 095bf7a1f, 4ef6695ce, 26dc41664, c68fe75ea, 1e2425f28.\n\nI'll try opening them in GIMP and double check...\n\nEdit: 4ef6695ce has 3 layers in GIMP, not sure why `rasterio` isn't seeing them. Thanks for making me check and apologies for the confusion! :)",
      "votes": null
    },
    {
      "id": "1235945",
      "postDate": "03/12/2021 15:58:04",
      "content": "<p>Unless I've royally screwed up, at least 1e2425f28 isn't grayscale. Here's a tile from that file <code>1e2425f28_4096_8192_8192_12288</code>:</p>\n<p><a href=\"https://postimg.cc/vc4hDrWm\" target=\"_blank\"><img src=\"https://i.postimg.cc/vc4hDrWm/1e2425f28-4096-8192-8192-12288.jpg\" alt=\"1e2425f28-4096-8192-8192-12288\"></a></p>\n<p>I'm not using a package to rasterize.</p>",
      "rawMarkdown": "Unless I've royally screwed up, at least 1e2425f28 isn't grayscale. Here's a tile from that file `1e2425f28_4096_8192_8192_12288`:\n\n<a href='https://postimg.cc/vc4hDrWm' target='_blank'><img src='https://i.postimg.cc/vc4hDrWm/1e2425f28-4096-8192-8192-12288.jpg' border='0' alt='1e2425f28-4096-8192-8192-12288'/></a>\n\nI'm not using a package to rasterize.",
      "votes": null
    },
    {
      "id": "1235958",
      "postDate": "03/12/2021 16:12:42",
      "content": "<p>In case anyone else was facing the same issue as me, the solution is here: <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186</a></p>",
      "rawMarkdown": "In case anyone else was facing the same issue as me, the solution is here: https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186",
      "votes": null
    },
    {
      "id": "1236058",
      "postDate": "03/12/2021 17:58:43",
      "content": "<p>Im using tifffile from python on R. It handle this just fine. </p>",
      "rawMarkdown": "Im using tifffile from python on R. It handle this just fine.",
      "votes": null
    },
    {
      "id": "1237338",
      "postDate": "03/14/2021 04:12:50",
      "content": "<p>In the training set, '095bf7a1f.tiff' is a single channel image (gray scale). <br>\nNot sure if this is intentional since the whole idea of histology staining is to provide some color changes to pathological changes. Should single channel Images be ignored in this case? </p>",
      "rawMarkdown": "In the training set, '095bf7a1f.tiff' is a single channel image (gray scale). \nNot sure if this is intentional since the whole idea of histology staining is to provide some color changes to pathological changes. Should single channel Images be ignored in this case?",
      "votes": null
    },
    {
      "id": "1237357",
      "postDate": "03/14/2021 04:29:33",
      "content": "<p>They are all color images. Theres a couple of posts below about it.</p>",
      "rawMarkdown": "They are all color images. Theres a couple of posts below about it.",
      "votes": null
    },
    {
      "id": "1238279",
      "postDate": "03/14/2021 20:27:11",
      "content": "<p><a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883</a></p>",
      "rawMarkdown": "https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883",
      "votes": null
    },
    {
      "id": "1238340",
      "postDate": "03/14/2021 22:50:15",
      "content": "<p>Thank you very much for helping me out</p>",
      "rawMarkdown": "Thank you very much for helping me out",
      "votes": null
    },
    {
      "id": "1255738",
      "postDate": "03/29/2021 06:39:52",
      "content": "<p>Hi,<br>\nany update on the judges review timeline? The old dates are still online.<br>\nThanks!</p>",
      "rawMarkdown": "Hi,\nany update on the judges review timeline? The old dates are still online.\nThanks!",
      "votes": null
    },
    {
      "id": "1260585",
      "postDate": "04/02/2021 08:38:36",
      "content": "<p>thanks for updating.</p>",
      "rawMarkdown": "thanks for updating.",
      "votes": null
    },
    {
      "id": "1298883",
      "postDate": "05/09/2021 09:44:35",
      "content": "<p><a href=\"https://www.kaggle.com/addisonhoward\" target=\"_blank\">@addisonhoward</a>  does private set contains data from Different patient numbers from Train or they may contain some of those eg. Test has three patient ids matching to that of Train.</p>",
      "rawMarkdown": "addisonhoward  does private set contains data from Different patient numbers from Train or they may contain some of those eg. Test has three patient ids matching to that of Train.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1232691,
      "author_name": "wuliaokaola",
      "author_url": "",
      "post_date": "03/09/2021 23:53:39",
      "content": "<p>Thanks for the updating. I had given up, but now I can try this endless competition again. :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1232897,
      "author_name": "haqishen",
      "author_url": "",
      "post_date": "03/10/2021 03:16:44",
      "content": "<p>Do we need to download training data again?</p>",
      "votes": null,
      "replies": [
        {
          "id": 1232957,
          "author_name": "paulsuen",
          "author_url": "",
          "post_date": "03/10/2021 04:05:28",
          "content": "<p>Yes, they modified the files in the training and testing datasets.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1233074,
      "author_name": "chenyangouc",
      "author_url": "",
      "post_date": "03/10/2021 05:44:52",
      "content": "<p>So cool! We will try a larger model on a larger dataset.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1233159,
      "author_name": "rpsantosakaggle",
      "author_url": "",
      "post_date": "03/10/2021 07:42:53",
      "content": "<p>The private set will have .tiff files larger than 4GB?  or 5GB? Thankyou</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1233705,
      "author_name": "markalavin",
      "author_url": "",
      "post_date": "03/10/2021 16:22:30",
      "content": "<p>I'm having a problem trying to run my tiling program on the new data:</p>\n<p>102.0s</p>\n<h2>157</h2>\n<p>102.0s<br>\n158<br>\nException encountered at \"In [40]\":<br>\n102.0s</p>\n<h2>159</h2>\n<p>102.0s<br>\n160<br>\nerror                                     Traceback (most recent call last)<br>\n102.0s<br>\n161<br>\n in <br>\n102.0s<br>\n162<br>\n      2 input_dir = \"/kaggle/input/hubmap-kidney-segmentation/train/\"<br>\n102.0s<br>\n163<br>\n      3 output_dir = '/kaggle/working/train/'<br>\n102.0s<br>\n164<br>\n----&gt; 4 tile_and_build_tfr( input_dir, output_dir, do_mask = True, do_normalize = P['DO_NORMALIZE'] )<br>\n102.0s<br>\n165<br>\n      5 get_ipython().system('ls -l /kaggle/working/train')<br>\n102.0s<br>\n166</p>\n<p>102.0s<br>\n167<br>\n in tile_and_build_tfr(input_dir, output_dir, do_mask, do_normalize, tile_size, tile_overlap)<br>\n102.0s<br>\n168<br>\n     45         if do_mask:<br>\n102.0s<br>\n169<br>\n     46             mask_shape = (baseimage.shape[0], baseimage.shape[1])<br>\n102.0s<br>\n170<br>\n---&gt; 47             mask = read_mask( mask_file_name, mask_shape)<br>\n102.0s<br>\n171<br>\n     48         else:<br>\n102.0s<br>\n172<br>\n     49             mask = None<br>\n102.0s<br>\n173</p>\n<p>102.0s<br>\n174<br>\n in read_mask(mask_file, mask_shape)<br>\n102.0s<br>\n175<br>\n      8 <br>\n102.0s<br>\n176<br>\n      9     mask = np.zeros(mask_shape, dtype = np.uint8 )<br>\n102.0s<br>\n177<br>\n---&gt; 10     cv2.fillPoly(mask, polys, 1)<br>\n102.0s<br>\n178<br>\n     11     mask = mask.astype(bool)<br>\n102.0s<br>\n179<br>\n     12     return mask<br>\n102.0s<br>\n180</p>\n<p>102.0s<br>\n181<br>\nerror: OpenCV(4.4.0) /tmp/pip-req-build-2mquh264/opencv/modules/imgproc/src/drawing.cpp:2395: error: (-215:Assertion failed) p.checkVector(2, CV_32S) &gt;= 0 in function 'fillPoly'</p>",
      "votes": null,
      "replies": [
        {
          "id": 1233871,
          "author_name": "rpsantosakaggle",
          "author_url": "",
          "post_date": "03/10/2021 18:38:34",
          "content": "<p>Have u noticed that the new data has some \"new features\"?  The format/encoding is  bit different from the older ones. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1233889,
          "author_name": "markalavin",
          "author_url": "",
          "post_date": "03/10/2021 18:52:18",
          "content": "<p>Oh, drat!   Is this documented anywhere?    In any case, I have solved my problem (bad .json format?) by switching to use the RLE representation of the glom masks in <code>train.csv</code>.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235673,
          "author_name": "erikdali",
          "author_url": "",
          "post_date": "03/12/2021 11:23:49",
          "content": "<p>Seems like a couple of the tiff files might have a different compression method or color space. I'm not sure as the cloud environment I get assigned changes.</p>\n<p>Here's how I'm loading the files:</p>\n<pre><code>from skimage import transform, io\nimport PIL.Image\nPIL.Image.MAX_IMAGE_PIXELS = 100000000000\n</code></pre>\n<pre><code>try:\n  img = io.imread(img_path).squeeze()\nexcept:\n  img = io.imread(img_path, plugin='pil').squeeze()\n</code></pre>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235772,
          "author_name": "anjum48",
          "author_url": "",
          "post_date": "03/12/2021 13:13:43",
          "content": "<p></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235869,
          "author_name": "erikdali",
          "author_url": "",
          "post_date": "03/12/2021 14:34:53",
          "content": "<p>Which one is BW? Are you sure?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235924,
          "author_name": "anjum48",
          "author_url": "",
          "post_date": "03/12/2021 15:44:27",
          "content": "<p>Hmm, I'm using <code>rasterio</code> and am only seeing 1 channel in 095bf7a1f, 4ef6695ce, 26dc41664, c68fe75ea, 1e2425f28.</p>\n<p>I'll try opening them in GIMP and double check…</p>\n<p>Edit: 4ef6695ce has 3 layers in GIMP, not sure why <code>rasterio</code> isn't seeing them. Thanks for making me check and apologies for the confusion! :)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235945,
          "author_name": "erikdali",
          "author_url": "",
          "post_date": "03/12/2021 15:58:04",
          "content": "<p>Unless I've royally screwed up, at least 1e2425f28 isn't grayscale. Here's a tile from that file <code>1e2425f28_4096_8192_8192_12288</code>:</p>\n<p><a href=\"https://postimg.cc/vc4hDrWm\" target=\"_blank\"><img src=\"https://i.postimg.cc/vc4hDrWm/1e2425f28-4096-8192-8192-12288.jpg\" alt=\"1e2425f28-4096-8192-8192-12288\"></a></p>\n<p>I'm not using a package to rasterize.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1235958,
          "author_name": "anjum48",
          "author_url": "",
          "post_date": "03/12/2021 16:12:42",
          "content": "<p>In case anyone else was facing the same issue as me, the solution is here: <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1236058,
          "author_name": "rpsantosakaggle",
          "author_url": "",
          "post_date": "03/12/2021 17:58:43",
          "content": "<p>Im using tifffile from python on R. It handle this just fine. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1235557,
      "author_name": "tetelias",
      "author_url": "",
      "post_date": "03/12/2021 08:41:07",
      "content": "<p>I'm a bit confused about data availability:<br>\nin \"Data\" tab you can find \"Glomeruli FTU annotations exist for all 20 tissue samples\"; files \"2ec3f1bb9.json\", \"3589adb90.json\", \"57512b7f1.json\", \"aa05346ff.json\", \"d488c759a.json\", that correspond to 5 tiff files in test folder are mentioned in \"HuBMAP-20-dataset_information.csv\", but there's only 15 lines in \"train.csv\" and only anatomical jsons in test foilder. Do we have 15 files with annotation or 20?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1237338,
      "author_name": "nehadhimiz",
      "author_url": "",
      "post_date": "03/14/2021 04:12:50",
      "content": "<p>In the training set, '095bf7a1f.tiff' is a single channel image (gray scale). <br>\nNot sure if this is intentional since the whole idea of histology staining is to provide some color changes to pathological changes. Should single channel Images be ignored in this case? </p>",
      "votes": null,
      "replies": [
        {
          "id": 1237357,
          "author_name": "erikdali",
          "author_url": "",
          "post_date": "03/14/2021 04:29:33",
          "content": "<p>They are all color images. Theres a couple of posts below about it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1238279,
          "author_name": "rpsantosakaggle",
          "author_url": "",
          "post_date": "03/14/2021 20:27:11",
          "content": "<p><a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1238340,
          "author_name": "nehadhimiz",
          "author_url": "",
          "post_date": "03/14/2021 22:50:15",
          "content": "<p>Thank you very much for helping me out</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1255738,
      "author_name": "friedchips",
      "author_url": "",
      "post_date": "03/29/2021 06:39:52",
      "content": "<p>Hi,<br>\nany update on the judges review timeline? The old dates are still online.<br>\nThanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1260585,
      "author_name": "aajay20",
      "author_url": "",
      "post_date": "04/02/2021 08:38:36",
      "content": "<p>thanks for updating.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1298883,
      "author_name": "jaideepvalani",
      "author_url": "",
      "post_date": "05/09/2021 09:44:35",
      "content": "<p><a href=\"https://www.kaggle.com/addisonhoward\" target=\"_blank\">@addisonhoward</a>  does private set contains data from Different patient numbers from Train or they may contain some of those eg. Test has three patient ids matching to that of Train.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1232650": "Hi all,\n\nEarlier in the competition, [it was noted](https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/207884) that the dataset had a number of issues and needed a refresh. \n\nWe appreciate your patience, and we have finally completed the refresh. We've extended the competition deadline by two months (to end on May 10, 11:59pm UTC), and adjusted other timelines accordingly. The judges review timeline may still be subject to change given the new schedule.\n\nThe data update is complete and submissions have been re-enabled, but it may take some time for us to finish re-running all existing submissions. Any new submissions will be scored as they're received. \n\nThanks again for your patience and best of luck modeling!\n\nKaggle and HuBMAP Team",
    "1232691": "Thanks for the updating. I had given up, but now I can try this endless competition again. :)",
    "1232897": "Do we need to download training data again?",
    "1232957": "Yes, they modified the files in the training and testing datasets.",
    "1233074": "So cool! We will try a larger model on a larger dataset.",
    "1233159": "The private set will have .tiff files larger than 4GB?  or 5GB? Thankyou",
    "1233705": "I'm having a problem trying to run my tiling program on the new data:\n\n102.0s\n157\n---------------------------------------------------------------------------\n102.0s\n158\nException encountered at \"In [40]\":\n102.0s\n159\n---------------------------------------------------------------------------\n102.0s\n160\nerror                                     Traceback (most recent call last)\n102.0s\n161\n<ipython-input-40-e06cdd71d685> in <module>\n102.0s\n162\n      2 input_dir = \"/kaggle/input/hubmap-kidney-segmentation/train/\"\n102.0s\n163\n      3 output_dir = '/kaggle/working/train/'\n102.0s\n164\n----> 4 tile_and_build_tfr( input_dir, output_dir, do_mask = True, do_normalize = P['DO_NORMALIZE'] )\n102.0s\n165\n      5 get_ipython().system('ls -l /kaggle/working/train')\n102.0s\n166\n\n102.0s\n167\n<ipython-input-39-9005e41d64ae> in tile_and_build_tfr(input_dir, output_dir, do_mask, do_normalize, tile_size, tile_overlap)\n102.0s\n168\n     45         if do_mask:\n102.0s\n169\n     46             mask_shape = (baseimage.shape[0], baseimage.shape[1])\n102.0s\n170\n---> 47             mask = read_mask( mask_file_name, mask_shape)\n102.0s\n171\n     48         else:\n102.0s\n172\n     49             mask = None\n102.0s\n173\n\n102.0s\n174\n<ipython-input-15-5a248ee894a5> in read_mask(mask_file, mask_shape)\n102.0s\n175\n      8 \n102.0s\n176\n      9     mask = np.zeros(mask_shape, dtype = np.uint8 )\n102.0s\n177\n---> 10     cv2.fillPoly(mask, polys, 1)\n102.0s\n178\n     11     mask = mask.astype(bool)\n102.0s\n179\n     12     return mask\n102.0s\n180\n\n102.0s\n181\nerror: OpenCV(4.4.0) /tmp/pip-req-build-2mquh264/opencv/modules/imgproc/src/drawing.cpp:2395: error: (-215:Assertion failed) p.checkVector(2, CV_32S) >= 0 in function 'fillPoly'",
    "1233871": "Have u noticed that the new data has some \"new features\"?  The format/encoding is  bit different from the older ones.",
    "1233889": "Oh, drat!   Is this documented anywhere?    In any case, I have solved my problem (bad .json format?) by switching to use the RLE representation of the glom masks in ```train.csv```.",
    "1235557": "I'm a bit confused about data availability:\nin \"Data\" tab you can find \"Glomeruli FTU annotations exist for all 20 tissue samples\"; files \"2ec3f1bb9.json\", \"3589adb90.json\", \"57512b7f1.json\", \"aa05346ff.json\", \"d488c759a.json\", that correspond to 5 tiff files in test folder are mentioned in \"HuBMAP-20-dataset_information.csv\", but there's only 15 lines in \"train.csv\" and only anatomical jsons in test foilder. Do we have 15 files with annotation or 20?",
    "1235673": "Seems like a couple of the tiff files might have a different compression method or color space. I'm not sure as the cloud environment I get assigned changes.\n\nHere's how I'm loading the files:\n\n```\nfrom skimage import transform, io\nimport PIL.Image\nPIL.Image.MAX_IMAGE_PIXELS = 100000000000\n\n```\n```\ntry:\n  img = io.imread(img_path).squeeze()\nexcept:\n  img = io.imread(img_path, plugin='pil').squeeze()\n```",
    "1235772": "~~Some of the images are greyscale and only have one channel. You need to account for this when you read them~~",
    "1235869": "Which one is BW? Are you sure?",
    "1235924": "Hmm, I'm using `rasterio` and am only seeing 1 channel in 095bf7a1f, 4ef6695ce, 26dc41664, c68fe75ea, 1e2425f28.\n\nI'll try opening them in GIMP and double check...\n\nEdit: 4ef6695ce has 3 layers in GIMP, not sure why `rasterio` isn't seeing them. Thanks for making me check and apologies for the confusion! :)",
    "1235945": "Unless I've royally screwed up, at least 1e2425f28 isn't grayscale. Here's a tile from that file `1e2425f28_4096_8192_8192_12288`:\n\n<a href='https://postimg.cc/vc4hDrWm' target='_blank'><img src='https://i.postimg.cc/vc4hDrWm/1e2425f28-4096-8192-8192-12288.jpg' border='0' alt='1e2425f28-4096-8192-8192-12288'/></a>\n\nI'm not using a package to rasterize.",
    "1235958": "In case anyone else was facing the same issue as me, the solution is here: https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883#1233186",
    "1236058": "Im using tifffile from python on R. It handle this just fine.",
    "1237338": "In the training set, '095bf7a1f.tiff' is a single channel image (gray scale). \nNot sure if this is intentional since the whole idea of histology staining is to provide some color changes to pathological changes. Should single channel Images be ignored in this case?",
    "1237357": "They are all color images. Theres a couple of posts below about it.",
    "1238279": "https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/224883",
    "1238340": "Thank you very much for helping me out",
    "1255738": "Hi,\nany update on the judges review timeline? The old dates are still online.\nThanks!",
    "1260585": "thanks for updating.",
    "1298883": "addisonhoward  does private set contains data from Different patient numbers from Train or they may contain some of those eg. Test has three patient ids matching to that of Train."
  },
  "source": "meta"
}