{
  "id": 200955,
  "title": "[placeholder] starterkit ... resnet34-unet LB 0.855+",
  "url": "/competitions/hubmap-kidney-segmentation/discussion/200955",
  "author_name": "hengck23",
  "post_date": "2020-12-02T14:11:23.888000",
  "votes": 143,
  "comment_count": 88,
  "views": 0,
  "content": "<p>… to be updated ….</p>\n<ul>\n<li>more to come as i progress and improve my code</li>\n</ul>\n<hr>\n<p>code, trained model, etc are here: <a href=\"https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing</a></p>\n<p>1) 2020-dec-02 : </p>\n<ul>\n<li>image tiling and merging </li>\n<li>able to train and do inference with a simple unet-resnet34 <br>\n(a well-trained single fold LB is about 0.830)</li>\n<li>visualize prediction  </li>\n</ul>\n<p>2) 2020-dec-04 : </p>\n<ul>\n<li>updated augmentation: resnet34-unet LB 0.837 (single fold)</li>\n<li>refer to the discussion at <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626</a> for higher LB 0.845</li>\n</ul>\n<p>3) 2020-dec-11 : </p>\n<ul>\n<li>updated augmentation: more hue, saturation, contrast, intensity variation</li>\n<li>implement enhanced batch normalisation. you can train segmentation from scratch.<br>\nBatch Normalization with Enhanced Linear Transformation<br>\n<a href=\"https://arxiv.org/pdf/2011.14150.pdf\" target=\"_blank\">https://arxiv.org/pdf/2011.14150.pdf</a></li>\n<li>change sampling method: <ul>\n<li>for testing use cortex json</li>\n<li>for training use hue-saturation</li>\n<li>for merging tile, use weighted average to remove boundary artifacts</li></ul></li>\n<li>LB 0.85+ : try different fold and threshold(for probability). the results are quite different</li>\n</ul>\n<p>4) next update</p>\n<ul>\n<li>augmentation to address hue-saturation problem/ ink and markings, etc</li>\n<li>how to ensemble?</li>\n<li>post processing? predict score to each instance?</li>\n</ul>\n<p>….<br>\nlong-termed plan:</p>\n<ul>\n<li>GAN for data generation</li>\n<li>weak supervised using public test</li>\n<li>self supervsied using all kaggle data + external + HUBMAP</li>\n<li>transformer in segmentation modeling …</li>\n</ul>",
  "messages": [
    {
      "id": 1099658,
      "postDate": "2020-12-02T14:11:23.887Z",
      "content": "<p>… to be updated ….</p>\n<ul>\n<li>more to come as i progress and improve my code</li>\n</ul>\n<hr>\n<p>code, trained model, etc are here: <a href=\"https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing</a></p>\n<p>1) 2020-dec-02 : </p>\n<ul>\n<li>image tiling and merging </li>\n<li>able to train and do inference with a simple unet-resnet34 <br>\n(a well-trained single fold LB is about 0.830)</li>\n<li>visualize prediction  </li>\n</ul>\n<p>2) 2020-dec-04 : </p>\n<ul>\n<li>updated augmentation: resnet34-unet LB 0.837 (single fold)</li>\n<li>refer to the discussion at <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626</a> for higher LB 0.845</li>\n</ul>\n<p>3) 2020-dec-11 : </p>\n<ul>\n<li>updated augmentation: more hue, saturation, contrast, intensity variation</li>\n<li>implement enhanced batch normalisation. you can train segmentation from scratch.<br>\nBatch Normalization with Enhanced Linear Transformation<br>\n<a href=\"https://arxiv.org/pdf/2011.14150.pdf\" target=\"_blank\">https://arxiv.org/pdf/2011.14150.pdf</a></li>\n<li>change sampling method: <ul>\n<li>for testing use cortex json</li>\n<li>for training use hue-saturation</li>\n<li>for merging tile, use weighted average to remove boundary artifacts</li></ul></li>\n<li>LB 0.85+ : try different fold and threshold(for probability). the results are quite different</li>\n</ul>\n<p>4) next update</p>\n<ul>\n<li>augmentation to address hue-saturation problem/ ink and markings, etc</li>\n<li>how to ensemble?</li>\n<li>post processing? predict score to each instance?</li>\n</ul>\n<p>….<br>\nlong-termed plan:</p>\n<ul>\n<li>GAN for data generation</li>\n<li>weak supervised using public test</li>\n<li>self supervsied using all kaggle data + external + HUBMAP</li>\n<li>transformer in segmentation modeling …</li>\n</ul>",
      "rawMarkdown": "... to be updated ....\n- more to come as i progress and improve my code\n\n\n---\ncode, trained model, etc are here: https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\n\n1) 2020-dec-02 : \n- image tiling and merging \n- able to train and do inference with a simple unet-resnet34 \n  (a well-trained single fold LB is about 0.830)\n- visualize prediction  \n\n2) 2020-dec-04 : \n- updated augmentation: resnet34-unet LB 0.837 (single fold)\n- refer to the discussion at https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626 for higher LB 0.845\n\n3) 2020-dec-11 : \n- updated augmentation: more hue, saturation, contrast, intensity variation\n- implement enhanced batch normalisation. you can train segmentation from scratch.\nBatch Normalization with Enhanced Linear Transformation\nhttps://arxiv.org/pdf/2011.14150.pdf\n- change sampling method: \n    - for testing use cortex json\n    - for training use hue-saturation\n    - for merging tile, use weighted average to remove boundary artifacts\n- LB 0.85+ : try different fold and threshold(for probability). the results are quite different\n\n4) next update\n- augmentation to address hue-saturation problem/ ink and markings, etc\n- how to ensemble?\n- post processing? predict score to each instance?\n\n....\nlong-termed plan:\n- GAN for data generation\n- weak supervised using public test\n- self supervsied using all kaggle data + external + HUBMAP\n- transformer in segmentation modeling ...\n",
      "votes": 142
    },
    {
      "id": 1100565,
      "postDate": "2020-12-03T06:46:59.413Z",
      "content": "<p>my prediction for the final ranking:</p>\n<ul>\n<li>public LB : 0.90 ~ 0.93 (shakeup 0.01)</li>\n</ul>\n<p>my prediction is based on:</p>\n<ul>\n<li>train, validation error</li>\n<li>human error (I try to reannotate some train images)</li>\n<li>visual inspection of prediction results on train/test data </li>\n<li>study the distribution of probability scores and embedding (e.g. tsne)</li>\n</ul>\n<p>key to winning<br>\n1) definitely data!</p>\n<ul>\n<li>how to generate data? augmentation/GAN …</li>\n<li>offline pseudo label (public test) … definitely helps!</li>\n<li>online pseudo label (hidden private test) </li>\n<li>external data</li>\n</ul>\n<p>2) post processing</p>\n<ul>\n<li>boundary artifacts …. filtering of FP (some are due to wrong label) … train another model for post processing</li>\n<li>pre-processing? (not sure if you get the outliers in private test?)<ul>\n<li>ink/marking on stains</li>\n<li>different stain hue</li>\n<li>scale </li></ul></li>\n</ul>\n<p>3) modeling</p>\n<ul>\n<li>I feel that this is less important. standard models work well, e.g. fpn, unet.</li>\n<li>important thing is how to design one that suits your magic, e.g. design one for pseudo label , post-processing fp, high speed, good memory, etc …</li>\n</ul>",
      "rawMarkdown": "my prediction for the final ranking:\n- public LB : 0.90 ~ 0.93 (shakeup 0.01)\n\nmy prediction is based on:\n- train, validation error\n- human error (I try to reannotate some train images)\n- visual inspection of prediction results on train/test data \n- study the distribution of probability scores and embedding (e.g. tsne)\n\nkey to winning\n1) definitely data!\n- how to generate data? augmentation/GAN ...\n- offline pseudo label (public test) ... definitely helps!\n- online pseudo label (hidden private test) \n- external data\n\n\n2) post processing\n- boundary artifacts .... filtering of FP (some are due to wrong label) ... train another model for post processing\n- pre-processing? (not sure if you get the outliers in private test?)\n  - ink/marking on stains\n  - different stain hue\n  - scale \n\n\n3) modeling\n- I feel that this is less important. standard models work well, e.g. fpn, unet.\n- important thing is how to design one that suits your magic, e.g. design one for pseudo label , post-processing fp, high speed, good memory, etc ...",
      "votes": 12,
      "replies": [
        {
          "id": 1101334,
          "postDate": "2020-12-03T20:14:23.520Z",
          "content": "<p>Can you expound more  on <code>study the distribution of probability scores and embedding (e.g. tsne)</code> ?</p>",
          "rawMarkdown": "Can you expound more  on `study the distribution of probability scores and embedding (e.g. tsne)` ?"
        }
      ]
    },
    {
      "id": 1104643,
      "postDate": "2020-12-07T06:34:03.880Z",
      "content": "<p>preview of next code update:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Febe5ebab1707bfdd11f4f0a8df7f421b%2FSelection_146.png?generation=1607322785976029&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F6dcc1c77979b6362e96a6269ba9cbb35%2FSelection_136.png?generation=1607322811733295&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F89e99250b0e0ce282c306d3b5f702438%2FSelection_137.png?generation=1607322839958767&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "preview of next code update:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Febe5ebab1707bfdd11f4f0a8df7f421b%2FSelection_146.png?generation=1607322785976029&alt=media)\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F6dcc1c77979b6362e96a6269ba9cbb35%2FSelection_136.png?generation=1607322811733295&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F89e99250b0e0ce282c306d3b5f702438%2FSelection_137.png?generation=1607322839958767&alt=media)",
      "votes": 8,
      "replies": [
        {
          "id": 1104702,
          "postDate": "2020-12-07T07:22:20.177Z",
          "content": "<p>this is a competition where CV/LB correlation is not enough</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F406dae1e113fc61cf0d9020f5f4ae566%2FSelection_149.png?generation=1607325729310460&amp;alt=media\" alt=\"\"></p>",
          "rawMarkdown": "this is a competition where CV/LB correlation is not enough\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F406dae1e113fc61cf0d9020f5f4ae566%2FSelection_149.png?generation=1607325729310460&alt=media)",
          "votes": 2
        },
        {
          "id": 1104727,
          "postDate": "2020-12-07T07:33:20.117Z",
          "rawMarkdown": "",
          "votes": 1,
          "isDeleted": true
        },
        {
          "id": 1108748,
          "postDate": "2020-12-11T01:10:20.860Z",
          "content": "<p>code preview: finally, getting rid of the boundary artifacts, see hubmap_v2.py</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F4c45306d59d92c213a15d84599162aa3%2FSelection_181.png?generation=1607648975343396&amp;alt=media\" alt=\"\"></p>",
          "rawMarkdown": "code preview: finally, getting rid of the boundary artifacts, see hubmap_v2.py\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F4c45306d59d92c213a15d84599162aa3%2FSelection_181.png?generation=1607648975343396&alt=media)",
          "votes": 7
        },
        {
          "id": 1108791,
          "postDate": "2020-12-11T02:11:25.237Z",
          "content": "<p>Nice, thanks for sharing</p>",
          "rawMarkdown": "Nice, thanks for sharing"
        }
      ]
    },
    {
      "id": 1103565,
      "postDate": "2020-12-06T03:18:52.590Z",
      "content": "<p>external dataset:</p>\n<p><a href=\"https://data.mendeley.com/datasets/k7nvtgn2x6/3\" target=\"_blank\">https://data.mendeley.com/datasets/k7nvtgn2x6/3</a><br>\n\"Data for glomeruli characterization in histopathological images\"<br>\nDATASET_A: Raw data with 31 whole slide images (WSI) in SVS format. The size of the WSI range between 21651x10498 pixels and 49799 x 32359 pixles acquired at 20x. </p>\n<p>to read SVS format: <a href=\"http://www.slideio.com/\" target=\"_blank\">http://www.slideio.com/</a></p>",
      "rawMarkdown": "external dataset:\n\nhttps://data.mendeley.com/datasets/k7nvtgn2x6/3\n\"Data for glomeruli characterization in histopathological images\"\nDATASET_A: Raw data with 31 whole slide images (WSI) in SVS format. The size of the WSI range between 21651x10498 pixels and 49799 x 32359 pixles acquired at 20x. \n\nto read SVS format: http://www.slideio.com/",
      "votes": 8
    },
    {
      "id": 1109794,
      "postDate": "2020-12-12T04:32:11.793Z",
      "content": "<h1>🐜 🐜  🐜</h1>\n<p>2020-dec-11 : Bug !!!!   The test image cortex structure region is not accurate. sample windows (tiles) from it will miss some glomerulus<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F25dce65ac9366dfa4f8054007e000278%2FSelection_197.png?generation=1607747567069688&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "# 🐜 🐜  🐜 \n2020-dec-11 : Bug !!!!   The test image cortex structure region is not accurate. sample windows (tiles) from it will miss some glomerulus\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F25dce65ac9366dfa4f8054007e000278%2FSelection_197.png?generation=1607747567069688&alt=media)",
      "votes": 5,
      "replies": [
        {
          "id": 1149345,
          "postDate": "2021-01-11T19:23:32.737Z",
          "content": "<p>Yes, it does so for many test images. Especially for afa5e… it looks like the polygon for the cortex is wrongly marked by the competition hosts and if we apply that mask to fill in and clip out non-cortex glomeruli we end up with not all of them detected. </p>\n<p>Here I have a sample:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1586379%2Ffec7d95dc30de76c23913fc6ef7fa1b1%2Fdownload.png?generation=1610392848512076&amp;alt=media\" alt=\"\"><br>\nYou can see that the masks are clipped in the far right.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1586379%2Fa3199f4e38d2fa51a136f018d447e41e%2Fdownload.png?generation=1610392929660788&amp;alt=media\" alt=\"\"></p>\n<p>Is this the same behavior you are mentioning? If so, have you proceeded to use this polygon to clip unwanted glomeruli or not? <br>\nThanks,<br>\nCK</p>",
          "rawMarkdown": "Yes, it does so for many test images. Especially for afa5e... it looks like the polygon for the cortex is wrongly marked by the competition hosts and if we apply that mask to fill in and clip out non-cortex glomeruli we end up with not all of them detected. \n\nHere I have a sample:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1586379%2Ffec7d95dc30de76c23913fc6ef7fa1b1%2Fdownload.png?generation=1610392848512076&alt=media)\nYou can see that the masks are clipped in the far right.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1586379%2Fa3199f4e38d2fa51a136f018d447e41e%2Fdownload.png?generation=1610392929660788&alt=media)\n\nIs this the same behavior you are mentioning? If so, have you proceeded to use this polygon to clip unwanted glomeruli or not? \nThanks,\nCK"
        }
      ]
    },
    {
      "id": 1099802,
      "postDate": "2020-12-02T15:49:47.637Z",
      "content": "<p>my plan is to modify the below to add transformer to unet.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fd356e6c94eb7c5cfb5732b9083ffaf22%2FSelection_034.png?generation=1606924184309685&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "my plan is to modify the below to add transformer to unet.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fd356e6c94eb7c5cfb5732b9083ffaf22%2FSelection_034.png?generation=1606924184309685&alt=media)\n",
      "votes": 5,
      "replies": [
        {
          "id": 1100491,
          "postDate": "2020-12-03T05:40:10.923Z",
          "content": "<p>Can you provide a link to this article? Thank you</p>",
          "rawMarkdown": "Can you provide a link to this article? Thank you",
          "votes": 1
        },
        {
          "id": 1108020,
          "postDate": "2020-12-10T07:28:53.767Z",
          "content": "<p><a href=\"https://arxiv.org/abs/2011.09763\" target=\"_blank\">https://arxiv.org/abs/2011.09763</a> <a href=\"https://www.kaggle.com/tikboa\" target=\"_blank\">@tikboa</a> </p>",
          "rawMarkdown": "https://arxiv.org/abs/2011.09763 @tikboa ",
          "votes": 1
        }
      ]
    },
    {
      "id": 1103498,
      "postDate": "2020-12-06T01:11:14.300Z",
      "content": "<p>a few probing tricks used in the previous challenge</p>\n<p>(1) submit only one image at a time : </p>\n<ul>\n<li>enable you to analyze which image is the best, which is the worst. <br>\n(e.g. you discover the problem of low image score is blurry image, different scale, etc …)</li>\n</ul>\n<p>(2) submit prediction for only a certain region of the image (e.g likely to be fp region)</p>\n<ul>\n<li>same as above. </li>\n</ul>\n<p>(3) submit a single prediction area (e.g. a single rectangle covering the whole image or only part of the image. here, we actually the polygon of the cortex, etc)</p>\n<ul>\n<li>enable you to know the density/number object instances if you know the distribution or average size of the object)</li>\n<li>enable you to know the total area of object instances</li>\n</ul>\n<p>(4) submit only object instance of only one size</p>\n<ul>\n<li>let you know the relationship between dice score and size.  it lets you answer questions like:<ul>\n<li>should I just ignore small ground truth</li>\n<li>should I focus on detecting more instances or just improving dice score of the current predicted instances</li></ul></li>\n</ul>\n<p>(5) submit only object instance of very high confidence score (or within certain score band)</p>\n<ul>\n<li>combined with (3) which tells you the total area, you roughly know the number of correct detection and average dice score. some competition requires a score re-calibration</li>\n</ul>\n<p>warning!!! </p>\n<ul>\n<li>you can only probe the public set but not the private one</li>\n<li>the more you probe and know about the public set, you are over-fitting the public set<br>\n(knowing more is not always good … know your assumption!)</li>\n</ul>",
      "rawMarkdown": "a few probing tricks used in the previous challenge\n\n(1) submit only one image at a time : \n- enable you to analyze which image is the best, which is the worst. \n(e.g. you discover the problem of low image score is blurry image, different scale, etc ...)\n\n(2) submit prediction for only a certain region of the image (e.g likely to be fp region)\n- same as above. \n\n(3) submit a single prediction area (e.g. a single rectangle covering the whole image or only part of the image. here, we actually the polygon of the cortex, etc)\n- enable you to know the density/number object instances if you know the distribution or average size of the object)\n- enable you to know the total area of object instances\n\n(4) submit only object instance of only one size\n- let you know the relationship between dice score and size.  it lets you answer questions like:\n   - should I just ignore small ground truth\n   - should I focus on detecting more instances or just improving dice score of the current predicted instances\n\n(5) submit only object instance of very high confidence score (or within certain score band)\n- combined with (3) which tells you the total area, you roughly know the number of correct detection and average dice score. some competition requires a score re-calibration\n\n\nwarning!!! \n- you can only probe the public set but not the private one\n- the more you probe and know about the public set, you are over-fitting the public set\n(knowing more is not always good ... know your assumption!)",
      "votes": 6,
      "replies": [
        {
          "id": 1103520,
          "postDate": "2020-12-06T01:57:54.037Z",
          "content": "<p>as an example of problem finding<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F58d4ae6b317bdef81f5c7f0ebff8ce2f%2FSelection_103.png?generation=1607219871325761&amp;alt=media\" alt=\"\"></p>\n<p>… maybe I need a simple color transform or stain normalisation network?</p>",
          "rawMarkdown": "as an example of problem finding\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F58d4ae6b317bdef81f5c7f0ebff8ce2f%2FSelection_103.png?generation=1607219871325761&alt=media)\n\n\n... maybe I need a simple color transform or stain normalisation network?",
          "votes": 4
        },
        {
          "id": 1104178,
          "postDate": "2020-12-06T17:18:55.853Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a>, i have one question, firstly we have no public test set segment labels, but how you calculated the dice score as your upper excel shows?👀</p>",
          "rawMarkdown": "Hi @hengck23, i have one question, firstly we have no public test set segment labels, but how you calculated the dice score as your upper excel shows?👀"
        },
        {
          "id": 1104180,
          "postDate": "2020-12-06T17:24:33.507Z",
          "content": "<p>I get it, you just use plb to calculate the dice score, smart idea</p>",
          "rawMarkdown": "I get it, you just use plb to calculate the dice score, smart idea"
        }
      ]
    },
    {
      "id": 1108801,
      "postDate": "2020-12-11T02:37:19.677Z",
      "content": "<p>Boundary loss<br>\n<a href=\"http://proceedings.mlr.press/v102/kervadec19a/kervadec19a.pdf\" target=\"_blank\">http://proceedings.mlr.press/v102/kervadec19a/kervadec19a.pdf</a><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb94aa4a2408f2fe36dc449c3ee21aae7%2FSelection_186.png?generation=1607654237573602&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "Boundary loss\nhttp://proceedings.mlr.press/v102/kervadec19a/kervadec19a.pdf\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb94aa4a2408f2fe36dc449c3ee21aae7%2FSelection_186.png?generation=1607654237573602&alt=media)",
      "votes": 3
    },
    {
      "id": 1107840,
      "postDate": "2020-12-10T02:04:29.363Z",
      "content": "<p>counting network or label propagation<br>\n<a href=\"https://www.robots.ox.ac.uk/~vgg/publications/2018/Lu18/lu18.pdf\" target=\"_blank\">https://www.robots.ox.ac.uk/~vgg/publications/2018/Lu18/lu18.pdf</a><br>\nClass-Agnostic Counting<br>\nErika Lu, Weidi Xie, and Andrew Zisserman</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c047451090e1eaec5618ee46fcf9056%2FSelection_165.png?generation=1607565826614306&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb3b6c05910f373903a428b57f9ba496a%2FSelection_164.png?generation=1607565865568893&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "counting network or label propagation\nhttps://www.robots.ox.ac.uk/~vgg/publications/2018/Lu18/lu18.pdf\nClass-Agnostic Counting\nErika Lu, Weidi Xie, and Andrew Zisserman\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c047451090e1eaec5618ee46fcf9056%2FSelection_165.png?generation=1607565826614306&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb3b6c05910f373903a428b57f9ba496a%2FSelection_164.png?generation=1607565865568893&alt=media)\n",
      "votes": 3
    },
    {
      "id": 1100495,
      "postDate": "2020-12-03T05:48:05.003Z",
      "content": "<p>all data analysis prize (judge prize) related stuffs here:</p>\n<ol>\n<li><p>you may want to learn about huBMAP here<br>\n<a href=\"https://expand.iu.edu/browse/sice/cns/courses/hubmap-visible-human-mooc\" target=\"_blank\">https://expand.iu.edu/browse/sice/cns/courses/hubmap-visible-human-mooc</a><br>\n<a href=\"https://www.youtube.com/channel/UCbSvPJ9dXASL14KoDeutMFg/videos\" target=\"_blank\">https://www.youtube.com/channel/UCbSvPJ9dXASL14KoDeutMFg/videos</a></p></li>\n<li><p>some interesting analysis of your model performance: e.g. worst-case scenario<br>\n\"Identifying Model Weakness with Adversarial Examiner\"-AAAI 2020</p></li>\n</ol>",
      "rawMarkdown": "all data analysis prize (judge prize) related stuffs here:\n1. you may want to learn about huBMAP here\nhttps://expand.iu.edu/browse/sice/cns/courses/hubmap-visible-human-mooc\nhttps://www.youtube.com/channel/UCbSvPJ9dXASL14KoDeutMFg/videos\n\n2. some interesting analysis of your model performance: e.g. worst-case scenario\n\"Identifying Model Weakness with Adversarial Examiner\"-AAAI 2020",
      "votes": 3
    },
    {
      "id": 1099795,
      "postDate": "2020-12-02T15:45:38.507Z",
      "content": "<p>code version: 2020-dec-02</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe08539770a9921ea8ed4714e007c3321%2FSelection_029.png?generation=1606951417354956&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9d7a596419d8b050a5abc2eba6e08c18%2FSelection_030.png?generation=1606951459356411&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "code version: 2020-dec-02\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe08539770a9921ea8ed4714e007c3321%2FSelection_029.png?generation=1606951417354956&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9d7a596419d8b050a5abc2eba6e08c18%2FSelection_030.png?generation=1606951459356411&alt=media)",
      "votes": 3,
      "replies": [
        {
          "id": 1100230,
          "postDate": "2020-12-02T23:36:36.753Z",
          "content": "<p>the boundary prediction for a tile is a headache.</p>\n<ul>\n<li>I can do post processing to merge predictions 2 neighboring tiles</li>\n<li>maybe I should remove boundary ground truth if they are too small … after all, my training tiles are overlapping</li>\n</ul>",
          "rawMarkdown": "the boundary prediction for a tile is a headache.\n- I can do post processing to merge predictions 2 neighboring tiles\n- maybe I should remove boundary ground truth if they are too small ... after all, my training tiles are overlapping"
        },
        {
          "id": 1105419,
          "postDate": "2020-12-07T21:15:16.530Z",
          "content": "<p>This is a very thorough observation. Regarding of the ground truth, I don't know how this completion get the ground truth. Based on my experiences on medical images for labeling ground truth with MD, nurses, techs etc, it is subjective in some extent and easy for making human errors. Similarly for this competition, it is not surprising to see artifacts in these images.  </p>",
          "rawMarkdown": "This is a very thorough observation. Regarding of the ground truth, I don't know how this completion get the ground truth. Based on my experiences on medical images for labeling ground truth with MD, nurses, techs etc, it is subjective in some extent and easy for making human errors. Similarly for this competition, it is not surprising to see artifacts in these images.  ",
          "votes": 1
        },
        {
          "id": 1105660,
          "postDate": "2020-12-08T04:14:23.080Z",
          "content": "<p><a href=\"https://www.kaggle.com/qingtaozhou\" target=\"_blank\">@qingtaozhou</a> <br>\n\"Based on my experiences on medical images for labeling ground truth with MD, nurses, techs \"</p>\n<p>Can you tell me more about your experiences on labeling tissue images? </p>\n<p>from my inspection of images for this kidney competition, I think:</p>\n<ol>\n<li>there is some software that does automatic or semi-automatic labeling first</li>\n<li>then there are some expert to verify or correct the initial annotations</li>\n</ol>\n<p>(this is quite obvious because some errors are machine-made and cannot be made by a human)</p>",
          "rawMarkdown": "@qingtaozhou \n\"Based on my experiences on medical images for labeling ground truth with MD, nurses, techs \"\n\nCan you tell me more about your experiences on labeling tissue images? \n\nfrom my inspection of images for this kidney competition, I think:\n1. there is some software that does automatic or semi-automatic labeling first\n2. then there are some expert to verify or correct the initial annotations\n\n(this is quite obvious because some errors are machine-made and cannot be made by a human)"
        },
        {
          "id": 1105947,
          "postDate": "2020-12-08T11:08:04.570Z",
          "content": "<p>You are right. These are two steps for labeling the ground truth.  There are many deep learning products on medical image field either for clinical purpose(FDA approved) or for research purpose.  However, technology has limits. If you use two different software (FDA approval) for the same image segmentation, there are still differences for labeling.   For the second step that you mention, experts verify or correct the initial annotations. My understanding is that experts make subject decision or may miss something. In the clinical settings, people usually use inter-observer to evaluate the variation for the same thing. For this competition, I don't know how they labeled the ground truth and didn't have experiences specifically for glomeruli. My comments could be biased. </p>",
          "rawMarkdown": "You are right. These are two steps for labeling the ground truth.  There are many deep learning products on medical image field either for clinical purpose(FDA approved) or for research purpose.  However, technology has limits. If you use two different software (FDA approval) for the same image segmentation, there are still differences for labeling.   For the second step that you mention, experts verify or correct the initial annotations. My understanding is that experts make subject decision or may miss something. In the clinical settings, people usually use inter-observer to evaluate the variation for the same thing. For this competition, I don't know how they labeled the ground truth and didn't have experiences specifically for glomeruli. My comments could be biased. "
        }
      ]
    },
    {
      "id": 1108798,
      "postDate": "2020-12-11T02:31:48.170Z",
      "content": "<p>dice is non-symmetric metric … don't be afraid to detect more and make some mistake<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F149b10fd4c34d39acda60b89707cab45%2FSelection_185.png?generation=1607653899026735&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "dice is non-symmetric metric ... don't be afraid to detect more and make some mistake\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F149b10fd4c34d39acda60b89707cab45%2FSelection_185.png?generation=1607653899026735&alt=media)",
      "votes": 4
    },
    {
      "id": 1099660,
      "postDate": "2020-12-02T14:12:39.877Z",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F83e682c2a26ca0a9ca6f998d5987bd9f%2FSelection_029.png?generation=1606918300495338&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F7c148869c78d728d57468afd108005c2%2FSelection_030.png?generation=1606918314707954&amp;alt=media\" alt=\"\"></p>\n<p>see attached \"data.py\"</p>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F83e682c2a26ca0a9ca6f998d5987bd9f%2FSelection_029.png?generation=1606918300495338&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F7c148869c78d728d57468afd108005c2%2FSelection_030.png?generation=1606918314707954&alt=media)\n\n\nsee attached \"data.py\"",
      "votes": 4,
      "replies": [
        {
          "id": 1108032,
          "postDate": "2020-12-10T07:47:25.450Z",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> Could you please explain how you decide to reject/keep tiles? Specifically could I please ask what does this bit of code do? </p>\n<pre><code>    height, width, _ = image_small.shape\n    vv = cv2.resize(image_small, dsize=None, fx=1 / 32, fy=1 / 32, \n        interpolation=cv2.INTER_LINEAR)\n    vv = cv2.cvtColor(vv, cv2.COLOR_RGB2HSV)\n    # image_show('v[0]', v[:,:,0])\n    # image_show('v[1]', v[:,:,1])\n    # image_show('v[2]', v[:,:,2])\n    # cv2.waitKey(0)\n    vv = (vv[:, :, 1] &gt; 32).astype(np.float32)\n    vv = cv2.resize(vv, dsize=(width, height), interpolation=cv2.INTER_LINEAR)\n</code></pre>",
          "rawMarkdown": "@hengck23 Could you please explain how you decide to reject/keep tiles? Specifically could I please ask what does this bit of code do? \n\n```\n    height, width, _ = image_small.shape\n    vv = cv2.resize(image_small, dsize=None, fx=1 / 32, fy=1 / 32, \n        interpolation=cv2.INTER_LINEAR)\n    vv = cv2.cvtColor(vv, cv2.COLOR_RGB2HSV)\n    # image_show('v[0]', v[:,:,0])\n    # image_show('v[1]', v[:,:,1])\n    # image_show('v[2]', v[:,:,2])\n    # cv2.waitKey(0)\n    vv = (vv[:, :, 1] > 32).astype(np.float32)\n    vv = cv2.resize(vv, dsize=(width, height), interpolation=cv2.INTER_LINEAR)\n```"
        },
        {
          "id": 1108038,
          "postDate": "2020-12-10T07:58:10.600Z",
          "content": "<p>in the image, a window (or called tile) can contain some tissue pixels or no tissue pixels at all.</p>\n<p>those that does not contain tissue pixels at all could be rejected immediately and not processed (because we know that the number of glomeruli detected must be null)</p>\n<p>we convert rgb image into hsv. we assume tissue pixel is one that saturation value &gt; 32.</p>\n<hr>\n<p>note, </p>\n<ul>\n<li>a better way is to use the mask of Cortex region provided in the json file provided.</li>\n<li>later I find that threshold of saturation value &gt; 32. works for all images in the train. but it fails for one image in the test (see discussion at the top of the thread). you may want  reset the threshold, etc.</li>\n<li>in the worst case, you can reject none and process all windows</li>\n</ul>",
          "rawMarkdown": "in the image, a window (or called tile) can contain some tissue pixels or no tissue pixels at all.\n\nthose that does not contain tissue pixels at all could be rejected immediately and not processed (because we know that the number of glomeruli detected must be null)\n\nwe convert rgb image into hsv. we assume tissue pixel is one that saturation value > 32.\n\n---\nnote, \n- a better way is to use the mask of Cortex region provided in the json file provided.\n- later I find that threshold of saturation value > 32. works for all images in the train. but it fails for one image in the test (see discussion at the top of the thread). you may want  reset the threshold, etc.\n- in the worst case, you can reject none and process all windows",
          "votes": 6
        },
        {
          "id": 1108708,
          "postDate": "2020-12-10T23:39:56.960Z",
          "content": "<p>Thank you - that makes sense. :)</p>",
          "rawMarkdown": "Thank you - that makes sense. :)"
        }
      ]
    },
    {
      "id": 1112076,
      "postDate": "2020-12-14T09:06:47.163Z",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3964990%2Fc47049044214a2a580b8e156a6400526%2FSnipaste_2020-12-14_17-02-56.jpg?generation=1607936611952354&amp;alt=media\" alt=\"\"><br>\nDear washen,im not really understand about why ‘for training use hue-saturation’ is related to 'change sampling method'.Why isn't it related to augmentation?<br>\nThanks!</p>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3964990%2Fc47049044214a2a580b8e156a6400526%2FSnipaste_2020-12-14_17-02-56.jpg?generation=1607936611952354&alt=media)\nDear washen,im not really understand about why ‘for training use hue-saturation’ is related to 'change sampling method'.Why isn't it related to augmentation?\nThanks!",
      "votes": 1,
      "replies": [
        {
          "id": 1115276,
          "postDate": "2020-12-16T06:46:12.967Z",
          "content": "<p>I think, training images which are above some specific saturation threshold will be sampled for training, rest will be rejected. This is the idea behind sampling for training set.</p>",
          "rawMarkdown": "I think, training images which are above some specific saturation threshold will be sampled for training, rest will be rejected. This is the idea behind sampling for training set."
        },
        {
          "id": 1117396,
          "postDate": "2020-12-18T02:27:54.463Z",
          "content": "<p>thanks for your reply,i get it!</p>",
          "rawMarkdown": "thanks for your reply,i get it!"
        }
      ]
    },
    {
      "id": 1110678,
      "postDate": "2020-12-13T00:37:18.907Z",
      "content": "<p><a href=\"https://arxiv.org/abs/2012.05780\" target=\"_blank\">https://arxiv.org/abs/2012.05780</a><br>\nOneNet: Towards End-to-End One-Stage Object Detection<br>\nobject detection without NMS</p>",
      "rawMarkdown": "https://arxiv.org/abs/2012.05780\nOneNet: Towards End-to-End One-Stage Object Detection\nobject detection without NMS",
      "votes": 1
    },
    {
      "id": 1100673,
      "postDate": "2020-12-03T08:39:13.490Z",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> <br>\nThanks for posting! This is a great help.</p>\n<p>By the way, your code only predicts public test data, <br>\nbut have you tried predicting it for private test data as well?</p>\n<pre><code>def make_image_id (mode):\n    ...\n    test_image_id = {\n        0 : 'b9a3865fc',\n        1 : 'b2dc8411c',\n        2 : '26dc41664',\n        3 : 'c68fe75ea',\n        4 : 'afa5e8098',\n    }\n    ...\n    if 'test'==mode:\n        test_id = [ test_image_id[i] for i in [0,1,2,3,4] ]\n\n        return test_id\n</code></pre>",
      "rawMarkdown": "@hengck23 \nThanks for posting! This is a great help.\n\nBy the way, your code only predicts public test data, \nbut have you tried predicting it for private test data as well?\n\n```\ndef make_image_id (mode):\n    ...\n    test_image_id = {\n        0 : 'b9a3865fc',\n        1 : 'b2dc8411c',\n        2 : '26dc41664',\n        3 : 'c68fe75ea',\n        4 : 'afa5e8098',\n    }\n    ...\n    if 'test'==mode:\n        test_id = [ test_image_id[i] for i in [0,1,2,3,4] ]\n\n        return test_id\n```",
      "votes": 1,
      "replies": [
        {
          "id": 1100683,
          "postDate": "2020-12-03T08:45:54.427Z",
          "content": "<p>not yet. the code is still under development</p>",
          "rawMarkdown": "not yet. the code is still under development\n",
          "votes": 2
        },
        {
          "id": 1100686,
          "postDate": "2020-12-03T08:47:28.963Z",
          "content": "<p>I see. Thank you!</p>",
          "rawMarkdown": "I see. Thank you!"
        },
        {
          "id": 1126875,
          "postDate": "2020-12-26T04:13:33.507Z",
          "content": "<p>hello, can you accomplish the process of evaluating private test data?</p>",
          "rawMarkdown": "hello, can you accomplish the process of evaluating private test data?"
        },
        {
          "id": 1127061,
          "postDate": "2020-12-26T08:09:33.853Z",
          "content": "<p>hello, frog brother. today I use your run_submit.py to submit not purely submission.csv generated in offline, but it occurs the problem as below:<br>\nyour notebook tried to allocate more memory than is available.it has restarted<br>\ni can't to fix it, could you give me some suggestions? thanks very much!</p>",
          "rawMarkdown": "hello, frog brother. today I use your run_submit.py to submit not purely submission.csv generated in offline, but it occurs the problem as below:\nyour notebook tried to allocate more memory than is available.it has restarted\ni can't to fix it, could you give me some suggestions? thanks very much!"
        },
        {
          "id": 1230660,
          "postDate": "2021-03-08T10:38:22.567Z",
          "content": "<p>hello, could you please tell me that where to get the private test set image? a lot thanks!</p>",
          "rawMarkdown": "hello, could you please tell me that where to get the private test set image? a lot thanks!"
        }
      ]
    },
    {
      "id": 1129375,
      "postDate": "2020-12-28T09:47:51.557Z",
      "content": "<p>direct polygon detection<br>\n<a href=\"https://www.youtube.com/watch?v=sysySMr3YN4\" target=\"_blank\">https://www.youtube.com/watch?v=sysySMr3YN4</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c545ac92b64a6aba27d749ecd407753%2FSelection_224.png?generation=1609148868739418&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "direct polygon detection\nhttps://www.youtube.com/watch?v=sysySMr3YN4\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c545ac92b64a6aba27d749ecd407753%2FSelection_224.png?generation=1609148868739418&alt=media)",
      "votes": 2
    },
    {
      "id": 1102675,
      "postDate": "2020-12-05T07:10:17.627Z",
      "content": "<p>the json file actually provide the ground truth ploygon coordinates.<br>\nhence you can use the following works:</p>\n<ul>\n<li>\"PolyTransform: Deep Polygon Transformer for Instance Segmentation\"</li>\n</ul>\n<p><a href=\"https://openaccess.thecvf.com/content_CVPR_2020/papers/Liang_PolyTransform_Deep_Polygon_Transformer_for_Instance_Segmentation_CVPR_2020_paper.pdf\" target=\"_blank\">https://openaccess.thecvf.com/content_CVPR_2020/papers/Liang_PolyTransform_Deep_Polygon_Transformer_for_Instance_Segmentation_CVPR_2020_paper.pdf</a></p>\n<ul>\n<li><p>Annotating Object Instances with a Polygon-RNN<br>\nEfficient Annotation of Segmentation Datasets with Polygon-RNN++<br>\n<a href=\"http://www.cs.toronto.edu/polyrnn/poly_cvpr17/\" target=\"_blank\">http://www.cs.toronto.edu/polyrnn/poly_cvpr17/</a><br>\n<a href=\"http://www.cs.toronto.edu/polyrnn/\" target=\"_blank\">http://www.cs.toronto.edu/polyrnn/</a></p></li>\n<li><p>Fast Interactive Object Annotation with Curve-GCN<br>\n<a href=\"https://arxiv.org/pdf/1903.06874.pdf\" target=\"_blank\">https://arxiv.org/pdf/1903.06874.pdf</a></p></li>\n<li><p>Cell Detection with Star-convex Polygons<br>\n<a href=\"https://arxiv.org/pdf/1806.03535.pdf\" target=\"_blank\">https://arxiv.org/pdf/1806.03535.pdf</a></p></li>\n</ul>",
      "rawMarkdown": "the json file actually provide the ground truth ploygon coordinates.\nhence you can use the following works:\n- \"PolyTransform: Deep Polygon Transformer for Instance Segmentation\"\n\nhttps://openaccess.thecvf.com/content_CVPR_2020/papers/Liang_PolyTransform_Deep_Polygon_Transformer_for_Instance_Segmentation_CVPR_2020_paper.pdf\n\n\n- Annotating Object Instances with a Polygon-RNN\nEfficient Annotation of Segmentation Datasets with Polygon-RNN++\nhttp://www.cs.toronto.edu/polyrnn/poly_cvpr17/\nhttp://www.cs.toronto.edu/polyrnn/\n\n- Fast Interactive Object Annotation with Curve-GCN\nhttps://arxiv.org/pdf/1903.06874.pdf\n\n- Cell Detection with Star-convex Polygons\nhttps://arxiv.org/pdf/1806.03535.pdf",
      "votes": 2
    },
    {
      "id": 1101335,
      "postDate": "2020-12-03T20:15:22.583Z",
      "content": "<p>some interesting papers. the keyword for googling is \"WSI whole slide image\":</p>\n<p><a href=\"https://paperswithcode.com/task/whole-slide-images\" target=\"_blank\">https://paperswithcode.com/task/whole-slide-images</a>.</p>\n<ul>\n<li><a href=\"https://arxiv.org/pdf/2007.13952v1.pdf\" target=\"_blank\">https://arxiv.org/pdf/2007.13952v1.pdf</a><br>\nEasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology<br>\n(maybe some part of it can be used in Judge Prize)</li>\n</ul>",
      "rawMarkdown": "some interesting papers. the keyword for googling is \"WSI whole slide image\":\n\nhttps://paperswithcode.com/task/whole-slide-images.\n\n- https://arxiv.org/pdf/2007.13952v1.pdf\nEasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology\n(maybe some part of it can be used in Judge Prize)\n\n",
      "votes": 2,
      "replies": [
        {
          "id": 1101787,
          "postDate": "2020-12-04T09:01:39.997Z",
          "content": "<p>this paper only changes one line of code</p>\n<p>\"Specifically, rather than using a single neuron, we propose to additionally consider<br>\neach neuron’s neighborhood for calculating the outputs of the linear transformation\"</p>\n<p>Batch Normalization with Enhanced Linear Transformation<br>\n<a href=\"https://arxiv.org/pdf/2011.14150.pdf\" target=\"_blank\">https://arxiv.org/pdf/2011.14150.pdf</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F91ff4f86a1ac137aa7fd8ab43636ecd1%2FSelection_028.png?generation=1607072497284444&amp;alt=media\" alt=\"\"></p>",
          "rawMarkdown": "this paper only changes one line of code\n\n\"Specifically, rather than using a single neuron, we propose to additionally consider\neach neuron’s neighborhood for calculating the outputs of the linear transformation\"\n\nBatch Normalization with Enhanced Linear Transformation\nhttps://arxiv.org/pdf/2011.14150.pdf\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F91ff4f86a1ac137aa7fd8ab43636ecd1%2FSelection_028.png?generation=1607072497284444&alt=media)\n",
          "votes": 2
        }
      ]
    },
    {
      "id": 1100701,
      "postDate": "2020-12-03T08:56:51.260Z",
      "content": "<p>You mentioned:</p>\n<blockquote>\n  <p>human error (I try to reannotate some train images)</p>\n</blockquote>\n<p>Is re-annotating allowed? I see some annotation errors in the training set as well.</p>",
      "rawMarkdown": "You mentioned:\n\n> human error (I try to reannotate some train images)\n\nIs re-annotating allowed? I see some annotation errors in the training set as well.",
      "votes": 2,
      "replies": [
        {
          "id": 1100702,
          "postDate": "2020-12-03T08:58:58.950Z",
          "content": "<p>yes, it is allowed for train</p>",
          "rawMarkdown": "yes, it is allowed for train\n\n ",
          "votes": 2
        },
        {
          "id": 1101917,
          "postDate": "2020-12-04T11:59:22.100Z",
          "content": "<p>hi, what tool are you using for annotation. i am not able to find a tool for annotating tiff files </p>",
          "rawMarkdown": "hi, what tool are you using for annotation. i am not able to find a tool for annotating tiff files "
        },
        {
          "id": 1101957,
          "postDate": "2020-12-04T12:55:55.433Z",
          "content": "<p>generate the mask in png, then just use GIMP to draw or erase the mask.<br>\nsee <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200070\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200070</a><br>\nfor what is the annotation errors</p>",
          "rawMarkdown": "generate the mask in png, then just use GIMP to draw or erase the mask.\nsee https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200070\nfor what is the annotation errors",
          "votes": 1
        }
      ]
    },
    {
      "id": 1128164,
      "postDate": "2020-12-27T08:18:32.083Z",
      "content": "<p>Hi,How to get the .csv files.</p>",
      "rawMarkdown": "Hi,How to get the .csv files.",
      "votes": -2
    },
    {
      "id": 1294887,
      "postDate": "2021-05-06T03:05:16.130Z",
      "content": "<p>Hi! Was anyone able to port his run_submit.py script to kaggle notebook that not exceeds the RAM size?</p>",
      "rawMarkdown": "Hi! Was anyone able to port his run_submit.py script to kaggle notebook that not exceeds the RAM size?"
    },
    {
      "id": 1289818,
      "postDate": "2021-05-01T12:00:06.860Z",
      "content": "<p>Is there some paper about \"transformer in segmentation modeling\"?</p>",
      "rawMarkdown": "Is there some paper about \"transformer in segmentation modeling\"?"
    },
    {
      "id": 1214014,
      "postDate": "2021-02-22T14:32:23.370Z",
      "content": "<p>impressive work!</p>",
      "rawMarkdown": "impressive work!"
    },
    {
      "id": 1136473,
      "postDate": "2021-01-03T05:37:49.540Z",
      "content": "<p>Hello,<br>\ni meet an error related to Qt in linux. Do you have any idea? i find the solutions in google but all fail.</p>\n<p>Got keys from plugin meta data (\"xcb\")<br>\nQFactoryLoader::QFactoryLoader() checking directory path \"/root/anaconda3/envs/python377/bin/platforms\" …<br>\nloaded library \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms/libqxcb.so\"<br>\nqt.qpa.xcb: could not connect to display :0.0<br>\nqt.qpa.xcb: could not connect to display :0.0<br>\nqt.qpa.plugin: <strong><em><em>Could not load the Qt platform plugin \"xcb\" in \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms\" even though it was found.\nThis application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.</em></em></strong></p>\n<p>Available platform plugins are: xcb (from /root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms), xcb, eglfs, minimal, minimalegl, offscreen, vnc.</p>",
      "rawMarkdown": "Hello,\ni meet an error related to Qt in linux. Do you have any idea? i find the solutions in google but all fail.\n\nGot keys from plugin meta data (\"xcb\")\nQFactoryLoader::QFactoryLoader() checking directory path \"/root/anaconda3/envs/python377/bin/platforms\" ...\nloaded library \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms/libqxcb.so\"\nqt.qpa.xcb: could not connect to display :0.0\nqt.qpa.xcb: could not connect to display :0.0\nqt.qpa.plugin: ****Could not load the Qt platform plugin \"xcb\" in \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms\" even though it was found.\nThis application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.****\n\nAvailable platform plugins are: xcb (from /root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms), xcb, eglfs, minimal, minimalegl, offscreen, vnc.\n\n\n",
      "replies": [
        {
          "id": 1136479,
          "postDate": "2021-01-03T05:53:32.247Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1136709,
          "postDate": "2021-01-03T11:01:54.763Z",
          "content": "<p><em>This application failed to start because no Qt platform plugin could be initialized</em><br>\n<a href=\"https://stackoverflow.com/questions/60042568/this-application-failed-to-start-because-no-qt-platform-plugin-could-be-initiali\" target=\"_blank\">https://stackoverflow.com/questions/60042568/this-application-failed-to-start-because-no-qt-platform-plugin-could-be-initiali</a></p>\n<p>Solution: install older version of Opencv</p>\n<pre><code>pip3 install opencv-python==4.1.2.30\n</code></pre>",
          "rawMarkdown": "*This application failed to start because no Qt platform plugin could be initialized*\nhttps://stackoverflow.com/questions/60042568/this-application-failed-to-start-because-no-qt-platform-plugin-could-be-initiali\n\nSolution: install older version of Opencv\n```\npip3 install opencv-python==4.1.2.30\n```"
        }
      ]
    },
    {
      "id": 1124301,
      "postDate": "2020-12-23T19:57:06.237Z",
      "content": "<p>Congrats on your competition GM already. I came across your wisdom multiple times wherever I go in the community. Thanks! </p>",
      "rawMarkdown": "Congrats on your competition GM already. I came across your wisdom multiple times wherever I go in the community. Thanks! "
    },
    {
      "id": 1124173,
      "postDate": "2020-12-23T17:57:42.637Z",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> .Thanks for sharing! May i know what exactly \"boundary artifacts\"  is? and why will it exist?</p>",
      "rawMarkdown": "Hi, @hengck23 .Thanks for sharing! May i know what exactly \"boundary artifacts\"  is? and why will it exist?"
    },
    {
      "id": 1123225,
      "postDate": "2020-12-23T03:58:20.723Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> thank you for your sharing, this is my first to join kaggel competition, the code is very helpful for me.  i have a dummpy question about  <code>def to_tile(image, mask=None,\n            scale=tile_scale, size=tile_size,\n            step=tile_average_step, min_score=tile_min_score)</code> , in the function the step=192, how do you get the number?</p>",
      "rawMarkdown": "Hi @hengck23 thank you for your sharing, this is my first to join kaggel competition, the code is very helpful for me.  i have a dummpy question about  `def to_tile(image, mask=None,\n            scale=tile_scale, size=tile_size,\n            step=tile_average_step, min_score=tile_min_score)` , in the function the step=192, how do you get the number?"
    },
    {
      "id": 1117651,
      "postDate": "2020-12-18T09:51:35.933Z",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> I checked your <code>2020-12-11/result/en-resnet34-256-aug-corrected/fold6/log.train.txt</code> text file. I saw from the <code>log.submit.txt</code>, that you chose the <code>fold6/checkpoint/00008500_model.pth</code> although the  0.914 validation dice was not the best, because 00009000_mode had 0.918. So my guess is, that it was not based on the validation dice accuracy.<br>\nHow you evaluated your model? <br>\nI know that it worked for you, because you have a good score, but I don't know how I can choose the best model.</p>",
      "rawMarkdown": "@hengck23 I checked your `2020-12-11/result/en-resnet34-256-aug-corrected/fold6/log.train.txt` text file. I saw from the `log.submit.txt`, that you chose the `fold6/checkpoint/00008500_model.pth` although the  0.914 validation dice was not the best, because 00009000_mode had 0.918. So my guess is, that it was not based on the validation dice accuracy.\nHow you evaluated your model? \nI know that it worked for you, because you have a good score, but I don't know how I can choose the best model."
    },
    {
      "id": 1110893,
      "postDate": "2020-12-13T07:24:07.667Z",
      "content": "<p>hello, due to your idea is developed now, so your submit.csv is generated offline. when your code is fixed, could you publish the submission notebook? i'm looking forward to your reply, thanks!</p>",
      "rawMarkdown": "hello, due to your idea is developed now, so your submit.csv is generated offline. when your code is fixed, could you publish the submission notebook? i'm looking forward to your reply, thanks!"
    },
    {
      "id": 1110838,
      "postDate": "2020-12-13T05:57:37.737Z",
      "content": "<p>hello, when i run the run_submit.py to evaluate the test set, there is a problem in test sample: c68fe75ea. I found that when execute the function \"to_tile\" and reture tile, the tile['tile_image'] is empty, it's obviously caused by the \"coord\" is empty in hubmap_v2.py. <br>\nand i lower the value of \"tile_min_score\" to 0, but the tile['tile_image'] is also empty, i can't understand. i'm looking forward to your reply, thanks!</p>\n<p>ok, that's caused by the c68fe75ea.json is empty , i download it from kaggle again</p>",
      "rawMarkdown": "hello, when i run the run_submit.py to evaluate the test set, there is a problem in test sample: c68fe75ea. I found that when execute the function \"to_tile\" and reture tile, the tile['tile_image'] is empty, it's obviously caused by the \"coord\" is empty in hubmap_v2.py. \nand i lower the value of \"tile_min_score\" to 0, but the tile['tile_image'] is also empty, i can't understand. i'm looking forward to your reply, thanks!\n\nok, that's caused by the c68fe75ea.json is empty , i download it from kaggle again"
    },
    {
      "id": 1110218,
      "postDate": "2020-12-12T14:26:24.720Z",
      "content": "<p>hello, i run the code of version12.11, and the num_iteration is 8000000, it''s so huge!!!!  it's nessary to train on such huge num_iteration? the total saved checkpoints are very large since the iter_save is the list with length 16000!</p>",
      "rawMarkdown": "hello, i run the code of version12.11, and the num_iteration is 8000000, it''s so huge!!!!  it's nessary to train on such huge num_iteration? the total saved checkpoints are very large since the iter_save is the list with length 16000!",
      "replies": [
        {
          "id": 1110308,
          "postDate": "2020-12-12T15:56:41.523Z",
          "content": "<p><a href=\"https://www.kaggle.com/sistmrl\" target=\"_blank\">@sistmrl</a> </p>\n<p>80000000 is not the actual iteration needed.</p>\n<p>you can just stop when the loss is low.<br>\nyou can refer to the earlier versions (see result folder) to see how many iterations are needed.</p>",
          "rawMarkdown": "@sistmrl \n\n80000000 is not the actual iteration needed.\n\nyou can just stop when the loss is low.\nyou can refer to the earlier versions (see result folder) to see how many iterations are needed."
        }
      ]
    },
    {
      "id": 1109835,
      "postDate": "2020-12-12T05:55:46.570Z",
      "content": "<p>hello, in code of 12.11 version,  I have a problem of run_data.py, I didn't see any mask on these path, how to get these mask, thanks!<br>\n        try:<br>\n            mask_file = data_dir + '/train/{}.corrected_shift_mask.png'.format(id)<br>\n            mask = read_mask(mask_file)<br>\n        except:<br>\n            mask_file = data_dir + '/train/{}.corrected_mask.png'.format(id)<br>\n            mask = read_mask(mask_file)</p>",
      "rawMarkdown": "hello, in code of 12.11 version,  I have a problem of run_data.py, I didn't see any mask on these path, how to get these mask, thanks!\n        try:\n            mask_file = data_dir + '/train/{}.corrected_shift_mask.png'.format(id)\n            mask = read_mask(mask_file)\n        except:\n            mask_file = data_dir + '/train/{}.corrected_mask.png'.format(id)\n            mask = read_mask(mask_file)",
      "replies": [
        {
          "id": 1109867,
          "postDate": "2020-12-12T07:03:19.410Z",
          "content": "<p>i correct the shift and remove some false annotation.<br>\nthese are done by hand.</p>\n<p>you can generate the mask yourself or try to use the original mask.</p>",
          "rawMarkdown": "i correct the shift and remove some false annotation.\nthese are done by hand.\n\nyou can generate the mask yourself or try to use the original mask."
        }
      ]
    },
    {
      "id": 1109506,
      "postDate": "2020-12-11T19:12:06.147Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a>, really thanks for sharing. What are corrected_shift_mask files, seems like you generate these file offline, it gives an not found error when run run_data.py in your last version.</p>",
      "rawMarkdown": "Hi @hengck23, really thanks for sharing. What are corrected_shift_mask files, seems like you generate these file offline, it gives an not found error when run run_data.py in your last version.",
      "replies": [
        {
          "id": 1109686,
          "postDate": "2020-12-12T00:49:06.100Z",
          "content": "<p>i correct the shift and remove some false annotation.<br>\nthese are done by hand.</p>\n<p>you can generate the mask yourself or try to use the original mask.</p>",
          "rawMarkdown": "i correct the shift and remove some false annotation.\nthese are done by hand.\n\nyou can generate the mask yourself or try to use the original mask.",
          "votes": 1
        },
        {
          "id": 1109802,
          "postDate": "2020-12-12T04:42:48.627Z",
          "content": "<p>Ok, thanks again</p>",
          "rawMarkdown": "Ok, thanks again"
        },
        {
          "id": 1126842,
          "postDate": "2020-12-26T02:55:03.160Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 1108360,
      "postDate": "2020-12-10T15:17:32.057Z",
      "content": "<p>hello, how to run  your code of the version(12.04)</p>",
      "rawMarkdown": "hello, how to run  your code of the version(12.04)",
      "replies": [
        {
          "id": 1108722,
          "postDate": "2020-12-11T00:11:35.033Z",
          "content": "<p>follow the readme ppt of the first version (2020-dec-02)</p>",
          "rawMarkdown": "follow the readme ppt of the first version (2020-dec-02)",
          "votes": 1
        }
      ]
    },
    {
      "id": 1107063,
      "postDate": "2020-12-09T11:04:23.470Z",
      "content": "<p><a href=\"http://www.orbit.bio/deep-learning-object-segmentation/\" target=\"_blank\">http://www.orbit.bio/deep-learning-object-segmentation/</a><br>\nquote \"If you just want to reuse a pre-trained model (e.g. our glomeruli detection model), copy&amp;paste the segmentation script into the script editor (under tools), exchange the model name, edit the image IDs you want to segment (or use a search query) and click on run.\"</p>",
      "rawMarkdown": "http://www.orbit.bio/deep-learning-object-segmentation/\nquote \"If you just want to reuse a pre-trained model (e.g. our glomeruli detection model), copy&paste the segmentation script into the script editor (under tools), exchange the model name, edit the image IDs you want to segment (or use a search query) and click on run.\""
    },
    {
      "id": 1105296,
      "postDate": "2020-12-07T18:27:44.937Z",
      "content": "<p>Thank you for this useful post. </p>\n<p>I downloaded your code from here:  <a href=\"https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing</a></p>\n<p>Do you create the submission.csv online, or offline? I your code offline worked perfectly, but when i tried to run in a kaggle notebook I run out of RAM. I run only the <code>run_submit.py</code> in <code>server = 'kaggle'</code> mode.</p>",
      "rawMarkdown": "Thank you for this useful post. \n\nI downloaded your code from here:  https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\n\nDo you create the submission.csv online, or offline? I your code offline worked perfectly, but when i tried to run in a kaggle notebook I run out of RAM. I run only the `run_submit.py` in `server = 'kaggle'` mode.",
      "replies": [
        {
          "id": 1105312,
          "postDate": "2020-12-07T18:55:17.723Z",
          "content": "<p>i currently generate submission.csv on my local machine. submission code is under development. i will focus on the submission code once my method is fixed.</p>\n<p>meanwhile, you can submit like</p>\n<pre><code># https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/198317\nimport pandas as pd\n\n\n\nlocal_file = '../input/hubmap-dummy/submission-fold2-00006500_model.csv'\n\ndf_submit = pd.read_csv('../input/hubmap-kidney-segmentation/sample_submission.csv', index_col='id')\ndf_local  = pd.read_csv(local_file, index_col='id')\n\ndf_submit.loc[df_local.index.values] = df_local.values  \ndf_submit.to_csv('submission.csv')\nprint(df_submit)\nprint('sucessful!')\n</code></pre>",
          "rawMarkdown": "i currently generate submission.csv on my local machine. submission code is under development. i will focus on the submission code once my method is fixed.\n\nmeanwhile, you can submit like\n\n```\n# https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/198317\nimport pandas as pd\n\n\n \nlocal_file = '../input/hubmap-dummy/submission-fold2-00006500_model.csv'\n\ndf_submit = pd.read_csv('../input/hubmap-kidney-segmentation/sample_submission.csv', index_col='id')\ndf_local  = pd.read_csv(local_file, index_col='id')\n\ndf_submit.loc[df_local.index.values] = df_local.values  \ndf_submit.to_csv('submission.csv')\nprint(df_submit)\nprint('sucessful!')\n\n```",
          "votes": 4
        },
        {
          "id": 1127243,
          "postDate": "2020-12-26T11:13:10.870Z",
          "content": "<p>How do you predict in the private dataset then if you are predicting in local?</p>",
          "rawMarkdown": "How do you predict in the private dataset then if you are predicting in local?\n"
        }
      ]
    },
    {
      "id": 1102705,
      "postDate": "2020-12-05T08:01:38.713Z",
      "content": "<p>I wonder why the different tile_size used between train and inference phase?</p>",
      "rawMarkdown": "I wonder why the different tile_size used between train and inference phase?",
      "replies": [
        {
          "id": 1107844,
          "postDate": "2020-12-10T02:14:10.333Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1110294,
          "postDate": "2020-12-12T15:48:39.977Z",
          "content": "<p><a href=\"https://www.kaggle.com/sdeagggg\" target=\"_blank\">@sdeagggg</a>  His readme ppt says that </p>\n<blockquote>\n  <p>inference crop/step can be different from train. crop should be bigger in inference to avoid boundary &gt; artifacts</p>\n</blockquote>",
          "rawMarkdown": "@sdeagggg  His readme ppt says that \n> inference crop/step can be different from train. crop should be bigger in inference to avoid boundary > artifacts"
        }
      ]
    },
    {
      "id": 1102660,
      "postDate": "2020-12-05T06:47:02.557Z",
      "content": "<p>Hi, heng, nice to meet you.</p>\n<p>I lile your codes :) </p>\n<p>I have a question.</p>\n<p>What dataset do I need to use in the data directory?</p>",
      "rawMarkdown": "Hi, heng, nice to meet you.\n\nI lile your codes :) \n\nI have a question.\n\nWhat dataset do I need to use in the data directory?",
      "replies": [
        {
          "id": 1102667,
          "postDate": "2020-12-05T06:56:38.770Z",
          "content": "<p>refer to </p>\n<pre><code>2020-12-04/code/dummy_01/unet_resnet34_1\n1. run_prepare_tile.py\n\n#make tile train image\ndef run_make_train_tile():\n....\n\n2. dataset.py\n#check tile image and datset class \ndef run_check_dataset():\n\n    image_id = make_image_id ('train-0')\n    dataset = HuDataset(image_id, '0.25_480_240_train') #'0.25_320_192_train'\n    print(dataset)\n</code></pre>",
          "rawMarkdown": "refer to \n\n```\n2020-12-04/code/dummy_01/unet_resnet34_1\n1. run_prepare_tile.py\n\n#make tile train image\ndef run_make_train_tile():\n....\n\n2. dataset.py\n#check tile image and datset class \ndef run_check_dataset():\n\n    image_id = make_image_id ('train-0')\n    dataset = HuDataset(image_id, '0.25_480_240_train') #'0.25_320_192_train'\n    print(dataset)\n\n\n```",
          "votes": 3
        },
        {
          "id": 1103027,
          "postDate": "2020-12-05T15:21:24.600Z",
          "content": "<p>Thanks for the kind response :)</p>",
          "rawMarkdown": "Thanks for the kind response :)"
        },
        {
          "id": 1103996,
          "postDate": "2020-12-06T14:04:55.697Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> how do you set tile_average_step and tile_size in your data generate code? as far as i know,  the tile_average_step more bigger, the overlap more little. How you generate tiles with final size 256 with scale=0.25, because your code use tile_size=480, you just generate 480 image, and then resize it to 256?</p>",
          "rawMarkdown": "Hi @hengck23 how do you set tile_average_step and tile_size in your data generate code? as far as i know,  the tile_average_step more bigger, the overlap more little. How you generate tiles with final size 256 with scale=0.25, because your code use tile_size=480, you just generate 480 image, and then resize it to 256?\n"
        },
        {
          "id": 1104059,
          "postDate": "2020-12-06T15:09:19.417Z",
          "content": "<p>I got it after read detail, thanks for sharing again <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a></p>",
          "rawMarkdown": "I got it after read detail, thanks for sharing again @hengck23"
        }
      ]
    },
    {
      "id": 1102091,
      "postDate": "2020-12-04T15:21:27.310Z",
      "content": "<p>Have you experienced stochasticity when training? I tried setting seeds and running the same fold with the same code. The different is up to 0.007. Is it due to the nature of the dataset? Thanks for the help.</p>",
      "rawMarkdown": "Have you experienced stochasticity when training? I tried setting seeds and running the same fold with the same code. The different is up to 0.007. Is it due to the nature of the dataset? Thanks for the help.",
      "replies": [
        {
          "id": 1102697,
          "postDate": "2020-12-05T07:51:33.130Z",
          "content": "<p>i haven't check this yet. by you can change to sdg optimizer and try again</p>",
          "rawMarkdown": "i haven't check this yet. by you can change to sdg optimizer and try again"
        }
      ]
    },
    {
      "id": 1100864,
      "postDate": "2020-12-03T12:16:48.533Z",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a>! It's good to see you in this competition. </p>",
      "rawMarkdown": "Hey @hengck23! It's good to see you in this competition. "
    },
    {
      "id": 1100286,
      "postDate": "2020-12-03T01:15:43.693Z",
      "content": "<p>things to do:</p>\n<ul>\n<li><p>build size distribution for a single slide. see if we can use size to filter off fp, etc<br>\n(the size of glomeruli is more or less fix in a single slide)</p></li>\n<li><p>pseudo label the train instance (glomeruli) for difficult cases. this is to resample for training.</p></li>\n</ul>",
      "rawMarkdown": "things to do:\n- build size distribution for a single slide. see if we can use size to filter off fp, etc\n(the size of glomeruli is more or less fix in a single slide)\n\n- pseudo label the train instance (glomeruli) for difficult cases. this is to resample for training."
    },
    {
      "id": 1229983,
      "postDate": "2021-03-07T18:19:21.237Z",
      "content": "<p>thanks for your job!</p>",
      "rawMarkdown": "thanks for your job!"
    }
  ],
  "comments": [
    {
      "id": 1100565,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-03T06:46:59.413000",
      "content": "<p>my prediction for the final ranking:</p>\n<ul>\n<li>public LB : 0.90 ~ 0.93 (shakeup 0.01)</li>\n</ul>\n<p>my prediction is based on:</p>\n<ul>\n<li>train, validation error</li>\n<li>human error (I try to reannotate some train images)</li>\n<li>visual inspection of prediction results on train/test data </li>\n<li>study the distribution of probability scores and embedding (e.g. tsne)</li>\n</ul>\n<p>key to winning<br>\n1) definitely data!</p>\n<ul>\n<li>how to generate data? augmentation/GAN …</li>\n<li>offline pseudo label (public test) … definitely helps!</li>\n<li>online pseudo label (hidden private test) </li>\n<li>external data</li>\n</ul>\n<p>2) post processing</p>\n<ul>\n<li>boundary artifacts …. filtering of FP (some are due to wrong label) … train another model for post processing</li>\n<li>pre-processing? (not sure if you get the outliers in private test?)<ul>\n<li>ink/marking on stains</li>\n<li>different stain hue</li>\n<li>scale </li></ul></li>\n</ul>\n<p>3) modeling</p>\n<ul>\n<li>I feel that this is less important. standard models work well, e.g. fpn, unet.</li>\n<li>important thing is how to design one that suits your magic, e.g. design one for pseudo label , post-processing fp, high speed, good memory, etc …</li>\n</ul>",
      "votes": 12,
      "replies": [
        {
          "id": 1101334,
          "author_name": "William Green",
          "author_url": "",
          "post_date": "2020-12-03T20:14:23.520000",
          "content": "<p>Can you expound more  on <code>study the distribution of probability scores and embedding (e.g. tsne)</code> ?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1104643,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-07T06:34:03.880000",
      "content": "<p>preview of next code update:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Febe5ebab1707bfdd11f4f0a8df7f421b%2FSelection_146.png?generation=1607322785976029&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F6dcc1c77979b6362e96a6269ba9cbb35%2FSelection_136.png?generation=1607322811733295&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F89e99250b0e0ce282c306d3b5f702438%2FSelection_137.png?generation=1607322839958767&amp;alt=media\" alt=\"\"></p>",
      "votes": 8,
      "replies": [
        {
          "id": 1104702,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-07T07:22:20.177000",
          "content": "<p>this is a competition where CV/LB correlation is not enough</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F406dae1e113fc61cf0d9020f5f4ae566%2FSelection_149.png?generation=1607325729310460&amp;alt=media\" alt=\"\"></p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1104727,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-07T07:33:20.117000",
          "content": "",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1108748,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-11T01:10:20.860000",
          "content": "<p>code preview: finally, getting rid of the boundary artifacts, see hubmap_v2.py</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F4c45306d59d92c213a15d84599162aa3%2FSelection_181.png?generation=1607648975343396&amp;alt=media\" alt=\"\"></p>",
          "votes": 7,
          "replies": []
        },
        {
          "id": 1108791,
          "author_name": "cswwp",
          "author_url": "",
          "post_date": "2020-12-11T02:11:25.237000",
          "content": "<p>Nice, thanks for sharing</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1103565,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-06T03:18:52.590000",
      "content": "<p>external dataset:</p>\n<p><a href=\"https://data.mendeley.com/datasets/k7nvtgn2x6/3\" target=\"_blank\">https://data.mendeley.com/datasets/k7nvtgn2x6/3</a><br>\n\"Data for glomeruli characterization in histopathological images\"<br>\nDATASET_A: Raw data with 31 whole slide images (WSI) in SVS format. The size of the WSI range between 21651x10498 pixels and 49799 x 32359 pixles acquired at 20x. </p>\n<p>to read SVS format: <a href=\"http://www.slideio.com/\" target=\"_blank\">http://www.slideio.com/</a></p>",
      "votes": 8,
      "replies": []
    },
    {
      "id": 1109794,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-12T04:32:11.793000",
      "content": "<h1>🐜 🐜  🐜</h1>\n<p>2020-dec-11 : Bug !!!!   The test image cortex structure region is not accurate. sample windows (tiles) from it will miss some glomerulus<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F25dce65ac9366dfa4f8054007e000278%2FSelection_197.png?generation=1607747567069688&amp;alt=media\" alt=\"\"></p>",
      "votes": 5,
      "replies": [
        {
          "id": 1149345,
          "author_name": "ck090",
          "author_url": "",
          "post_date": "2021-01-11T19:23:32.737000",
          "content": "<p>Yes, it does so for many test images. Especially for afa5e… it looks like the polygon for the cortex is wrongly marked by the competition hosts and if we apply that mask to fill in and clip out non-cortex glomeruli we end up with not all of them detected. </p>\n<p>Here I have a sample:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1586379%2Ffec7d95dc30de76c23913fc6ef7fa1b1%2Fdownload.png?generation=1610392848512076&amp;alt=media\" alt=\"\"><br>\nYou can see that the masks are clipped in the far right.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1586379%2Fa3199f4e38d2fa51a136f018d447e41e%2Fdownload.png?generation=1610392929660788&amp;alt=media\" alt=\"\"></p>\n<p>Is this the same behavior you are mentioning? If so, have you proceeded to use this polygon to clip unwanted glomeruli or not? <br>\nThanks,<br>\nCK</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1099802,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-02T15:49:47.637000",
      "content": "<p>my plan is to modify the below to add transformer to unet.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fd356e6c94eb7c5cfb5732b9083ffaf22%2FSelection_034.png?generation=1606924184309685&amp;alt=media\" alt=\"\"></p>",
      "votes": 5,
      "replies": [
        {
          "id": 1100491,
          "author_name": "tik_boa",
          "author_url": "",
          "post_date": "2020-12-03T05:40:10.923000",
          "content": "<p>Can you provide a link to this article? Thank you</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1108020,
          "author_name": "Aman Arora",
          "author_url": "",
          "post_date": "2020-12-10T07:28:53.767000",
          "content": "<p><a href=\"https://arxiv.org/abs/2011.09763\" target=\"_blank\">https://arxiv.org/abs/2011.09763</a> <a href=\"https://www.kaggle.com/tikboa\" target=\"_blank\">@tikboa</a> </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1103498,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-06T01:11:14.300000",
      "content": "<p>a few probing tricks used in the previous challenge</p>\n<p>(1) submit only one image at a time : </p>\n<ul>\n<li>enable you to analyze which image is the best, which is the worst. <br>\n(e.g. you discover the problem of low image score is blurry image, different scale, etc …)</li>\n</ul>\n<p>(2) submit prediction for only a certain region of the image (e.g likely to be fp region)</p>\n<ul>\n<li>same as above. </li>\n</ul>\n<p>(3) submit a single prediction area (e.g. a single rectangle covering the whole image or only part of the image. here, we actually the polygon of the cortex, etc)</p>\n<ul>\n<li>enable you to know the density/number object instances if you know the distribution or average size of the object)</li>\n<li>enable you to know the total area of object instances</li>\n</ul>\n<p>(4) submit only object instance of only one size</p>\n<ul>\n<li>let you know the relationship between dice score and size.  it lets you answer questions like:<ul>\n<li>should I just ignore small ground truth</li>\n<li>should I focus on detecting more instances or just improving dice score of the current predicted instances</li></ul></li>\n</ul>\n<p>(5) submit only object instance of very high confidence score (or within certain score band)</p>\n<ul>\n<li>combined with (3) which tells you the total area, you roughly know the number of correct detection and average dice score. some competition requires a score re-calibration</li>\n</ul>\n<p>warning!!! </p>\n<ul>\n<li>you can only probe the public set but not the private one</li>\n<li>the more you probe and know about the public set, you are over-fitting the public set<br>\n(knowing more is not always good … know your assumption!)</li>\n</ul>",
      "votes": 6,
      "replies": [
        {
          "id": 1103520,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-06T01:57:54.037000",
          "content": "<p>as an example of problem finding<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F58d4ae6b317bdef81f5c7f0ebff8ce2f%2FSelection_103.png?generation=1607219871325761&amp;alt=media\" alt=\"\"></p>\n<p>… maybe I need a simple color transform or stain normalisation network?</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1104178,
          "author_name": "cswwp",
          "author_url": "",
          "post_date": "2020-12-06T17:18:55.853000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a>, i have one question, firstly we have no public test set segment labels, but how you calculated the dice score as your upper excel shows?👀</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1104180,
          "author_name": "cswwp",
          "author_url": "",
          "post_date": "2020-12-06T17:24:33.507000",
          "content": "<p>I get it, you just use plb to calculate the dice score, smart idea</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1108801,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-11T02:37:19.677000",
      "content": "<p>Boundary loss<br>\n<a href=\"http://proceedings.mlr.press/v102/kervadec19a/kervadec19a.pdf\" target=\"_blank\">http://proceedings.mlr.press/v102/kervadec19a/kervadec19a.pdf</a><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb94aa4a2408f2fe36dc449c3ee21aae7%2FSelection_186.png?generation=1607654237573602&amp;alt=media\" alt=\"\"></p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 1107840,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-10T02:04:29.363000",
      "content": "<p>counting network or label propagation<br>\n<a href=\"https://www.robots.ox.ac.uk/~vgg/publications/2018/Lu18/lu18.pdf\" target=\"_blank\">https://www.robots.ox.ac.uk/~vgg/publications/2018/Lu18/lu18.pdf</a><br>\nClass-Agnostic Counting<br>\nErika Lu, Weidi Xie, and Andrew Zisserman</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c047451090e1eaec5618ee46fcf9056%2FSelection_165.png?generation=1607565826614306&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb3b6c05910f373903a428b57f9ba496a%2FSelection_164.png?generation=1607565865568893&amp;alt=media\" alt=\"\"></p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 1100495,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-03T05:48:05.003000",
      "content": "<p>all data analysis prize (judge prize) related stuffs here:</p>\n<ol>\n<li><p>you may want to learn about huBMAP here<br>\n<a href=\"https://expand.iu.edu/browse/sice/cns/courses/hubmap-visible-human-mooc\" target=\"_blank\">https://expand.iu.edu/browse/sice/cns/courses/hubmap-visible-human-mooc</a><br>\n<a href=\"https://www.youtube.com/channel/UCbSvPJ9dXASL14KoDeutMFg/videos\" target=\"_blank\">https://www.youtube.com/channel/UCbSvPJ9dXASL14KoDeutMFg/videos</a></p></li>\n<li><p>some interesting analysis of your model performance: e.g. worst-case scenario<br>\n\"Identifying Model Weakness with Adversarial Examiner\"-AAAI 2020</p></li>\n</ol>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 1099795,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-02T15:45:38.507000",
      "content": "<p>code version: 2020-dec-02</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe08539770a9921ea8ed4714e007c3321%2FSelection_029.png?generation=1606951417354956&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9d7a596419d8b050a5abc2eba6e08c18%2FSelection_030.png?generation=1606951459356411&amp;alt=media\" alt=\"\"></p>",
      "votes": 3,
      "replies": [
        {
          "id": 1100230,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-02T23:36:36.753000",
          "content": "<p>the boundary prediction for a tile is a headache.</p>\n<ul>\n<li>I can do post processing to merge predictions 2 neighboring tiles</li>\n<li>maybe I should remove boundary ground truth if they are too small … after all, my training tiles are overlapping</li>\n</ul>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1105419,
          "author_name": "Sophie",
          "author_url": "",
          "post_date": "2020-12-07T21:15:16.530000",
          "content": "<p>This is a very thorough observation. Regarding of the ground truth, I don't know how this completion get the ground truth. Based on my experiences on medical images for labeling ground truth with MD, nurses, techs etc, it is subjective in some extent and easy for making human errors. Similarly for this competition, it is not surprising to see artifacts in these images.  </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1105660,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-08T04:14:23.080000",
          "content": "<p><a href=\"https://www.kaggle.com/qingtaozhou\" target=\"_blank\">@qingtaozhou</a> <br>\n\"Based on my experiences on medical images for labeling ground truth with MD, nurses, techs \"</p>\n<p>Can you tell me more about your experiences on labeling tissue images? </p>\n<p>from my inspection of images for this kidney competition, I think:</p>\n<ol>\n<li>there is some software that does automatic or semi-automatic labeling first</li>\n<li>then there are some expert to verify or correct the initial annotations</li>\n</ol>\n<p>(this is quite obvious because some errors are machine-made and cannot be made by a human)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1105947,
          "author_name": "Sophie",
          "author_url": "",
          "post_date": "2020-12-08T11:08:04.570000",
          "content": "<p>You are right. These are two steps for labeling the ground truth.  There are many deep learning products on medical image field either for clinical purpose(FDA approved) or for research purpose.  However, technology has limits. If you use two different software (FDA approval) for the same image segmentation, there are still differences for labeling.   For the second step that you mention, experts verify or correct the initial annotations. My understanding is that experts make subject decision or may miss something. In the clinical settings, people usually use inter-observer to evaluate the variation for the same thing. For this competition, I don't know how they labeled the ground truth and didn't have experiences specifically for glomeruli. My comments could be biased. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1108798,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-11T02:31:48.170000",
      "content": "<p>dice is non-symmetric metric … don't be afraid to detect more and make some mistake<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F149b10fd4c34d39acda60b89707cab45%2FSelection_185.png?generation=1607653899026735&amp;alt=media\" alt=\"\"></p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 1099660,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-02T14:12:39.877000",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F83e682c2a26ca0a9ca6f998d5987bd9f%2FSelection_029.png?generation=1606918300495338&amp;alt=media\" alt=\"\"></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F7c148869c78d728d57468afd108005c2%2FSelection_030.png?generation=1606918314707954&amp;alt=media\" alt=\"\"></p>\n<p>see attached \"data.py\"</p>",
      "votes": 4,
      "replies": [
        {
          "id": 1108032,
          "author_name": "Aman Arora",
          "author_url": "",
          "post_date": "2020-12-10T07:47:25.450000",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> Could you please explain how you decide to reject/keep tiles? Specifically could I please ask what does this bit of code do? </p>\n<pre><code>    height, width, _ = image_small.shape\n    vv = cv2.resize(image_small, dsize=None, fx=1 / 32, fy=1 / 32, \n        interpolation=cv2.INTER_LINEAR)\n    vv = cv2.cvtColor(vv, cv2.COLOR_RGB2HSV)\n    # image_show('v[0]', v[:,:,0])\n    # image_show('v[1]', v[:,:,1])\n    # image_show('v[2]', v[:,:,2])\n    # cv2.waitKey(0)\n    vv = (vv[:, :, 1] &gt; 32).astype(np.float32)\n    vv = cv2.resize(vv, dsize=(width, height), interpolation=cv2.INTER_LINEAR)\n</code></pre>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1108038,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-10T07:58:10.600000",
          "content": "<p>in the image, a window (or called tile) can contain some tissue pixels or no tissue pixels at all.</p>\n<p>those that does not contain tissue pixels at all could be rejected immediately and not processed (because we know that the number of glomeruli detected must be null)</p>\n<p>we convert rgb image into hsv. we assume tissue pixel is one that saturation value &gt; 32.</p>\n<hr>\n<p>note, </p>\n<ul>\n<li>a better way is to use the mask of Cortex region provided in the json file provided.</li>\n<li>later I find that threshold of saturation value &gt; 32. works for all images in the train. but it fails for one image in the test (see discussion at the top of the thread). you may want  reset the threshold, etc.</li>\n<li>in the worst case, you can reject none and process all windows</li>\n</ul>",
          "votes": 6,
          "replies": []
        },
        {
          "id": 1108708,
          "author_name": "Aman Arora",
          "author_url": "",
          "post_date": "2020-12-10T23:39:56.960000",
          "content": "<p>Thank you - that makes sense. :)</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1112076,
      "author_name": "mashepburn8",
      "author_url": "",
      "post_date": "2020-12-14T09:06:47.163000",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3964990%2Fc47049044214a2a580b8e156a6400526%2FSnipaste_2020-12-14_17-02-56.jpg?generation=1607936611952354&amp;alt=media\" alt=\"\"><br>\nDear washen,im not really understand about why ‘for training use hue-saturation’ is related to 'change sampling method'.Why isn't it related to augmentation?<br>\nThanks!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1115276,
          "author_name": "Kishan Joshi",
          "author_url": "",
          "post_date": "2020-12-16T06:46:12.967000",
          "content": "<p>I think, training images which are above some specific saturation threshold will be sampled for training, rest will be rejected. This is the idea behind sampling for training set.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1117396,
          "author_name": "mashepburn8",
          "author_url": "",
          "post_date": "2020-12-18T02:27:54.463000",
          "content": "<p>thanks for your reply,i get it!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1110678,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-13T00:37:18.907000",
      "content": "<p><a href=\"https://arxiv.org/abs/2012.05780\" target=\"_blank\">https://arxiv.org/abs/2012.05780</a><br>\nOneNet: Towards End-to-End One-Stage Object Detection<br>\nobject detection without NMS</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1100673,
      "author_name": "fam_taro",
      "author_url": "",
      "post_date": "2020-12-03T08:39:13.490000",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> <br>\nThanks for posting! This is a great help.</p>\n<p>By the way, your code only predicts public test data, <br>\nbut have you tried predicting it for private test data as well?</p>\n<pre><code>def make_image_id (mode):\n    ...\n    test_image_id = {\n        0 : 'b9a3865fc',\n        1 : 'b2dc8411c',\n        2 : '26dc41664',\n        3 : 'c68fe75ea',\n        4 : 'afa5e8098',\n    }\n    ...\n    if 'test'==mode:\n        test_id = [ test_image_id[i] for i in [0,1,2,3,4] ]\n\n        return test_id\n</code></pre>",
      "votes": 1,
      "replies": [
        {
          "id": 1100683,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-03T08:45:54.427000",
          "content": "<p>not yet. the code is still under development</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1100686,
          "author_name": "fam_taro",
          "author_url": "",
          "post_date": "2020-12-03T08:47:28.963000",
          "content": "<p>I see. Thank you!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1126875,
          "author_name": "mrlcv",
          "author_url": "",
          "post_date": "2020-12-26T04:13:33.507000",
          "content": "<p>hello, can you accomplish the process of evaluating private test data?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1127061,
          "author_name": "mrlcv",
          "author_url": "",
          "post_date": "2020-12-26T08:09:33.853000",
          "content": "<p>hello, frog brother. today I use your run_submit.py to submit not purely submission.csv generated in offline, but it occurs the problem as below:<br>\nyour notebook tried to allocate more memory than is available.it has restarted<br>\ni can't to fix it, could you give me some suggestions? thanks very much!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1230660,
          "author_name": "Ambition_Kingo",
          "author_url": "",
          "post_date": "2021-03-08T10:38:22.567000",
          "content": "<p>hello, could you please tell me that where to get the private test set image? a lot thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1129375,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-28T09:47:51.557000",
      "content": "<p>direct polygon detection<br>\n<a href=\"https://www.youtube.com/watch?v=sysySMr3YN4\" target=\"_blank\">https://www.youtube.com/watch?v=sysySMr3YN4</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c545ac92b64a6aba27d749ecd407753%2FSelection_224.png?generation=1609148868739418&amp;alt=media\" alt=\"\"></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1102675,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-05T07:10:17.627000",
      "content": "<p>the json file actually provide the ground truth ploygon coordinates.<br>\nhence you can use the following works:</p>\n<ul>\n<li>\"PolyTransform: Deep Polygon Transformer for Instance Segmentation\"</li>\n</ul>\n<p><a href=\"https://openaccess.thecvf.com/content_CVPR_2020/papers/Liang_PolyTransform_Deep_Polygon_Transformer_for_Instance_Segmentation_CVPR_2020_paper.pdf\" target=\"_blank\">https://openaccess.thecvf.com/content_CVPR_2020/papers/Liang_PolyTransform_Deep_Polygon_Transformer_for_Instance_Segmentation_CVPR_2020_paper.pdf</a></p>\n<ul>\n<li><p>Annotating Object Instances with a Polygon-RNN<br>\nEfficient Annotation of Segmentation Datasets with Polygon-RNN++<br>\n<a href=\"http://www.cs.toronto.edu/polyrnn/poly_cvpr17/\" target=\"_blank\">http://www.cs.toronto.edu/polyrnn/poly_cvpr17/</a><br>\n<a href=\"http://www.cs.toronto.edu/polyrnn/\" target=\"_blank\">http://www.cs.toronto.edu/polyrnn/</a></p></li>\n<li><p>Fast Interactive Object Annotation with Curve-GCN<br>\n<a href=\"https://arxiv.org/pdf/1903.06874.pdf\" target=\"_blank\">https://arxiv.org/pdf/1903.06874.pdf</a></p></li>\n<li><p>Cell Detection with Star-convex Polygons<br>\n<a href=\"https://arxiv.org/pdf/1806.03535.pdf\" target=\"_blank\">https://arxiv.org/pdf/1806.03535.pdf</a></p></li>\n</ul>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1101335,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2020-12-03T20:15:22.583000",
      "content": "<p>some interesting papers. the keyword for googling is \"WSI whole slide image\":</p>\n<p><a href=\"https://paperswithcode.com/task/whole-slide-images\" target=\"_blank\">https://paperswithcode.com/task/whole-slide-images</a>.</p>\n<ul>\n<li><a href=\"https://arxiv.org/pdf/2007.13952v1.pdf\" target=\"_blank\">https://arxiv.org/pdf/2007.13952v1.pdf</a><br>\nEasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology<br>\n(maybe some part of it can be used in Judge Prize)</li>\n</ul>",
      "votes": 2,
      "replies": [
        {
          "id": 1101787,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-04T09:01:39.997000",
          "content": "<p>this paper only changes one line of code</p>\n<p>\"Specifically, rather than using a single neuron, we propose to additionally consider<br>\neach neuron’s neighborhood for calculating the outputs of the linear transformation\"</p>\n<p>Batch Normalization with Enhanced Linear Transformation<br>\n<a href=\"https://arxiv.org/pdf/2011.14150.pdf\" target=\"_blank\">https://arxiv.org/pdf/2011.14150.pdf</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F91ff4f86a1ac137aa7fd8ab43636ecd1%2FSelection_028.png?generation=1607072497284444&amp;alt=media\" alt=\"\"></p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1100701,
      "author_name": "seb",
      "author_url": "",
      "post_date": "2020-12-03T08:56:51.260000",
      "content": "<p>You mentioned:</p>\n<blockquote>\n  <p>human error (I try to reannotate some train images)</p>\n</blockquote>\n<p>Is re-annotating allowed? I see some annotation errors in the training set as well.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1100702,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-03T08:58:58.950000",
          "content": "<p>yes, it is allowed for train</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1101917,
          "author_name": "yuvaramsingh",
          "author_url": "",
          "post_date": "2020-12-04T11:59:22.100000",
          "content": "<p>hi, what tool are you using for annotation. i am not able to find a tool for annotating tiff files </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1101957,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2020-12-04T12:55:55.433000",
          "content": "<p>generate the mask in png, then just use GIMP to draw or erase the mask.<br>\nsee <a href=\"https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200070\" target=\"_blank\">https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200070</a><br>\nfor what is the annotation errors</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1128164,
      "author_name": "HAaHAa",
      "author_url": "",
      "post_date": "2020-12-27T08:18:32.083000",
      "content": "<p>Hi,How to get the .csv files.</p>",
      "votes": -2,
      "replies": []
    },
    {
      "id": 1294887,
      "author_name": "Bessenyei Szilárd",
      "author_url": "",
      "post_date": "2021-05-06T03:05:16.130000",
      "content": "<p>Hi! Was anyone able to port his run_submit.py script to kaggle notebook that not exceeds the RAM size?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1289818,
      "author_name": "shigengtian",
      "author_url": "",
      "post_date": "2021-05-01T12:00:06.860000",
      "content": "<p>Is there some paper about \"transformer in segmentation modeling\"?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1214014,
      "author_name": "L Shi",
      "author_url": "",
      "post_date": "2021-02-22T14:32:23.370000",
      "content": "<p>impressive work!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1136473,
      "author_name": "DL Bao",
      "author_url": "",
      "post_date": "2021-01-03T05:37:49.540000",
      "content": "<p>Hello,<br>\ni meet an error related to Qt in linux. Do you have any idea? i find the solutions in google but all fail.</p>\n<p>Got keys from plugin meta data (\"xcb\")<br>\nQFactoryLoader::QFactoryLoader() checking directory path \"/root/anaconda3/envs/python377/bin/platforms\" …<br>\nloaded library \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms/libqxcb.so\"<br>\nqt.qpa.xcb: could not connect to display :0.0<br>\nqt.qpa.xcb: could not connect to display :0.0<br>\nqt.qpa.plugin: <strong><em><em>Could not load the Qt platform plugin \"xcb\" in \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms\" even though it was found.\nThis application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.</em></em></strong></p>\n<p>Available platform plugins are: xcb (from /root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms), xcb, eglfs, minimal, minimalegl, offscreen, vnc.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1136479,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-01-03T05:53:32.247000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1136709,
          "author_name": "Bessenyei Szilárd",
          "author_url": "",
          "post_date": "2021-01-03T11:01:54.763000",
          "content": "<p><em>This application failed to start because no Qt platform plugin could be initialized</em><br>\n<a href=\"https://stackoverflow.com/questions/60042568/this-application-failed-to-start-because-no-qt-platform-plugin-could-be-initiali\" target=\"_blank\">https://stackoverflow.com/questions/60042568/this-application-failed-to-start-because-no-qt-platform-plugin-could-be-initiali</a></p>\n<p>Solution: install older version of Opencv</p>\n<pre><code>pip3 install opencv-python==4.1.2.30\n</code></pre>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1124301,
      "author_name": "Sinan Calisir",
      "author_url": "",
      "post_date": "2020-12-23T19:57:06.237000",
      "content": "<p>Congrats on your competition GM already. I came across your wisdom multiple times wherever I go in the community. Thanks! </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1124173,
      "author_name": "Dapeng",
      "author_url": "",
      "post_date": "2020-12-23T17:57:42.637000",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> .Thanks for sharing! May i know what exactly \"boundary artifacts\"  is? and why will it exist?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1123225,
      "author_name": "君莫笑",
      "author_url": "",
      "post_date": "2020-12-23T03:58:20.723000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> thank you for your sharing, this is my first to join kaggel competition, the code is very helpful for me.  i have a dummpy question about  <code>def to_tile(image, mask=None,\n            scale=tile_scale, size=tile_size,\n            step=tile_average_step, min_score=tile_min_score)</code> , in the function the step=192, how do you get the number?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1117651,
      "author_name": "Bessenyei Szilárd",
      "author_url": "",
      "post_date": "2020-12-18T09:51:35.933000",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> I checked your <code>2020-12-11/result/en-resnet34-256-aug-corrected/fold6/log.train.txt</code> text file. I saw from the <code>log.submit.txt</code>, that you chose the <code>fold6/checkpoint/00008500_model.pth</code> although the  0.914 validation dice was not the best, because 00009000_mode had 0.918. So my guess is, that it was not based on the validation dice accuracy.<br>\nHow you evaluated your model? <br>\nI know that it worked for you, because you have a good score, but I don't know how I can choose the best model.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1110893,
      "author_name": "mrlcv",
      "author_url": "",
      "post_date": "2020-12-13T07:24:07.667000",
      "content": "<p>hello, due to your idea is developed now, so your submit.csv is generated offline. when your code is fixed, could you publish the submission notebook? i'm looking forward to your reply, thanks!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1110838,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-13T05:57:37.737000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1110218,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-12T14:26:24.720000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1110308,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-12T15:56:41.523000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1109835,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-12T05:55:46.570000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1109867,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-12T07:03:19.410000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1109506,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-11T19:12:06.147000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1109686,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-12T00:49:06.100000",
          "content": "",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1109802,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-12T04:42:48.627000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1126842,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-26T02:55:03.160000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1108360,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-10T15:17:32.057000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1108722,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-11T00:11:35.033000",
          "content": "",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1107063,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-09T11:04:23.470000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1105296,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-07T18:27:44.937000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1105312,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-07T18:55:17.723000",
          "content": "",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1127243,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-26T11:13:10.870000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1102705,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-05T08:01:38.713000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1107844,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-10T02:14:10.333000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1110294,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-12T15:48:39.977000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1102660,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-05T06:47:02.557000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1102667,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-05T06:56:38.770000",
          "content": "",
          "votes": 3,
          "replies": []
        },
        {
          "id": 1103027,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-05T15:21:24.600000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1103996,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-06T14:04:55.697000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1104059,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-06T15:09:19.417000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1102091,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-04T15:21:27.310000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 1102697,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-12-05T07:51:33.130000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1100864,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-03T12:16:48.533000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1100286,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-12-03T01:15:43.693000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1229983,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-03-07T18:19:21.237000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1099658": "... to be updated ....\n- more to come as i progress and improve my code\n\n\n---\ncode, trained model, etc are here: https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\n\n1) 2020-dec-02 : \n- image tiling and merging \n- able to train and do inference with a simple unet-resnet34 \n  (a well-trained single fold LB is about 0.830)\n- visualize prediction  \n\n2) 2020-dec-04 : \n- updated augmentation: resnet34-unet LB 0.837 (single fold)\n- refer to the discussion at https://www.kaggle.com/c/hubmap-kidney-segmentation/discussion/200626 for higher LB 0.845\n\n3) 2020-dec-11 : \n- updated augmentation: more hue, saturation, contrast, intensity variation\n- implement enhanced batch normalisation. you can train segmentation from scratch.\nBatch Normalization with Enhanced Linear Transformation\nhttps://arxiv.org/pdf/2011.14150.pdf\n- change sampling method: \n    - for testing use cortex json\n    - for training use hue-saturation\n    - for merging tile, use weighted average to remove boundary artifacts\n- LB 0.85+ : try different fold and threshold(for probability). the results are quite different\n\n4) next update\n- augmentation to address hue-saturation problem/ ink and markings, etc\n- how to ensemble?\n- post processing? predict score to each instance?\n\n....\nlong-termed plan:\n- GAN for data generation\n- weak supervised using public test\n- self supervsied using all kaggle data + external + HUBMAP\n- transformer in segmentation modeling ...\n",
    "1100565": "my prediction for the final ranking:\n- public LB : 0.90 ~ 0.93 (shakeup 0.01)\n\nmy prediction is based on:\n- train, validation error\n- human error (I try to reannotate some train images)\n- visual inspection of prediction results on train/test data \n- study the distribution of probability scores and embedding (e.g. tsne)\n\nkey to winning\n1) definitely data!\n- how to generate data? augmentation/GAN ...\n- offline pseudo label (public test) ... definitely helps!\n- online pseudo label (hidden private test) \n- external data\n\n\n2) post processing\n- boundary artifacts .... filtering of FP (some are due to wrong label) ... train another model for post processing\n- pre-processing? (not sure if you get the outliers in private test?)\n  - ink/marking on stains\n  - different stain hue\n  - scale \n\n\n3) modeling\n- I feel that this is less important. standard models work well, e.g. fpn, unet.\n- important thing is how to design one that suits your magic, e.g. design one for pseudo label , post-processing fp, high speed, good memory, etc ...",
    "1104643": "preview of next code update:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Febe5ebab1707bfdd11f4f0a8df7f421b%2FSelection_146.png?generation=1607322785976029&alt=media)\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F6dcc1c77979b6362e96a6269ba9cbb35%2FSelection_136.png?generation=1607322811733295&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F89e99250b0e0ce282c306d3b5f702438%2FSelection_137.png?generation=1607322839958767&alt=media)",
    "1103565": "external dataset:\n\nhttps://data.mendeley.com/datasets/k7nvtgn2x6/3\n\"Data for glomeruli characterization in histopathological images\"\nDATASET_A: Raw data with 31 whole slide images (WSI) in SVS format. The size of the WSI range between 21651x10498 pixels and 49799 x 32359 pixles acquired at 20x. \n\nto read SVS format: http://www.slideio.com/",
    "1109794": "# 🐜 🐜  🐜 \n2020-dec-11 : Bug !!!!   The test image cortex structure region is not accurate. sample windows (tiles) from it will miss some glomerulus\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F25dce65ac9366dfa4f8054007e000278%2FSelection_197.png?generation=1607747567069688&alt=media)",
    "1099802": "my plan is to modify the below to add transformer to unet.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fd356e6c94eb7c5cfb5732b9083ffaf22%2FSelection_034.png?generation=1606924184309685&alt=media)\n",
    "1103498": "a few probing tricks used in the previous challenge\n\n(1) submit only one image at a time : \n- enable you to analyze which image is the best, which is the worst. \n(e.g. you discover the problem of low image score is blurry image, different scale, etc ...)\n\n(2) submit prediction for only a certain region of the image (e.g likely to be fp region)\n- same as above. \n\n(3) submit a single prediction area (e.g. a single rectangle covering the whole image or only part of the image. here, we actually the polygon of the cortex, etc)\n- enable you to know the density/number object instances if you know the distribution or average size of the object)\n- enable you to know the total area of object instances\n\n(4) submit only object instance of only one size\n- let you know the relationship between dice score and size.  it lets you answer questions like:\n   - should I just ignore small ground truth\n   - should I focus on detecting more instances or just improving dice score of the current predicted instances\n\n(5) submit only object instance of very high confidence score (or within certain score band)\n- combined with (3) which tells you the total area, you roughly know the number of correct detection and average dice score. some competition requires a score re-calibration\n\n\nwarning!!! \n- you can only probe the public set but not the private one\n- the more you probe and know about the public set, you are over-fitting the public set\n(knowing more is not always good ... know your assumption!)",
    "1108801": "Boundary loss\nhttp://proceedings.mlr.press/v102/kervadec19a/kervadec19a.pdf\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb94aa4a2408f2fe36dc449c3ee21aae7%2FSelection_186.png?generation=1607654237573602&alt=media)",
    "1107840": "counting network or label propagation\nhttps://www.robots.ox.ac.uk/~vgg/publications/2018/Lu18/lu18.pdf\nClass-Agnostic Counting\nErika Lu, Weidi Xie, and Andrew Zisserman\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c047451090e1eaec5618ee46fcf9056%2FSelection_165.png?generation=1607565826614306&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fb3b6c05910f373903a428b57f9ba496a%2FSelection_164.png?generation=1607565865568893&alt=media)\n",
    "1100495": "all data analysis prize (judge prize) related stuffs here:\n1. you may want to learn about huBMAP here\nhttps://expand.iu.edu/browse/sice/cns/courses/hubmap-visible-human-mooc\nhttps://www.youtube.com/channel/UCbSvPJ9dXASL14KoDeutMFg/videos\n\n2. some interesting analysis of your model performance: e.g. worst-case scenario\n\"Identifying Model Weakness with Adversarial Examiner\"-AAAI 2020",
    "1099795": "code version: 2020-dec-02\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fe08539770a9921ea8ed4714e007c3321%2FSelection_029.png?generation=1606951417354956&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9d7a596419d8b050a5abc2eba6e08c18%2FSelection_030.png?generation=1606951459356411&alt=media)",
    "1108798": "dice is non-symmetric metric ... don't be afraid to detect more and make some mistake\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F149b10fd4c34d39acda60b89707cab45%2FSelection_185.png?generation=1607653899026735&alt=media)",
    "1099660": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F83e682c2a26ca0a9ca6f998d5987bd9f%2FSelection_029.png?generation=1606918300495338&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F7c148869c78d728d57468afd108005c2%2FSelection_030.png?generation=1606918314707954&alt=media)\n\n\nsee attached \"data.py\"",
    "1112076": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3964990%2Fc47049044214a2a580b8e156a6400526%2FSnipaste_2020-12-14_17-02-56.jpg?generation=1607936611952354&alt=media)\nDear washen,im not really understand about why ‘for training use hue-saturation’ is related to 'change sampling method'.Why isn't it related to augmentation?\nThanks!",
    "1110678": "https://arxiv.org/abs/2012.05780\nOneNet: Towards End-to-End One-Stage Object Detection\nobject detection without NMS",
    "1100673": "@hengck23 \nThanks for posting! This is a great help.\n\nBy the way, your code only predicts public test data, \nbut have you tried predicting it for private test data as well?\n\n```\ndef make_image_id (mode):\n    ...\n    test_image_id = {\n        0 : 'b9a3865fc',\n        1 : 'b2dc8411c',\n        2 : '26dc41664',\n        3 : 'c68fe75ea',\n        4 : 'afa5e8098',\n    }\n    ...\n    if 'test'==mode:\n        test_id = [ test_image_id[i] for i in [0,1,2,3,4] ]\n\n        return test_id\n```",
    "1129375": "direct polygon detection\nhttps://www.youtube.com/watch?v=sysySMr3YN4\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9c545ac92b64a6aba27d749ecd407753%2FSelection_224.png?generation=1609148868739418&alt=media)",
    "1102675": "the json file actually provide the ground truth ploygon coordinates.\nhence you can use the following works:\n- \"PolyTransform: Deep Polygon Transformer for Instance Segmentation\"\n\nhttps://openaccess.thecvf.com/content_CVPR_2020/papers/Liang_PolyTransform_Deep_Polygon_Transformer_for_Instance_Segmentation_CVPR_2020_paper.pdf\n\n\n- Annotating Object Instances with a Polygon-RNN\nEfficient Annotation of Segmentation Datasets with Polygon-RNN++\nhttp://www.cs.toronto.edu/polyrnn/poly_cvpr17/\nhttp://www.cs.toronto.edu/polyrnn/\n\n- Fast Interactive Object Annotation with Curve-GCN\nhttps://arxiv.org/pdf/1903.06874.pdf\n\n- Cell Detection with Star-convex Polygons\nhttps://arxiv.org/pdf/1806.03535.pdf",
    "1101335": "some interesting papers. the keyword for googling is \"WSI whole slide image\":\n\nhttps://paperswithcode.com/task/whole-slide-images.\n\n- https://arxiv.org/pdf/2007.13952v1.pdf\nEasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology\n(maybe some part of it can be used in Judge Prize)\n\n",
    "1100701": "You mentioned:\n\n> human error (I try to reannotate some train images)\n\nIs re-annotating allowed? I see some annotation errors in the training set as well.",
    "1128164": "Hi,How to get the .csv files.",
    "1294887": "Hi! Was anyone able to port his run_submit.py script to kaggle notebook that not exceeds the RAM size?",
    "1289818": "Is there some paper about \"transformer in segmentation modeling\"?",
    "1214014": "impressive work!",
    "1136473": "Hello,\ni meet an error related to Qt in linux. Do you have any idea? i find the solutions in google but all fail.\n\nGot keys from plugin meta data (\"xcb\")\nQFactoryLoader::QFactoryLoader() checking directory path \"/root/anaconda3/envs/python377/bin/platforms\" ...\nloaded library \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms/libqxcb.so\"\nqt.qpa.xcb: could not connect to display :0.0\nqt.qpa.xcb: could not connect to display :0.0\nqt.qpa.plugin: ****Could not load the Qt platform plugin \"xcb\" in \"/root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms\" even though it was found.\nThis application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.****\n\nAvailable platform plugins are: xcb (from /root/anaconda3/envs/python377/lib/python3.7/site-packages/cv2/qt/plugins/platforms), xcb, eglfs, minimal, minimalegl, offscreen, vnc.\n\n\n",
    "1124301": "Congrats on your competition GM already. I came across your wisdom multiple times wherever I go in the community. Thanks! ",
    "1124173": "Hi, @hengck23 .Thanks for sharing! May i know what exactly \"boundary artifacts\"  is? and why will it exist?",
    "1123225": "Hi @hengck23 thank you for your sharing, this is my first to join kaggel competition, the code is very helpful for me.  i have a dummpy question about  `def to_tile(image, mask=None,\n            scale=tile_scale, size=tile_size,\n            step=tile_average_step, min_score=tile_min_score)` , in the function the step=192, how do you get the number?",
    "1117651": "@hengck23 I checked your `2020-12-11/result/en-resnet34-256-aug-corrected/fold6/log.train.txt` text file. I saw from the `log.submit.txt`, that you chose the `fold6/checkpoint/00008500_model.pth` although the  0.914 validation dice was not the best, because 00009000_mode had 0.918. So my guess is, that it was not based on the validation dice accuracy.\nHow you evaluated your model? \nI know that it worked for you, because you have a good score, but I don't know how I can choose the best model.",
    "1110893": "hello, due to your idea is developed now, so your submit.csv is generated offline. when your code is fixed, could you publish the submission notebook? i'm looking forward to your reply, thanks!",
    "1110838": "hello, when i run the run_submit.py to evaluate the test set, there is a problem in test sample: c68fe75ea. I found that when execute the function \"to_tile\" and reture tile, the tile['tile_image'] is empty, it's obviously caused by the \"coord\" is empty in hubmap_v2.py. \nand i lower the value of \"tile_min_score\" to 0, but the tile['tile_image'] is also empty, i can't understand. i'm looking forward to your reply, thanks!\n\nok, that's caused by the c68fe75ea.json is empty , i download it from kaggle again",
    "1110218": "hello, i run the code of version12.11, and the num_iteration is 8000000, it''s so huge!!!!  it's nessary to train on such huge num_iteration? the total saved checkpoints are very large since the iter_save is the list with length 16000!",
    "1109835": "hello, in code of 12.11 version,  I have a problem of run_data.py, I didn't see any mask on these path, how to get these mask, thanks!\n        try:\n            mask_file = data_dir + '/train/{}.corrected_shift_mask.png'.format(id)\n            mask = read_mask(mask_file)\n        except:\n            mask_file = data_dir + '/train/{}.corrected_mask.png'.format(id)\n            mask = read_mask(mask_file)",
    "1109506": "Hi @hengck23, really thanks for sharing. What are corrected_shift_mask files, seems like you generate these file offline, it gives an not found error when run run_data.py in your last version.",
    "1108360": "hello, how to run  your code of the version(12.04)",
    "1107063": "http://www.orbit.bio/deep-learning-object-segmentation/\nquote \"If you just want to reuse a pre-trained model (e.g. our glomeruli detection model), copy&paste the segmentation script into the script editor (under tools), exchange the model name, edit the image IDs you want to segment (or use a search query) and click on run.\"",
    "1105296": "Thank you for this useful post. \n\nI downloaded your code from here:  https://drive.google.com/drive/folders/1Ygu1FL7Lig4dea-qLAS4f_SK3U4YZ5F4?usp=sharing\n\nDo you create the submission.csv online, or offline? I your code offline worked perfectly, but when i tried to run in a kaggle notebook I run out of RAM. I run only the `run_submit.py` in `server = 'kaggle'` mode.",
    "1102705": "I wonder why the different tile_size used between train and inference phase?",
    "1102660": "Hi, heng, nice to meet you.\n\nI lile your codes :) \n\nI have a question.\n\nWhat dataset do I need to use in the data directory?",
    "1102091": "Have you experienced stochasticity when training? I tried setting seeds and running the same fold with the same code. The different is up to 0.007. Is it due to the nature of the dataset? Thanks for the help.",
    "1100864": "Hey @hengck23! It's good to see you in this competition. ",
    "1100286": "things to do:\n- build size distribution for a single slide. see if we can use size to filter off fp, etc\n(the size of glomeruli is more or less fix in a single slide)\n\n- pseudo label the train instance (glomeruli) for difficult cases. this is to resample for training.",
    "1229983": "thanks for your job!"
  }
}