{
  "id": 27780,
  "title": "Jaccard(poly vs poly)",
  "url": "/competitions/dstl-satellite-imagery-feature-detection/discussion/27780",
  "author_name": "",
  "post_date": "2017-01-16T07:36:10.380Z",
  "votes": 2,
  "comment_count": 3,
  "views": 300,
  "content": "<p>I tried to write a test that checks if I calculate polygons from mask to mask correctly.</p>\n\n<p><a href=\"https://www.kaggle.com/iglovikov/dstl-satellite-imagery-feature-detection/jaccard-polygons-polygons-mask-polygons\">https://www.kaggle.com/iglovikov/dstl-satellite-imagery-feature-detection/jaccard-polygons-polygons-mask-polygons</a></p>\n\n<p>I would assume that </p>\n\n<p>poly =&gt; mask =&gt; poly </p>\n\n<p>should give very similar polygons on left and right.</p>\n\n<p>To check it I do:</p>\n\n<p>poly1 =&gt; mask1 =&gt; poly2 =&gt; mask2</p>\n\n<p>and calculate Jaccard(mask1, mask2), </p>\n\n<p>but the result is not even close to one for every train image. I am still looking for bugs, but if someone was struggling with similar issues, advice are highly appreciated.</p>",
  "messages": [
    {
      "id": "156417",
      "postDate": "01/16/2017 07:36:10",
      "content": "<p>I tried to write a test that checks if I calculate polygons from mask to mask correctly.</p>\n\n<p><a href=\"https://www.kaggle.com/iglovikov/dstl-satellite-imagery-feature-detection/jaccard-polygons-polygons-mask-polygons\">https://www.kaggle.com/iglovikov/dstl-satellite-imagery-feature-detection/jaccard-polygons-polygons-mask-polygons</a></p>\n\n<p>I would assume that </p>\n\n<p>poly =&gt; mask =&gt; poly </p>\n\n<p>should give very similar polygons on left and right.</p>\n\n<p>To check it I do:</p>\n\n<p>poly1 =&gt; mask1 =&gt; poly2 =&gt; mask2</p>\n\n<p>and calculate Jaccard(mask1, mask2), </p>\n\n<p>but the result is not even close to one for every train image. I am still looking for bugs, but if someone was struggling with similar issues, advice are highly appreciated.</p>",
      "rawMarkdown": "I tried to write a test that checks if I calculate polygons from mask to mask correctly.\r\n\r\nhttps://www.kaggle.com/iglovikov/dstl-satellite-imagery-feature-detection/jaccard-polygons-polygons-mask-polygons\r\n\r\nI would assume that \r\n\r\npoly => mask => poly \r\n\r\nshould give very similar polygons on left and right.\r\n\r\nTo check it I do:\r\n\r\npoly1 => mask1 => poly2 => mask2\r\n\r\nand calculate Jaccard(mask1, mask2), \r\n\r\nbut the result is not even close to one for every train image. I am still looking for bugs, but if someone was struggling with similar issues, advice are highly appreciated.",
      "votes": null
    },
    {
      "id": "156442",
      "postDate": "01/16/2017 10:07:02",
      "content": "<p>Works for me. Are you on the same scale ? Do mask1 &amp; mask2 have the same shape ?</p>\n\n<p>First thing I did to validate this, was a dummy model returning a fully activated mask like the sample submission, and checking that the coordinate values were similar. This way, you can confirm half of the way \"mask =&gt; poly\"</p>",
      "rawMarkdown": "Works for me. Are you on the same scale ? Do mask1 & mask2 have the same shape ?\r\n\r\nFirst thing I did to validate this, was a dummy model returning a fully activated mask like the sample submission, and checking that the coordinate values were similar. This way, you can confirm half of the way \"mask => poly\"",
      "votes": null
    },
    {
      "id": "156592",
      "postDate": "01/17/2017 06:30:49",
      "content": "<p>Hi,</p>\n\n<p>I found recently that there may be an error in Data Processing Tutorial. I was trying to identify cars and found that when I plot the polygons (for class 10) above original image, there is a big shift between actual cars and polygons they correspond to. In order to plot a polygon, I scale it with transform method from the shapely package:</p>\n\n<p>new_polygon = transform(lambda x,y: (x*scale_x, y*scale_y), old_polygon)</p>\n\n<p>where</p>\n\n<p>scale_x = img.shape[0] **2 / ((img.shape[0]+1)*x_max)</p>\n\n<p>scale_y = img.shape[1] **2 / ((img.shape[1]+1)*y_min)</p>\n\n<p>In the tutorial shape[0] == WIDTH and shape[1]==HEIGHT, and it says that height must be larger than width. However, when I switched width and height (width-&gt;y, height-&gt;x) all polygons matched car objects perfectly. I am not 100% sure that it is not a bug in my code, and going to check  it on weekends.</p>",
      "rawMarkdown": "Hi,\r\n\r\nI found recently that there may be an error in Data Processing Tutorial. I was trying to identify cars and found that when I plot the polygons (for class 10) above original image, there is a big shift between actual cars and polygons they correspond to. In order to plot a polygon, I scale it with transform method from the shapely package:\r\n\r\nnew_polygon = transform(lambda x,y: (x*scale_x, y*scale_y), old_polygon)\r\n\r\nwhere\r\n\r\nscale_x = img.shape[0] **2 / ((img.shape[0]+1)*x_max)\r\n\r\nscale_y = img.shape[1] **2 / ((img.shape[1]+1)*y_min)\r\n\r\nIn the tutorial shape[0] == WIDTH and shape[1]==HEIGHT, and it says that height must be larger than width. However, when I switched width and height (width->y, height->x) all polygons matched car objects perfectly. I am not 100% sure that it is not a bug in my code, and going to check  it on weekends.",
      "votes": null
    },
    {
      "id": "156988",
      "postDate": "01/18/2017 20:51:13",
      "content": "<p>Although we always talk about images being width x height, the arrays of data are organized img[height, width]. So you're probably getting the height and width mixed up. (Or a least, that's what happened to me.)</p>\n\n<p>(Also, on a more general note, I can't make any sense of the formula to transform grid to pixel coordinates given in data processing tutorial. I've come to the conclusion that it's wrong. Although approximately true since\n    W *W / (W+1) = W-1 + O(1/W)  [Taylor series expand about 0 wrt 1/W] </p>\n\n<p>'W-1' does make sense. Its a fencepost correction. If my images are 100 pixels  wide,  but we count from 0, then the last pixel is 99. )</p>",
      "rawMarkdown": "Although we always talk about images being width x height, the arrays of data are organized img[height, width]. So you're probably getting the height and width mixed up. (Or a least, that's what happened to me.)\r\n\r\n(Also, on a more general note, I can't make any sense of the formula to transform grid to pixel coordinates given in data processing tutorial. I've come to the conclusion that it's wrong. Although approximately true since\r\n    W *W / (W+1) = W-1 + O(1/W)  [Taylor series expand about 0 wrt 1/W] \r\n\r\n'W-1' does make sense. Its a fencepost correction. If my images are 100 pixels  wide,  but we count from 0, then the last pixel is 99. )",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 156442,
      "author_name": "mxdbld",
      "author_url": "",
      "post_date": "01/16/2017 10:07:02",
      "content": "<p>Works for me. Are you on the same scale ? Do mask1 &amp; mask2 have the same shape ?</p>\n\n<p>First thing I did to validate this, was a dummy model returning a fully activated mask like the sample submission, and checking that the coordinate values were similar. This way, you can confirm half of the way \"mask =&gt; poly\"</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 156592,
      "author_name": "alexlu",
      "author_url": "",
      "post_date": "01/17/2017 06:30:49",
      "content": "<p>Hi,</p>\n\n<p>I found recently that there may be an error in Data Processing Tutorial. I was trying to identify cars and found that when I plot the polygons (for class 10) above original image, there is a big shift between actual cars and polygons they correspond to. In order to plot a polygon, I scale it with transform method from the shapely package:</p>\n\n<p>new_polygon = transform(lambda x,y: (x*scale_x, y*scale_y), old_polygon)</p>\n\n<p>where</p>\n\n<p>scale_x = img.shape[0] **2 / ((img.shape[0]+1)*x_max)</p>\n\n<p>scale_y = img.shape[1] **2 / ((img.shape[1]+1)*y_min)</p>\n\n<p>In the tutorial shape[0] == WIDTH and shape[1]==HEIGHT, and it says that height must be larger than width. However, when I switched width and height (width-&gt;y, height-&gt;x) all polygons matched car objects perfectly. I am not 100% sure that it is not a bug in my code, and going to check  it on weekends.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 156988,
      "author_name": "threeplusone",
      "author_url": "",
      "post_date": "01/18/2017 20:51:13",
      "content": "<p>Although we always talk about images being width x height, the arrays of data are organized img[height, width]. So you're probably getting the height and width mixed up. (Or a least, that's what happened to me.)</p>\n\n<p>(Also, on a more general note, I can't make any sense of the formula to transform grid to pixel coordinates given in data processing tutorial. I've come to the conclusion that it's wrong. Although approximately true since\n    W *W / (W+1) = W-1 + O(1/W)  [Taylor series expand about 0 wrt 1/W] </p>\n\n<p>'W-1' does make sense. Its a fencepost correction. If my images are 100 pixels  wide,  but we count from 0, then the last pixel is 99. )</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "156417": "I tried to write a test that checks if I calculate polygons from mask to mask correctly.\r\n\r\nhttps://www.kaggle.com/iglovikov/dstl-satellite-imagery-feature-detection/jaccard-polygons-polygons-mask-polygons\r\n\r\nI would assume that \r\n\r\npoly => mask => poly \r\n\r\nshould give very similar polygons on left and right.\r\n\r\nTo check it I do:\r\n\r\npoly1 => mask1 => poly2 => mask2\r\n\r\nand calculate Jaccard(mask1, mask2), \r\n\r\nbut the result is not even close to one for every train image. I am still looking for bugs, but if someone was struggling with similar issues, advice are highly appreciated.",
    "156442": "Works for me. Are you on the same scale ? Do mask1 & mask2 have the same shape ?\r\n\r\nFirst thing I did to validate this, was a dummy model returning a fully activated mask like the sample submission, and checking that the coordinate values were similar. This way, you can confirm half of the way \"mask => poly\"",
    "156592": "Hi,\r\n\r\nI found recently that there may be an error in Data Processing Tutorial. I was trying to identify cars and found that when I plot the polygons (for class 10) above original image, there is a big shift between actual cars and polygons they correspond to. In order to plot a polygon, I scale it with transform method from the shapely package:\r\n\r\nnew_polygon = transform(lambda x,y: (x*scale_x, y*scale_y), old_polygon)\r\n\r\nwhere\r\n\r\nscale_x = img.shape[0] **2 / ((img.shape[0]+1)*x_max)\r\n\r\nscale_y = img.shape[1] **2 / ((img.shape[1]+1)*y_min)\r\n\r\nIn the tutorial shape[0] == WIDTH and shape[1]==HEIGHT, and it says that height must be larger than width. However, when I switched width and height (width->y, height->x) all polygons matched car objects perfectly. I am not 100% sure that it is not a bug in my code, and going to check  it on weekends.",
    "156988": "Although we always talk about images being width x height, the arrays of data are organized img[height, width]. So you're probably getting the height and width mixed up. (Or a least, that's what happened to me.)\r\n\r\n(Also, on a more general note, I can't make any sense of the formula to transform grid to pixel coordinates given in data processing tutorial. I've come to the conclusion that it's wrong. Although approximately true since\r\n    W *W / (W+1) = W-1 + O(1/W)  [Taylor series expand about 0 wrt 1/W] \r\n\r\n'W-1' does make sense. Its a fencepost correction. If my images are 100 pixels  wide,  but we count from 0, then the last pixel is 99. )"
  },
  "source": "meta"
}