{ "id": 27115, "title": "Another validation error", "url": "/competitions/dstl-satellite-imagery-feature-detection/discussion/27115", "author_name": "", "post_date": "2016-12-30T11:46:05.820Z", "votes": null, "comment_count": 9, "views": 371, "content": "

When I tried to submit my predicted WKT values, I received the following error message:

\n\n
\n

Evaluation Exception: For image 6110_2_3, class 1, there's an\n exception in geometry NetTopologySuite.Geometries.TopologyException:\n found non-noded intersection between LINESTRING(0.00361044017988\n -0.001738858250167, 0.003621217613253 -0.001782059697377) and LINESTRING(0.003615828896567 -0.001760458973772, 0.003583496596448\n -0.00174965861197) [ (0.0036158288965664545, -0.0017604589737718179, NaN) ] at\n Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry\n predMultipoly, IGeometry truthMultipoly) at\n Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary2\n submission, Dictionary2 solution)..

\n
\n\n

A quick search on GitHub suggests this might be a NetTopologySuite issue. However the post is already two years old and this issue seems to have been fixed. I do not have NaN in my WKT coordinates. So far, other than discarding the entire class for that image (or manually pin down the part of the polygon causing the problem), I do not have any workarounds for this problem.

", "messages": [ { "id": "153198", "postDate": "12/30/2016 11:46:05", "content": "

When I tried to submit my predicted WKT values, I received the following error message:

\n\n
\n

Evaluation Exception: For image 6110_2_3, class 1, there's an\n exception in geometry NetTopologySuite.Geometries.TopologyException:\n found non-noded intersection between LINESTRING(0.00361044017988\n -0.001738858250167, 0.003621217613253 -0.001782059697377) and LINESTRING(0.003615828896567 -0.001760458973772, 0.003583496596448\n -0.00174965861197) [ (0.0036158288965664545, -0.0017604589737718179, NaN) ] at\n Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry\n predMultipoly, IGeometry truthMultipoly) at\n Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary2\n submission, Dictionary2 solution)..

\n
\n\n

A quick search on GitHub suggests this might be a NetTopologySuite issue. However the post is already two years old and this issue seems to have been fixed. I do not have NaN in my WKT coordinates. So far, other than discarding the entire class for that image (or manually pin down the part of the polygon causing the problem), I do not have any workarounds for this problem.

", "rawMarkdown": "When I tried to submit my predicted WKT values, I received the following error message:\r\n\r\n> Evaluation Exception: For image 6110_2_3, class 1, there's an\r\n> exception in geometry NetTopologySuite.Geometries.TopologyException:\r\n> found non-noded intersection between LINESTRING(0.00361044017988\r\n> -0.001738858250167, 0.003621217613253 -0.001782059697377) and LINESTRING(0.003615828896567 -0.001760458973772, 0.003583496596448\r\n> -0.00174965861197) [ (0.0036158288965664545, -0.0017604589737718179, NaN) ] at\r\n> Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry\r\n> predMultipoly, IGeometry truthMultipoly) at\r\n> Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2\r\n> submission, Dictionary`2 solution)..\r\n\r\nA quick search on GitHub suggests this might be a NetTopologySuite issue. However the post is already two years old and this issue seems to have been fixed. I do not have NaN in my WKT coordinates. So far, other than discarding the entire class for that image (or manually pin down the part of the polygon causing the problem), I do not have any workarounds for this problem.", "votes": null }, { "id": "153212", "postDate": "12/30/2016 12:51:50", "content": "

use .buffer(0) of your Multipolygon object before turning them into wkt code.\nexample \nwktdata = multipoly.buffer(0).wkt\ninteadof \nwktdata = multipoly.wkt

", "rawMarkdown": "use .buffer(0) of your Multipolygon object before turning them into wkt code.\r\nexample \r\nwktdata = multipoly.buffer(0).wkt\r\ninteadof \r\nwktdata = multipoly.wkt", "votes": null }, { "id": "153213", "postDate": "12/30/2016 12:53:31", "content": "

you can use multipoly.is_valid to check if the object is valid, it's a boolean value.\ngood luck.

", "rawMarkdown": "you can use multipoly.is_valid to check if the object is valid, it's a boolean value.\r\ngood luck.", "votes": null }, { "id": "153299", "postDate": "12/30/2016 23:16:25", "content": "

@Mithrillion, unfortunately these TopologyException do exist in this competition's evaluation due to robustness issues, and that depends on what you submit. Some tips here about why they happen and how to avoid them.

", "rawMarkdown": "Mithrillion, unfortunately these `TopologyException` do exist in this competition's evaluation due to robustness issues, and that depends on what you submit. Some tips [here][1] about why they happen and how to avoid them. \n\n [1]: http://tsusiatsoftware.net/jts/jts-faq/jts-faq.html#D9", "votes": null }, { "id": "153545", "postDate": "01/02/2017 07:25:14", "content": "

@Wendy Kan, I'm having a similar issue but there's something weird about it:
\n

Evaluation Exception: For image 6120_4_4, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.0075655359665485018 -0.0073221133054570849, 0.0075956631869179949 -0.0072913968647580069) and LINESTRING(0.0075655359665485027 -0.0073221133054570849, 0.007551 -0.00731) [ (0.0075655359665485018, -0.0073221133054570849, NaN) ] at \n
\n
\nThe weird thing about it is that I don't have these coordinates in my submission the closest thing I could find to it is
0.00756525 -0.007365539999999999, 0.00756525 -0.00736288)
\nI am guessing it happens after the evaluation script compute the intersection between my submission and the ground truth, Could you please confirm that you apply a buffer(0) operation to that intersection before you compute its area. Thanks.
\nPS: all the geometry in my submission is valid, I have checked.

", "rawMarkdown": "Wendy Kan, I'm having a similar issue but there's something weird about it:
\r\n

Evaluation Exception: For image 6120_4_4, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.0075655359665485018 -0.0073221133054570849, 0.0075956631869179949 -0.0072913968647580069) and LINESTRING(0.0075655359665485027 -0.0073221133054570849, 0.007551 -0.00731) [ (0.0075655359665485018, -0.0073221133054570849, NaN) ] at \r\n
\r\n
\r\nThe weird thing about it is that I don't have these coordinates in my submission the closest thing I could find to it is
0.00756525 -0.007365539999999999, 0.00756525 -0.00736288)
\r\nI am guessing it happens after the evaluation script compute the intersection between my submission and the ground truth, Could you please confirm that you apply a buffer(0) operation to that intersection before you compute its area. Thanks.
\r\nPS: all the geometry in my submission is valid, I have checked.", "votes": null }, { "id": "153592", "postDate": "01/02/2017 12:52:34", "content": "

I am getting the same error.

\n\n

\"enter

", "rawMarkdown": "I am getting the same error. \r\n\r\n![enter image description here][1]\r\n\r\n\r\n [1]: https://s28.postimg.org/qozytct1p/error.png", "votes": null }, { "id": "153604", "postDate": "01/02/2017 13:50:06", "content": "

Same here. The funny thing is that when I tried submitting (a) with the faulty image/class replaced (as a sample submission) - it goes through. Usually it's just one image/class that I need to replace. It also gets better if I (over)-simplify my contours but I didn't try enough submissions to confirm that. Note that I have confirmed that shapely's is_valid is clean in all cases above (same as el's).

\n\n

The problem seems random, and I suspect that it is either due to rounding errors or some syntax in my submission since the error seems somewhat random and it is usually just one image/class that I need to hack/replace. I've checked and shapely's is_valid is clean even after I trim the decimal places to < 10.

\n\n

@Wendy / all - is there a API or tool we can use to check our submission files before uploading? It would be best if we can get a copy of the same checker used by the submission system. I suspect shapely's is_valid does not fully capture the validation behavior used.

", "rawMarkdown": "Same here. The funny thing is that when I tried submitting (a) with the faulty image/class replaced (as a sample submission) - it goes through. Usually it's just one image/class that I need to replace. It also gets better if I (over)-simplify my contours but I didn't try enough submissions to confirm that. Note that I have confirmed that shapely's is_valid is clean in all cases above (same as el's).\r\n\r\nThe problem seems random, and I suspect that it is either due to rounding errors or some syntax in my submission since the error seems somewhat random and it is usually just one image/class that I need to hack/replace. I've checked and shapely's is_valid is clean even after I trim the decimal places to < 10.\r\n\r\n@Wendy / all - is there a API or tool we can use to check our submission files before uploading? It would be best if we can get a copy of the same checker used by the submission system. I suspect shapely's is_valid does not fully capture the validation behavior used.", "votes": null }, { "id": "153615", "postDate": "01/02/2017 14:39:29", "content": "

If you search for \"non-noded intersection\" you get a lot of hits from various GIS packages. A lot of them (including shapely) use the geos library, so the non-noded issue is the same in all. I couldn't find a solid explanation of what it means, but the most common advice I see is to drop some precision from the polygons, which explains why smoothing would work for KyleLee.

\n\n

If you are getting this error are you using opencv or rasterio to create polygons?

", "rawMarkdown": "If you search for \"non-noded intersection\" you get a lot of hits from various GIS packages. A lot of them (including shapely) use the geos library, so the non-noded issue is the same in all. I couldn't find a solid explanation of what it means, but the most common advice I see is to drop some precision from the polygons, which explains why smoothing would work for KyleLee. \r\n\r\nIf you are getting this error are you using opencv or rasterio to create polygons?", "votes": null }, { "id": "153666", "postDate": "01/02/2017 18:01:55", "content": "

I was using opencv findContours + cascaded_union in shapely to create the polygons, but that was probably what caused the problem. Upon using rasterio to loop through mask shapes I could submit successfully - however I had to do some optimizations to speed up the process (mask smoothing/approximation, multithreading). On the rasterio code refer to shawn's post #6: https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection/forums/t/27120/modelling-strategies (thanks shawn!)

", "rawMarkdown": "I was using opencv findContours + cascaded_union in shapely to create the polygons, but that was probably what caused the problem. Upon using rasterio to loop through mask shapes I could submit successfully - however I had to do some optimizations to speed up the process (mask smoothing/approximation, multithreading). On the rasterio code refer to shawn's post #6: https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection/forums/t/27120/modelling-strategies (thanks shawn!)", "votes": null }, { "id": "153678", "postDate": "01/02/2017 18:44:14", "content": "

I got the error with both rasterio and potrace, I also lowered the precision to 8 decimal points. sometimes it works and sometimes it doesn't.\n
I've also noticed that shapely .wkt and to_wkt() yields different results.

", "rawMarkdown": "I got the error with both rasterio and potrace, I also lowered the precision to 8 decimal points. sometimes it works and sometimes it doesn't.\r\n
I've also noticed that shapely .wkt and to_wkt() yields different results.", "votes": null } ], "comments": [ { "id": 153212, "author_name": "eljeilany", "author_url": "", "post_date": "12/30/2016 12:51:50", "content": "

use .buffer(0) of your Multipolygon object before turning them into wkt code.\nexample \nwktdata = multipoly.buffer(0).wkt\ninteadof \nwktdata = multipoly.wkt

", "votes": null, "replies": [] }, { "id": 153213, "author_name": "eljeilany", "author_url": "", "post_date": "12/30/2016 12:53:31", "content": "

you can use multipoly.is_valid to check if the object is valid, it's a boolean value.\ngood luck.

", "votes": null, "replies": [] }, { "id": 153299, "author_name": "wendykan", "author_url": "", "post_date": "12/30/2016 23:16:25", "content": "

@Mithrillion, unfortunately these TopologyException do exist in this competition's evaluation due to robustness issues, and that depends on what you submit. Some tips here about why they happen and how to avoid them.

", "votes": null, "replies": [] }, { "id": 153545, "author_name": "eljeilany", "author_url": "", "post_date": "01/02/2017 07:25:14", "content": "

@Wendy Kan, I'm having a similar issue but there's something weird about it:
\n

Evaluation Exception: For image 6120_4_4, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.0075655359665485018 -0.0073221133054570849, 0.0075956631869179949 -0.0072913968647580069) and LINESTRING(0.0075655359665485027 -0.0073221133054570849, 0.007551 -0.00731) [ (0.0075655359665485018, -0.0073221133054570849, NaN) ] at \n
\n
\nThe weird thing about it is that I don't have these coordinates in my submission the closest thing I could find to it is
0.00756525 -0.007365539999999999, 0.00756525 -0.00736288)
\nI am guessing it happens after the evaluation script compute the intersection between my submission and the ground truth, Could you please confirm that you apply a buffer(0) operation to that intersection before you compute its area. Thanks.
\nPS: all the geometry in my submission is valid, I have checked.

", "votes": null, "replies": [] }, { "id": 153592, "author_name": "godaibo", "author_url": "", "post_date": "01/02/2017 12:52:34", "content": "

I am getting the same error.

\n\n

\"enter

", "votes": null, "replies": [] }, { "id": 153604, "author_name": "kylelee", "author_url": "", "post_date": "01/02/2017 13:50:06", "content": "

Same here. The funny thing is that when I tried submitting (a) with the faulty image/class replaced (as a sample submission) - it goes through. Usually it's just one image/class that I need to replace. It also gets better if I (over)-simplify my contours but I didn't try enough submissions to confirm that. Note that I have confirmed that shapely's is_valid is clean in all cases above (same as el's).

\n\n

The problem seems random, and I suspect that it is either due to rounding errors or some syntax in my submission since the error seems somewhat random and it is usually just one image/class that I need to hack/replace. I've checked and shapely's is_valid is clean even after I trim the decimal places to < 10.

\n\n

@Wendy / all - is there a API or tool we can use to check our submission files before uploading? It would be best if we can get a copy of the same checker used by the submission system. I suspect shapely's is_valid does not fully capture the validation behavior used.

", "votes": null, "replies": [] }, { "id": 153615, "author_name": "shawn775", "author_url": "", "post_date": "01/02/2017 14:39:29", "content": "

If you search for \"non-noded intersection\" you get a lot of hits from various GIS packages. A lot of them (including shapely) use the geos library, so the non-noded issue is the same in all. I couldn't find a solid explanation of what it means, but the most common advice I see is to drop some precision from the polygons, which explains why smoothing would work for KyleLee.

\n\n

If you are getting this error are you using opencv or rasterio to create polygons?

", "votes": null, "replies": [] }, { "id": 153666, "author_name": "kylelee", "author_url": "", "post_date": "01/02/2017 18:01:55", "content": "

I was using opencv findContours + cascaded_union in shapely to create the polygons, but that was probably what caused the problem. Upon using rasterio to loop through mask shapes I could submit successfully - however I had to do some optimizations to speed up the process (mask smoothing/approximation, multithreading). On the rasterio code refer to shawn's post #6: https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection/forums/t/27120/modelling-strategies (thanks shawn!)

", "votes": null, "replies": [] }, { "id": 153678, "author_name": "eljeilany", "author_url": "", "post_date": "01/02/2017 18:44:14", "content": "

I got the error with both rasterio and potrace, I also lowered the precision to 8 decimal points. sometimes it works and sometimes it doesn't.\n
I've also noticed that shapely .wkt and to_wkt() yields different results.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "153198": "When I tried to submit my predicted WKT values, I received the following error message:\r\n\r\n> Evaluation Exception: For image 6110_2_3, class 1, there's an\r\n> exception in geometry NetTopologySuite.Geometries.TopologyException:\r\n> found non-noded intersection between LINESTRING(0.00361044017988\r\n> -0.001738858250167, 0.003621217613253 -0.001782059697377) and LINESTRING(0.003615828896567 -0.001760458973772, 0.003583496596448\r\n> -0.00174965861197) [ (0.0036158288965664545, -0.0017604589737718179, NaN) ] at\r\n> Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry\r\n> predMultipoly, IGeometry truthMultipoly) at\r\n> Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2\r\n> submission, Dictionary`2 solution)..\r\n\r\nA quick search on GitHub suggests this might be a NetTopologySuite issue. However the post is already two years old and this issue seems to have been fixed. I do not have NaN in my WKT coordinates. So far, other than discarding the entire class for that image (or manually pin down the part of the polygon causing the problem), I do not have any workarounds for this problem.", "153212": "use .buffer(0) of your Multipolygon object before turning them into wkt code.\r\nexample \r\nwktdata = multipoly.buffer(0).wkt\r\ninteadof \r\nwktdata = multipoly.wkt", "153213": "you can use multipoly.is_valid to check if the object is valid, it's a boolean value.\r\ngood luck.", "153299": "Mithrillion, unfortunately these `TopologyException` do exist in this competition's evaluation due to robustness issues, and that depends on what you submit. Some tips [here][1] about why they happen and how to avoid them. \n\n [1]: http://tsusiatsoftware.net/jts/jts-faq/jts-faq.html#D9", "153545": "Wendy Kan, I'm having a similar issue but there's something weird about it:
\r\n

Evaluation Exception: For image 6120_4_4, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.0075655359665485018 -0.0073221133054570849, 0.0075956631869179949 -0.0072913968647580069) and LINESTRING(0.0075655359665485027 -0.0073221133054570849, 0.007551 -0.00731) [ (0.0075655359665485018, -0.0073221133054570849, NaN) ] at \r\n
\r\n
\r\nThe weird thing about it is that I don't have these coordinates in my submission the closest thing I could find to it is
0.00756525 -0.007365539999999999, 0.00756525 -0.00736288)
\r\nI am guessing it happens after the evaluation script compute the intersection between my submission and the ground truth, Could you please confirm that you apply a buffer(0) operation to that intersection before you compute its area. Thanks.
\r\nPS: all the geometry in my submission is valid, I have checked.", "153592": "I am getting the same error. \r\n\r\n![enter image description here][1]\r\n\r\n\r\n [1]: https://s28.postimg.org/qozytct1p/error.png", "153604": "Same here. The funny thing is that when I tried submitting (a) with the faulty image/class replaced (as a sample submission) - it goes through. Usually it's just one image/class that I need to replace. It also gets better if I (over)-simplify my contours but I didn't try enough submissions to confirm that. Note that I have confirmed that shapely's is_valid is clean in all cases above (same as el's).\r\n\r\nThe problem seems random, and I suspect that it is either due to rounding errors or some syntax in my submission since the error seems somewhat random and it is usually just one image/class that I need to hack/replace. I've checked and shapely's is_valid is clean even after I trim the decimal places to < 10.\r\n\r\n@Wendy / all - is there a API or tool we can use to check our submission files before uploading? It would be best if we can get a copy of the same checker used by the submission system. I suspect shapely's is_valid does not fully capture the validation behavior used.", "153615": "If you search for \"non-noded intersection\" you get a lot of hits from various GIS packages. A lot of them (including shapely) use the geos library, so the non-noded issue is the same in all. I couldn't find a solid explanation of what it means, but the most common advice I see is to drop some precision from the polygons, which explains why smoothing would work for KyleLee. \r\n\r\nIf you are getting this error are you using opencv or rasterio to create polygons?", "153666": "I was using opencv findContours + cascaded_union in shapely to create the polygons, but that was probably what caused the problem. Upon using rasterio to loop through mask shapes I could submit successfully - however I had to do some optimizations to speed up the process (mask smoothing/approximation, multithreading). On the rasterio code refer to shawn's post #6: https://www.kaggle.com/c/dstl-satellite-imagery-feature-detection/forums/t/27120/modelling-strategies (thanks shawn!)", "153678": "I got the error with both rasterio and potrace, I also lowered the precision to 8 decimal points. sometimes it works and sometimes it doesn't.\r\n
I've also noticed that shapely .wkt and to_wkt() yields different results." }, "source": "meta" }