Hi,
\nI want to share this code for dealing with the Topology Exception Errors during the evaluation of the submission.
\nWe receive error messages like this:
Evaluation Exception: For image 6100_0_2, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.006539 -0.005582, 0.006539 -0.00558) and LINESTRING(0.006533 -0.005568, 0.00654 -0.005582) [ (0.006539, -0.00558, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution).\nThe idea is to make a squared hole centered in the given point of the submission. Sometimes we have to iterate many times (depending on the side of the square).
\nThe usage will be the following.
repair_topology_exception('/home/guillermo/sample_submission.csv', \n precision=6, \n image_id='6100_0_2', \n n_class=4, \n point= (0.006539, -0.005582), \n side=1e-4)\nHi,
\nI want to share this code for dealing with the Topology Exception Errors during the evaluation of the submission.
\nWe receive error messages like this:
Evaluation Exception: For image 6100_0_2, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.006539 -0.005582, 0.006539 -0.00558) and LINESTRING(0.006533 -0.005568, 0.00654 -0.005582) [ (0.006539, -0.00558, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution).\nThe idea is to make a squared hole centered in the given point of the submission. Sometimes we have to iterate many times (depending on the side of the square).
\nThe usage will be the following.
repair_topology_exception('/home/guillermo/sample_submission.csv', \n precision=6, \n image_id='6100_0_2', \n n_class=4, \n point= (0.006539, -0.005582), \n side=1e-4)\nHow do you get to see what generated the error during the evaluation? In \"My Submissions\" I only see that an Error ocurred during the evaluation, but I have no idea of what went wrong.
\n\nAlso, is there any way of knowing beforehand in which points the submissions is going to complain to use your repair_topology_exception tool? Or in order to produce a valid csv you need a lot of trial and error?
Thanks!
", "rawMarkdown": "How do you get to see what generated the error during the evaluation? In \"My Submissions\" I only see that an Error ocurred during the evaluation, but I have no idea of what went wrong. \n\nAlso, is there any way of knowing beforehand in which points the submissions is going to complain to use your `repair_topology_exception` tool? Or in order to produce a valid csv you need a lot of trial and error?\n\nThanks!", "votes": null }, { "id": "161332", "postDate": "02/13/2017 07:07:02", "content": "Hi Iñigo,
\nIn order to see the error message you have to wait until the end when you make your submission. And sometimes the message last for very little time so you must be ready.
\nI think Kaggle people are working to fix that, because in the old version of the web the errors were shown on \"my submissions\".
As far as I know it's impossible to predict where those errors will happen because they are caused in the intersection of polygons, and the other polygon is unknown.
", "rawMarkdown": "Hi Iñigo, \nIn order to see the error message you have to wait until the end when you make your submission. And sometimes the message last for very little time so you must be ready. \nI think Kaggle people are working to fix that, because in the old version of the web the errors were shown on \"my submissions\".\n\nAs far as I know it's impossible to predict where those errors will happen because they are caused in the intersection of polygons, and the other polygon is unknown.", "votes": null }, { "id": "161390", "postDate": "02/13/2017 14:34:22", "content": "Thanks, it works!
", "rawMarkdown": "Thanks, it works!", "votes": null }, { "id": "161853", "postDate": "02/16/2017 03:47:31", "content": "Thanks for the quick response ironbar. I cannot understand how did they design such an evaluation method, that can generate errors during evaluation even though the submitted polygons are valid and it's impossible to know beforehand whether the evaluation is going to end with an exception or not.
\n\nI think that the organizers should think about a way of solving this issue, as it is not very clear to me whether my current submission will be succesfully evaluated with the Private score test-set (100% of the submission images) instead that with the Public score test-set (19 % of the submission images).
", "rawMarkdown": "Thanks for the quick response ironbar. I cannot understand how did they design such an evaluation method, that can generate errors during evaluation even though the submitted polygons are valid and it's impossible to know beforehand whether the evaluation is going to end with an exception or not. \n\nI think that the organizers should think about a way of solving this issue, as it is not very clear to me whether my current submission will be succesfully evaluated with the Private score test-set (100% of the submission images) instead that with the Public score test-set (19 % of the submission images).", "votes": null } ], "comments": [ { "id": 161297, "author_name": "portillo", "author_url": "", "post_date": "02/13/2017 04:04:42", "content": "How do you get to see what generated the error during the evaluation? In \"My Submissions\" I only see that an Error ocurred during the evaluation, but I have no idea of what went wrong.
\n\nAlso, is there any way of knowing beforehand in which points the submissions is going to complain to use your repair_topology_exception tool? Or in order to produce a valid csv you need a lot of trial and error?
Thanks!
", "votes": null, "replies": [ { "id": 161332, "author_name": "ironbar", "author_url": "", "post_date": "02/13/2017 07:07:02", "content": "Hi Iñigo,
\nIn order to see the error message you have to wait until the end when you make your submission. And sometimes the message last for very little time so you must be ready.
\nI think Kaggle people are working to fix that, because in the old version of the web the errors were shown on \"my submissions\".
As far as I know it's impossible to predict where those errors will happen because they are caused in the intersection of polygons, and the other polygon is unknown.
", "votes": null, "replies": [] }, { "id": 161853, "author_name": "portillo", "author_url": "", "post_date": "02/16/2017 03:47:31", "content": "Thanks for the quick response ironbar. I cannot understand how did they design such an evaluation method, that can generate errors during evaluation even though the submitted polygons are valid and it's impossible to know beforehand whether the evaluation is going to end with an exception or not.
\n\nI think that the organizers should think about a way of solving this issue, as it is not very clear to me whether my current submission will be succesfully evaluated with the Private score test-set (100% of the submission images) instead that with the Public score test-set (19 % of the submission images).
", "votes": null, "replies": [] } ] }, { "id": 161390, "author_name": "rkcosmos", "author_url": "", "post_date": "02/13/2017 14:34:22", "content": "Thanks, it works!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "159837": "Hi, \nI want to share this code for dealing with the Topology Exception Errors during the evaluation of the submission. \nWe receive error messages like this:\n\n Evaluation Exception: For image 6100_0_2, class 4, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: found non-noded intersection between LINESTRING(0.006539 -0.005582, 0.006539 -0.00558) and LINESTRING(0.006533 -0.005568, 0.00654 -0.005582) [ (0.006539, -0.00558, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution).\n\nThe idea is to make a squared hole centered in the given point of the submission. Sometimes we have to iterate many times (depending on the side of the square). \nThe usage will be the following.\n\n repair_topology_exception('/home/guillermo/sample_submission.csv', \n precision=6, \n image_id='6100_0_2', \n n_class=4, \n point= (0.006539, -0.005582), \n side=1e-4)", "161297": "How do you get to see what generated the error during the evaluation? In \"My Submissions\" I only see that an Error ocurred during the evaluation, but I have no idea of what went wrong. \n\nAlso, is there any way of knowing beforehand in which points the submissions is going to complain to use your `repair_topology_exception` tool? Or in order to produce a valid csv you need a lot of trial and error?\n\nThanks!", "161332": "Hi Iñigo, \nIn order to see the error message you have to wait until the end when you make your submission. And sometimes the message last for very little time so you must be ready. \nI think Kaggle people are working to fix that, because in the old version of the web the errors were shown on \"my submissions\".\n\nAs far as I know it's impossible to predict where those errors will happen because they are caused in the intersection of polygons, and the other polygon is unknown.", "161390": "Thanks, it works!", "161853": "Thanks for the quick response ironbar. I cannot understand how did they design such an evaluation method, that can generate errors during evaluation even though the submitted polygons are valid and it's impossible to know beforehand whether the evaluation is going to end with an exception or not. \n\nI think that the organizers should think about a way of solving this issue, as it is not very clear to me whether my current submission will be succesfully evaluated with the Private score test-set (100% of the submission images) instead that with the Public score test-set (19 % of the submission images)." }, "source": "meta" }