{
  "id": 26661,
  "title": "Submission errors",
  "url": "/competitions/dstl-satellite-imagery-feature-detection/discussion/26661",
  "author_name": "",
  "post_date": "2016-12-19T01:40:32.233Z",
  "votes": 2,
  "comment_count": 9,
  "views": 394,
  "content": "<p>There are 21 images in the original train_wkt.csv, 22 in the new train_wkt_v2.csv and 25 in the newest train_wkt_v3.csv.</p>\n\n<p>This corresponds to 4290, 4280 and 4250 rows in the submission file, respectively. The submission will only accept the former, even though the latter two have at some point this weekend been the more correct versions.</p>\n\n<p>Additionally, even if I submit with 4290 rows by excluding the imageIDs from train_wkt.csv from my submission, I get the following error:</p>\n\n<pre><code>Evaluation Exception: The given key was not present in the dictionary..\n</code></pre>\n\n<p>Which I presume means that one of the imageIDs shouldn't be present?</p>\n\n<p>Some clarification would be helpful.</p>",
  "messages": [
    {
      "id": "151136",
      "postDate": "12/19/2016 01:40:32",
      "content": "<p>There are 21 images in the original train_wkt.csv, 22 in the new train_wkt_v2.csv and 25 in the newest train_wkt_v3.csv.</p>\n\n<p>This corresponds to 4290, 4280 and 4250 rows in the submission file, respectively. The submission will only accept the former, even though the latter two have at some point this weekend been the more correct versions.</p>\n\n<p>Additionally, even if I submit with 4290 rows by excluding the imageIDs from train_wkt.csv from my submission, I get the following error:</p>\n\n<pre><code>Evaluation Exception: The given key was not present in the dictionary..\n</code></pre>\n\n<p>Which I presume means that one of the imageIDs shouldn't be present?</p>\n\n<p>Some clarification would be helpful.</p>",
      "rawMarkdown": "There are 21 images in the original train_wkt.csv, 22 in the new train_wkt_v2.csv and 25 in the newest train_wkt_v3.csv.\r\n\r\nThis corresponds to 4290, 4280 and 4250 rows in the submission file, respectively. The submission will only accept the former, even though the latter two have at some point this weekend been the more correct versions.\r\n\r\nAdditionally, even if I submit with 4290 rows by excluding the imageIDs from train_wkt.csv from my submission, I get the following error:\r\n\r\n    Evaluation Exception: The given key was not present in the dictionary..\r\n\r\nWhich I presume means that one of the imageIDs shouldn't be present?\r\n\r\nSome clarification would be helpful.",
      "votes": null
    },
    {
      "id": "151263",
      "postDate": "12/19/2016 19:24:39",
      "content": "<p>@grantbey,</p>\n\n<p>The submission file/format hasn't changed, but 4 images (the ones that we moved from test to train) have been ignored. Therefore, all you need to do is to submit the same format as <em>sample_submission.csv</em> and you should be good to go. For those 4 images, whatever you submit won't matter, so it's easiest to keep them empty \"MULTIPOLYGON EMPTY\". </p>",
      "rawMarkdown": "grantbey,\r\n\r\nThe submission file/format hasn't changed, but 4 images (the ones that we moved from test to train) have been ignored. Therefore, all you need to do is to submit the same format as *sample_submission.csv* and you should be good to go. For those 4 images, whatever you submit won't matter, so it's easiest to keep them empty \"MULTIPOLYGON EMPTY\".",
      "votes": null
    },
    {
      "id": "151265",
      "postDate": "12/19/2016 19:37:32",
      "content": "<p>Thanks for the reply @wendykan</p>\n\n<p>This is what I'm doing, however I still get the error mentioned above:</p>\n\n<pre><code>Evaluation Exception: The given key was not present in the dictionary..\n</code></pre>\n\n<p>Can you give some insight into what this means so that I might be able to correct it?</p>",
      "rawMarkdown": "Thanks for the reply @wendykan\r\n\r\nThis is what I'm doing, however I still get the error mentioned above:\r\n\r\n    Evaluation Exception: The given key was not present in the dictionary..\r\n\r\nCan you give some insight into what this means so that I might be able to correct it?",
      "votes": null
    },
    {
      "id": "151267",
      "postDate": "12/19/2016 19:55:40",
      "content": "<p>(I've taken it off the forum to help @grantbey) </p>",
      "rawMarkdown": "(I've taken it off the forum to help @grantbey)",
      "votes": null
    },
    {
      "id": "151343",
      "postDate": "12/20/2016 07:47:53",
      "content": "<p>I have the same problem. I checked my submission against sample_submission.csv and couldn't find the problem. All ImageId and ClassType seemed to be present.</p>",
      "rawMarkdown": "I have the same problem. I checked my submission against sample_submission.csv and couldn't find the problem. All ImageId and ClassType seemed to be present.",
      "votes": null
    },
    {
      "id": "151384",
      "postDate": "12/20/2016 14:08:26",
      "content": "<p>Hey @zfturbo, I haven't tried recently (@wendykan - I saw your email, I've just been a bit preoccupied) but I did identify one error in my work:</p>\n\n<p>The classType's are 1-indexed in the sample submission, that is they run from 1-10, whereas my code more naturally output 0-indexed classType's (i.e. from 0-9). It's a simple fix, although I haven't tested it yet. Maybe this is our problem?</p>\n\n<p>Let me know if this helps you!</p>",
      "rawMarkdown": "Hey @zfturbo, I haven't tried recently (@wendykan - I saw your email, I've just been a bit preoccupied) but I did identify one error in my work:\r\n\r\nThe classType's are 1-indexed in the sample submission, that is they run from 1-10, whereas my code more naturally output 0-indexed classType's (i.e. from 0-9). It's a simple fix, although I haven't tested it yet. Maybe this is our problem?\r\n\r\nLet me know if this helps you!",
      "votes": null
    },
    {
      "id": "151386",
      "postDate": "12/20/2016 14:25:30",
      "content": "<p><strong>grantbey</strong> - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.</p>\n\n<p>Error in submission evaluation should explain why TOP of leaderboard stays the same for days.</p>",
      "rawMarkdown": "**grantbey** - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.\r\n\r\nError in submission evaluation should explain why TOP of leaderboard stays the same for days.",
      "votes": null
    },
    {
      "id": "151387",
      "postDate": "12/20/2016 14:32:05",
      "content": "<p>[quote=ZFTurbo;151386]</p>\n\n<p><strong>grantbey</strong> - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.</p>\n\n<p>Error in submission evaluation should explain why TOP of leaderboard stays the same for days.</p>\n\n<p>[/quote]</p>\n\n<p>Damn, I was hoping this would be the solution. As I said, I haven't tried to submit for a while now. This competition seems fairly quiet still, so it's possible that not many people are submitting yet. One thinks if others were having this problem then the forums would reflect that. So far, it's you and me!</p>",
      "rawMarkdown": "[quote=ZFTurbo;151386]\r\n\r\n**grantbey** - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.\r\n\r\nError in submission evaluation should explain why TOP of leaderboard stays the same for days.\r\n\r\n[/quote]\r\n\r\nDamn, I was hoping this would be the solution. As I said, I haven't tried to submit for a while now. This competition seems fairly quiet still, so it's possible that not many people are submitting yet. One thinks if others were having this problem then the forums would reflect that. So far, it's you and me!",
      "votes": null
    },
    {
      "id": "151424",
      "postDate": "12/20/2016 19:03:35",
      "content": "<p>@ZFTurbo,</p>\n\n<p>If you sort the submission file to be in the exact same order of ImageId, ClassType as the sample_submission.csv, it should work. </p>\n\n<p>It is caused by our evaluation infrastructure only recognizing the first column as the \"key\", so it's unable to match two columns as the keys for submission files. </p>\n\n<p>Let me know if that works for you.</p>",
      "rawMarkdown": "ZFTurbo,\r\n\r\nIf you sort the submission file to be in the exact same order of ImageId, ClassType as the sample_submission.csv, it should work. \r\n\r\nIt is caused by our evaluation infrastructure only recognizing the first column as the \"key\", so it's unable to match two columns as the keys for submission files. \r\n\r\nLet me know if that works for you.",
      "votes": null
    },
    {
      "id": "151427",
      "postDate": "12/20/2016 19:51:42",
      "content": "<p><strong>@Wendy Kan</strong> It solves the problem. But now I have the following error:</p>\n\n<pre><code>Evaluation Exception: For image 6150_3_4, class 5, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: no outgoing dirEdge found [ (1.79996078946E-05, -0.0073208405217139, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution)..\n</code></pre>\n\n<p>The problem is: there is no NaNs in my submission. I don't have numbers mentioned in Exception for line: 6150_3_4,5,</p>",
      "rawMarkdown": "**@Wendy Kan** It solves the problem. But now I have the following error:\r\n\r\n    Evaluation Exception: For image 6150_3_4, class 5, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: no outgoing dirEdge found [ (1.79996078946E-05, -0.0073208405217139, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution)..\r\n\r\nThe problem is: there is no NaNs in my submission. I don't have numbers mentioned in Exception for line: 6150_3_4,5,",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 151263,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "12/19/2016 19:24:39",
      "content": "<p>@grantbey,</p>\n\n<p>The submission file/format hasn't changed, but 4 images (the ones that we moved from test to train) have been ignored. Therefore, all you need to do is to submit the same format as <em>sample_submission.csv</em> and you should be good to go. For those 4 images, whatever you submit won't matter, so it's easiest to keep them empty \"MULTIPOLYGON EMPTY\". </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151265,
      "author_name": "grantbey",
      "author_url": "",
      "post_date": "12/19/2016 19:37:32",
      "content": "<p>Thanks for the reply @wendykan</p>\n\n<p>This is what I'm doing, however I still get the error mentioned above:</p>\n\n<pre><code>Evaluation Exception: The given key was not present in the dictionary..\n</code></pre>\n\n<p>Can you give some insight into what this means so that I might be able to correct it?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151267,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "12/19/2016 19:55:40",
      "content": "<p>(I've taken it off the forum to help @grantbey) </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151343,
      "author_name": "zfturbo",
      "author_url": "",
      "post_date": "12/20/2016 07:47:53",
      "content": "<p>I have the same problem. I checked my submission against sample_submission.csv and couldn't find the problem. All ImageId and ClassType seemed to be present.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151384,
      "author_name": "grantbey",
      "author_url": "",
      "post_date": "12/20/2016 14:08:26",
      "content": "<p>Hey @zfturbo, I haven't tried recently (@wendykan - I saw your email, I've just been a bit preoccupied) but I did identify one error in my work:</p>\n\n<p>The classType's are 1-indexed in the sample submission, that is they run from 1-10, whereas my code more naturally output 0-indexed classType's (i.e. from 0-9). It's a simple fix, although I haven't tested it yet. Maybe this is our problem?</p>\n\n<p>Let me know if this helps you!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151386,
      "author_name": "zfturbo",
      "author_url": "",
      "post_date": "12/20/2016 14:25:30",
      "content": "<p><strong>grantbey</strong> - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.</p>\n\n<p>Error in submission evaluation should explain why TOP of leaderboard stays the same for days.</p>",
      "votes": null,
      "replies": [
        {
          "id": 151387,
          "author_name": "grantbey",
          "author_url": "",
          "post_date": "12/20/2016 14:32:05",
          "content": "<p>[quote=ZFTurbo;151386]</p>\n\n<p><strong>grantbey</strong> - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.</p>\n\n<p>Error in submission evaluation should explain why TOP of leaderboard stays the same for days.</p>\n\n<p>[/quote]</p>\n\n<p>Damn, I was hoping this would be the solution. As I said, I haven't tried to submit for a while now. This competition seems fairly quiet still, so it's possible that not many people are submitting yet. One thinks if others were having this problem then the forums would reflect that. So far, it's you and me!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 151424,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "12/20/2016 19:03:35",
      "content": "<p>@ZFTurbo,</p>\n\n<p>If you sort the submission file to be in the exact same order of ImageId, ClassType as the sample_submission.csv, it should work. </p>\n\n<p>It is caused by our evaluation infrastructure only recognizing the first column as the \"key\", so it's unable to match two columns as the keys for submission files. </p>\n\n<p>Let me know if that works for you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 151427,
      "author_name": "zfturbo",
      "author_url": "",
      "post_date": "12/20/2016 19:51:42",
      "content": "<p><strong>@Wendy Kan</strong> It solves the problem. But now I have the following error:</p>\n\n<pre><code>Evaluation Exception: For image 6150_3_4, class 5, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: no outgoing dirEdge found [ (1.79996078946E-05, -0.0073208405217139, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution)..\n</code></pre>\n\n<p>The problem is: there is no NaNs in my submission. I don't have numbers mentioned in Exception for line: 6150_3_4,5,</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "151136": "There are 21 images in the original train_wkt.csv, 22 in the new train_wkt_v2.csv and 25 in the newest train_wkt_v3.csv.\r\n\r\nThis corresponds to 4290, 4280 and 4250 rows in the submission file, respectively. The submission will only accept the former, even though the latter two have at some point this weekend been the more correct versions.\r\n\r\nAdditionally, even if I submit with 4290 rows by excluding the imageIDs from train_wkt.csv from my submission, I get the following error:\r\n\r\n    Evaluation Exception: The given key was not present in the dictionary..\r\n\r\nWhich I presume means that one of the imageIDs shouldn't be present?\r\n\r\nSome clarification would be helpful.",
    "151263": "grantbey,\r\n\r\nThe submission file/format hasn't changed, but 4 images (the ones that we moved from test to train) have been ignored. Therefore, all you need to do is to submit the same format as *sample_submission.csv* and you should be good to go. For those 4 images, whatever you submit won't matter, so it's easiest to keep them empty \"MULTIPOLYGON EMPTY\".",
    "151265": "Thanks for the reply @wendykan\r\n\r\nThis is what I'm doing, however I still get the error mentioned above:\r\n\r\n    Evaluation Exception: The given key was not present in the dictionary..\r\n\r\nCan you give some insight into what this means so that I might be able to correct it?",
    "151267": "(I've taken it off the forum to help @grantbey)",
    "151343": "I have the same problem. I checked my submission against sample_submission.csv and couldn't find the problem. All ImageId and ClassType seemed to be present.",
    "151384": "Hey @zfturbo, I haven't tried recently (@wendykan - I saw your email, I've just been a bit preoccupied) but I did identify one error in my work:\r\n\r\nThe classType's are 1-indexed in the sample submission, that is they run from 1-10, whereas my code more naturally output 0-indexed classType's (i.e. from 0-9). It's a simple fix, although I haven't tested it yet. Maybe this is our problem?\r\n\r\nLet me know if this helps you!",
    "151386": "**grantbey** - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.\r\n\r\nError in submission evaluation should explain why TOP of leaderboard stays the same for days.",
    "151387": "[quote=ZFTurbo;151386]\r\n\r\n**grantbey** - I have absolutely the same output as in sample_submission except real polygons added. It should work or at least some other error description happen.\r\n\r\nError in submission evaluation should explain why TOP of leaderboard stays the same for days.\r\n\r\n[/quote]\r\n\r\nDamn, I was hoping this would be the solution. As I said, I haven't tried to submit for a while now. This competition seems fairly quiet still, so it's possible that not many people are submitting yet. One thinks if others were having this problem then the forums would reflect that. So far, it's you and me!",
    "151424": "ZFTurbo,\r\n\r\nIf you sort the submission file to be in the exact same order of ImageId, ClassType as the sample_submission.csv, it should work. \r\n\r\nIt is caused by our evaluation infrastructure only recognizing the first column as the \"key\", so it's unable to match two columns as the keys for submission files. \r\n\r\nLet me know if that works for you.",
    "151427": "**@Wendy Kan** It solves the problem. But now I have the following error:\r\n\r\n    Evaluation Exception: For image 6150_3_4, class 5, there's an exception in geometry NetTopologySuite.Geometries.TopologyException: no outgoing dirEdge found [ (1.79996078946E-05, -0.0073208405217139, NaN) ] at Kaggle.Metrics.Custom.JaccardDSTL.compareMultipolygon(IGeometry predMultipoly, IGeometry truthMultipoly) at Kaggle.Metrics.Custom.JaccardDSTL.calculateIoUVector(Dictionary`2 submission, Dictionary`2 solution)..\r\n\r\nThe problem is: there is no NaNs in my submission. I don't have numbers mentioned in Exception for line: 6150_3_4,5,"
  },
  "source": "meta"
}