{
  "id": 58219,
  "title": "Score decreased from 0.172 to 0.06 just has changed head of csv",
  "url": "/competitions/cvpr-2018-autonomous-driving/discussion/58219",
  "author_name": "Hang Yang",
  "post_date": "2018-06-04T15:29:51.862000",
  "votes": 0,
  "comment_count": 9,
  "views": 0,
  "content": "<p>I have <strong>just changed</strong> the head of my submission csv file from <br>\"ImageId,LabelId,PixelCount,Confidence,EncodedPixels\" to <br>\"ImageId,LabelId,Confidence,PixelCount,EncodedPixels\"<br>\nbut the score has changed from 0.172 to 0.06<br>\nCan anyone tell me the reason ? </p>",
  "messages": [
    {
      "id": 339028,
      "postDate": "2018-06-06T07:04:52.223Z",
      "content": "<p>you can alse sent me your submission: <a href=\"https://www.kaggle.com/c/cvpr-2018-autonomous-driving/discussion/58329\">https://www.kaggle.com/c/cvpr-2018-autonomous-driving/discussion/58329</a></p>",
      "rawMarkdown": "you can alse sent me your submission: https://www.kaggle.com/c/cvpr-2018-autonomous-driving/discussion/58329",
      "votes": 1,
      "replies": [
        {
          "id": 339495,
          "postDate": "2018-06-07T03:34:29.733Z",
          "content": "<p>你好，我已经通过qq邮箱给你发送了我的csv结果文件（zip），我这边上传是报错了，但是格式是按照你们的格式来的，希望能帮忙看一下bug出在我这边还是你们那边<br>\n万分感谢！</p>",
          "rawMarkdown": "你好，我已经通过qq邮箱给你发送了我的csv结果文件（zip），我这边上传是报错了，但是格式是按照你们的格式来的，希望能帮忙看一下bug出在我这边还是你们那边<br>\n万分感谢！"
        }
      ]
    },
    {
      "id": 340324,
      "postDate": "2018-06-08T22:46:24.420Z",
      "content": "<p>So the evaluation code assumes the third column is confidence but not pixCount. Initially, my third column was pixCount, I corrected that and resubmitted but no difference was observed in the score. Is the evaluation code working in an expected way?</p>",
      "rawMarkdown": "So the evaluation code assumes the third column is confidence but not pixCount. Initially, my third column was pixCount, I corrected that and resubmitted but no difference was observed in the score. Is the evaluation code working in an expected way?"
    },
    {
      "id": 340136,
      "postDate": "2018-06-08T12:49:57.553Z",
      "content": "<p>it because after you change the order of column, program will use confidental as PixelCount.\nand \"overlap = pred.intersection / (gt.pixelCount + pred.pixelCount - pred.intersection);\"\nIt means here pred.pixelCount almost = 0.  so that overlap will be much higher than it should be. </p>\n\n<p>I will change code to let program count pixelCount by itself, instead of using user's. \nThank you so much for your report</p>",
      "rawMarkdown": "it because after you change the order of column, program will use confidental as PixelCount.\nand \"overlap = pred.intersection / (gt.pixelCount + pred.pixelCount - pred.intersection);\"\nIt means here pred.pixelCount almost = 0.  so that overlap will be much higher than it should be. \n\nI will change code to let program count pixelCount by itself, instead of using user's. \nThank you so much for your report"
    },
    {
      "id": 339024,
      "postDate": "2018-06-06T07:02:50.630Z",
      "content": "<p>the order should be: ImageId    LabelId Confidence  PixelCount  EncodedPixels</p>",
      "rawMarkdown": "the order should be: ImageId\tLabelId\tConfidence\tPixelCount\tEncodedPixels",
      "replies": [
        {
          "id": 339924,
          "postDate": "2018-06-08T00:56:46.007Z",
          "content": "<p>Do the order of the columns matter to the evaluation?</p>",
          "rawMarkdown": "Do the order of the columns matter to the evaluation?"
        },
        {
          "id": 339974,
          "postDate": "2018-06-08T03:53:34.967Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 339977,
          "postDate": "2018-06-08T04:21:54.257Z",
          "content": "<p>That's very interesting…</p>",
          "rawMarkdown": "That's very interesting…"
        },
        {
          "id": 339979,
          "postDate": "2018-06-08T04:32:01.260Z",
          "rawMarkdown": "",
          "isDeleted": true
        }
      ]
    },
    {
      "id": 338179,
      "postDate": "2018-06-04T15:29:51.863Z",
      "content": "<p>I have <strong>just changed</strong> the head of my submission csv file from <br>\"ImageId,LabelId,PixelCount,Confidence,EncodedPixels\" to <br>\"ImageId,LabelId,Confidence,PixelCount,EncodedPixels\"<br>\nbut the score has changed from 0.172 to 0.06<br>\nCan anyone tell me the reason ? </p>",
      "rawMarkdown": "I have **just changed** the head of my submission csv file from <br>\"ImageId,LabelId,PixelCount,Confidence,EncodedPixels\" to <br>\"ImageId,LabelId,Confidence,PixelCount,EncodedPixels\"<br>\nbut the score has changed from 0.172 to 0.06<br>\nCan anyone tell me the reason ? \n"
    }
  ],
  "comments": [
    {
      "id": 339028,
      "author_name": "Charles Shao",
      "author_url": "",
      "post_date": "2018-06-06T07:04:52.223000",
      "content": "<p>you can alse sent me your submission: <a href=\"https://www.kaggle.com/c/cvpr-2018-autonomous-driving/discussion/58329\">https://www.kaggle.com/c/cvpr-2018-autonomous-driving/discussion/58329</a></p>",
      "votes": 1,
      "replies": [
        {
          "id": 339495,
          "author_name": "Hang Yang",
          "author_url": "",
          "post_date": "2018-06-07T03:34:29.733000",
          "content": "<p>你好，我已经通过qq邮箱给你发送了我的csv结果文件（zip），我这边上传是报错了，但是格式是按照你们的格式来的，希望能帮忙看一下bug出在我这边还是你们那边<br>\n万分感谢！</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 340324,
      "author_name": "XiaokangWang",
      "author_url": "",
      "post_date": "2018-06-08T22:46:24.420000",
      "content": "<p>So the evaluation code assumes the third column is confidence but not pixCount. Initially, my third column was pixCount, I corrected that and resubmitted but no difference was observed in the score. Is the evaluation code working in an expected way?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 340136,
      "author_name": "Charles Shao",
      "author_url": "",
      "post_date": "2018-06-08T12:49:57.553000",
      "content": "<p>it because after you change the order of column, program will use confidental as PixelCount.\nand \"overlap = pred.intersection / (gt.pixelCount + pred.pixelCount - pred.intersection);\"\nIt means here pred.pixelCount almost = 0.  so that overlap will be much higher than it should be. </p>\n\n<p>I will change code to let program count pixelCount by itself, instead of using user's. \nThank you so much for your report</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 339024,
      "author_name": "Charles Shao",
      "author_url": "",
      "post_date": "2018-06-06T07:02:50.630000",
      "content": "<p>the order should be: ImageId    LabelId Confidence  PixelCount  EncodedPixels</p>",
      "votes": 0,
      "replies": [
        {
          "id": 339924,
          "author_name": "XiaokangWang",
          "author_url": "",
          "post_date": "2018-06-08T00:56:46.007000",
          "content": "<p>Do the order of the columns matter to the evaluation?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 339974,
          "author_name": "",
          "author_url": "",
          "post_date": "2018-06-08T03:53:34.967000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 339977,
          "author_name": "Wudi Wang",
          "author_url": "",
          "post_date": "2018-06-08T04:21:54.257000",
          "content": "<p>That's very interesting…</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 339979,
          "author_name": "",
          "author_url": "",
          "post_date": "2018-06-08T04:32:01.260000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "339028": "you can alse sent me your submission: https://www.kaggle.com/c/cvpr-2018-autonomous-driving/discussion/58329",
    "340324": "So the evaluation code assumes the third column is confidence but not pixCount. Initially, my third column was pixCount, I corrected that and resubmitted but no difference was observed in the score. Is the evaluation code working in an expected way?",
    "340136": "it because after you change the order of column, program will use confidental as PixelCount.\nand \"overlap = pred.intersection / (gt.pixelCount + pred.pixelCount - pred.intersection);\"\nIt means here pred.pixelCount almost = 0.  so that overlap will be much higher than it should be. \n\nI will change code to let program count pixelCount by itself, instead of using user's. \nThank you so much for your report",
    "339024": "the order should be: ImageId\tLabelId\tConfidence\tPixelCount\tEncodedPixels",
    "338179": "I have **just changed** the head of my submission csv file from <br>\"ImageId,LabelId,PixelCount,Confidence,EncodedPixels\" to <br>\"ImageId,LabelId,Confidence,PixelCount,EncodedPixels\"<br>\nbut the score has changed from 0.172 to 0.06<br>\nCan anyone tell me the reason ? \n"
  }
}