{
  "id": 98369,
  "title": "Scored 0.000 ???",
  "url": "/competitions/aptos2019-blindness-detection/discussion/98369",
  "author_name": "",
  "post_date": "2019-07-03T09:32:29.306903900Z",
  "votes": 8,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I got submission error as reported in discussion.</p>\n\n<p>So, I did below steps to check my score bleafly.</p>\n\n<ol>\n<li>Uploaded weights to dataset</li>\n<li>Launched new Kernel and loaded weights</li>\n<li>Inferenced and submitted</li>\n</ol>\n\n<p>I thought do above steps tell me my score.</p>\n\n<p>Surprisingly, the score was 0.000 !!!</p>\n\n<p>What happened, thank you for your help..</p>",
  "messages": [
    {
      "id": "567278",
      "postDate": "07/03/2019 09:32:29",
      "content": "<p>I got submission error as reported in discussion.</p>\n\n<p>So, I did below steps to check my score bleafly.</p>\n\n<ol>\n<li>Uploaded weights to dataset</li>\n<li>Launched new Kernel and loaded weights</li>\n<li>Inferenced and submitted</li>\n</ol>\n\n<p>I thought do above steps tell me my score.</p>\n\n<p>Surprisingly, the score was 0.000 !!!</p>\n\n<p>What happened, thank you for your help..</p>",
      "rawMarkdown": "I got submission error as reported in discussion.\n\nSo, I did below steps to check my score bleafly.\n\n1. Uploaded weights to dataset\n2. Launched new Kernel and loaded weights\n3. Inferenced and submitted\n\nI thought do above steps tell me my score.\n\nSurprisingly, the score was 0.000 !!!\n\nWhat happened, thank you for your help..",
      "votes": null
    },
    {
      "id": "567305",
      "postDate": "07/03/2019 10:19:05",
      "content": "<p>Kappa will be zero if you predict a constant for all images. </p>\n\n<p>Do you initialize your vector of predictions as zeroes? If so, there might be an error occuring during the inference process that includes the private test data (e.g. not enough memory), which could lead to submitting the original vector of zeroes and getting a score of 0.</p>",
      "rawMarkdown": "Kappa will be zero if you predict a constant for all images. \n\nDo you initialize your vector of predictions as zeroes? If so, there might be an error occuring during the inference process that includes the private test data (e.g. not enough memory), which could lead to submitting the original vector of zeroes and getting a score of 0.",
      "votes": null
    },
    {
      "id": "567307",
      "postDate": "07/03/2019 10:21:14",
      "content": "<p>If you use Pytorch, do you do:\n<code>for param in model.parameters():\n    param.requires_grad = False</code>\nand\n<code>model.eval()</code>\nafter loading weights?</p>",
      "rawMarkdown": "If you use Pytorch, do you do:\n`for param in model.parameters():\n    param.requires_grad = False`\nand\n`model.eval()`\nafter loading weights?",
      "votes": null
    },
    {
      "id": "567377",
      "postDate": "07/03/2019 12:20:29",
      "content": "<p><a href=\"/bohdansafoniuk\">@bohdansafoniuk</a>, <a href=\"/kozodoi\">@kozodoi</a> </p>\n\n<p>Thank you for your reply.\nI tried all. But not improved....</p>",
      "rawMarkdown": "bohdansafoniuk, @kozodoi \n\nThank you for your reply.\nI tried all. But not improved....",
      "votes": null
    },
    {
      "id": "567555",
      "postDate": "07/03/2019 17:25:12",
      "content": "<p>The sample submission has a score of 0.00. Maybe check that you aren't accidentally uploading the sample submission.</p>",
      "rawMarkdown": "The sample submission has a score of 0.00. Maybe check that you aren't accidentally uploading the sample submission.",
      "votes": null
    },
    {
      "id": "567976",
      "postDate": "07/04/2019 07:59:52",
      "content": "<p>I have similar situation when trying to separate training and inference stages. Pretrained fastai models have score -0.001, there are no explicit errors in output.</p>\n\n<p>Still wondering why this happens.</p>",
      "rawMarkdown": "I have similar situation when trying to separate training and inference stages. Pretrained fastai models have score -0.001, there are no explicit errors in output.\n\nStill wondering why this happens.",
      "votes": null
    },
    {
      "id": "567980",
      "postDate": "07/04/2019 08:12:35",
      "content": "<p>I faced a similar issue and mentioned it somewhere on some discussion and as per a user suggestion, simply creating a new script/notebook out of the existing code did solve the problem. I could not have believed it but he was right. You should try it too.</p>",
      "rawMarkdown": "I faced a similar issue and mentioned it somewhere on some discussion and as per a user suggestion, simply creating a new script/notebook out of the existing code did solve the problem. I could not have believed it but he was right. You should try it too.",
      "votes": null
    },
    {
      "id": "568226",
      "postDate": "07/04/2019 15:05:52",
      "content": "<p>Have same issue (my prediction non constant and i'm using pure pytorch)</p>",
      "rawMarkdown": "Have same issue (my prediction non constant and i'm using pure pytorch)",
      "votes": null
    },
    {
      "id": "570403",
      "postDate": "07/08/2019 08:34:39",
      "content": "<p>Make sure your weights have been loaded successfully.\nI had a similar issue when I updated the dataset so that weights had different number of layers.\nCommit was using the new weights, but the private rerun seemed to use the old ones.\nRemoving the dataset from kernel and adding it again helped.</p>",
      "rawMarkdown": "Make sure your weights have been loaded successfully.\nI had a similar issue when I updated the dataset so that weights had different number of layers.\nCommit was using the new weights, but the private rerun seemed to use the old ones.\nRemoving the dataset from kernel and adding it again helped.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 567305,
      "author_name": "kozodoi",
      "author_url": "",
      "post_date": "07/03/2019 10:19:05",
      "content": "<p>Kappa will be zero if you predict a constant for all images. </p>\n\n<p>Do you initialize your vector of predictions as zeroes? If so, there might be an error occuring during the inference process that includes the private test data (e.g. not enough memory), which could lead to submitting the original vector of zeroes and getting a score of 0.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 567307,
      "author_name": "bohdansafoniuk",
      "author_url": "",
      "post_date": "07/03/2019 10:21:14",
      "content": "<p>If you use Pytorch, do you do:\n<code>for param in model.parameters():\n    param.requires_grad = False</code>\nand\n<code>model.eval()</code>\nafter loading weights?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 567377,
      "author_name": "kaerunantoka",
      "author_url": "",
      "post_date": "07/03/2019 12:20:29",
      "content": "<p><a href=\"/bohdansafoniuk\">@bohdansafoniuk</a>, <a href=\"/kozodoi\">@kozodoi</a> </p>\n\n<p>Thank you for your reply.\nI tried all. But not improved....</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 567555,
      "author_name": "jbridgeliverpool",
      "author_url": "",
      "post_date": "07/03/2019 17:25:12",
      "content": "<p>The sample submission has a score of 0.00. Maybe check that you aren't accidentally uploading the sample submission.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 567976,
      "author_name": "safonau",
      "author_url": "",
      "post_date": "07/04/2019 07:59:52",
      "content": "<p>I have similar situation when trying to separate training and inference stages. Pretrained fastai models have score -0.001, there are no explicit errors in output.</p>\n\n<p>Still wondering why this happens.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 567980,
      "author_name": "thanatoz",
      "author_url": "",
      "post_date": "07/04/2019 08:12:35",
      "content": "<p>I faced a similar issue and mentioned it somewhere on some discussion and as per a user suggestion, simply creating a new script/notebook out of the existing code did solve the problem. I could not have believed it but he was right. You should try it too.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 568226,
      "author_name": "datadakhma",
      "author_url": "",
      "post_date": "07/04/2019 15:05:52",
      "content": "<p>Have same issue (my prediction non constant and i'm using pure pytorch)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 570403,
      "author_name": "vzaguskin",
      "author_url": "",
      "post_date": "07/08/2019 08:34:39",
      "content": "<p>Make sure your weights have been loaded successfully.\nI had a similar issue when I updated the dataset so that weights had different number of layers.\nCommit was using the new weights, but the private rerun seemed to use the old ones.\nRemoving the dataset from kernel and adding it again helped.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "567278": "I got submission error as reported in discussion.\n\nSo, I did below steps to check my score bleafly.\n\n1. Uploaded weights to dataset\n2. Launched new Kernel and loaded weights\n3. Inferenced and submitted\n\nI thought do above steps tell me my score.\n\nSurprisingly, the score was 0.000 !!!\n\nWhat happened, thank you for your help..",
    "567305": "Kappa will be zero if you predict a constant for all images. \n\nDo you initialize your vector of predictions as zeroes? If so, there might be an error occuring during the inference process that includes the private test data (e.g. not enough memory), which could lead to submitting the original vector of zeroes and getting a score of 0.",
    "567307": "If you use Pytorch, do you do:\n`for param in model.parameters():\n    param.requires_grad = False`\nand\n`model.eval()`\nafter loading weights?",
    "567377": "bohdansafoniuk, @kozodoi \n\nThank you for your reply.\nI tried all. But not improved....",
    "567555": "The sample submission has a score of 0.00. Maybe check that you aren't accidentally uploading the sample submission.",
    "567976": "I have similar situation when trying to separate training and inference stages. Pretrained fastai models have score -0.001, there are no explicit errors in output.\n\nStill wondering why this happens.",
    "567980": "I faced a similar issue and mentioned it somewhere on some discussion and as per a user suggestion, simply creating a new script/notebook out of the existing code did solve the problem. I could not have believed it but he was right. You should try it too.",
    "568226": "Have same issue (my prediction non constant and i'm using pure pytorch)",
    "570403": "Make sure your weights have been loaded successfully.\nI had a similar issue when I updated the dataset so that weights had different number of layers.\nCommit was using the new weights, but the private rerun seemed to use the old ones.\nRemoving the dataset from kernel and adding it again helped."
  },
  "source": "meta"
}