{
  "id": 129668,
  "title": "How I solved my Submission error..",
  "url": "/competitions/bengaliai-cv19/discussion/129668",
  "author_name": "",
  "post_date": "2020-02-09T22:35:03.397960600Z",
  "votes": 4,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I am writing this to save time for others, there are 2 hidden reasons that could cause these nasty errors although the commit will succeed on training and test parquets\n1. In the real inference, it seems there is a very faint image that raises an error if you used the pre-processing in <a href=\"https://www.kaggle.com/iafoss/image-preprocessing-128x128\">image-preprocessing-128x128</a> without first normalizing the photo.\n<code>img = (img0*(255.0/img0.max())).astype(np.uint8)</code></p>\n\n<ol>\n<li>inference parquets <em>possibly</em> contain different number of images, for the public train/test parquets, both have equal number images in each parquet making the use of various numpy array tricks possible but in inference you can not as it results in  jagged array.\nmy issue is that I used <code>np.ravel()</code> to flatten a list of lists. to flatten it, I had to stack arrays of apparently inequal lengths which resulted in the error.</li>\n</ol>",
  "messages": [
    {
      "id": "740870",
      "postDate": "02/09/2020 22:35:03",
      "content": "<p>I am writing this to save time for others, there are 2 hidden reasons that could cause these nasty errors although the commit will succeed on training and test parquets\n1. In the real inference, it seems there is a very faint image that raises an error if you used the pre-processing in <a href=\"https://www.kaggle.com/iafoss/image-preprocessing-128x128\">image-preprocessing-128x128</a> without first normalizing the photo.\n<code>img = (img0*(255.0/img0.max())).astype(np.uint8)</code></p>\n\n<ol>\n<li>inference parquets <em>possibly</em> contain different number of images, for the public train/test parquets, both have equal number images in each parquet making the use of various numpy array tricks possible but in inference you can not as it results in  jagged array.\nmy issue is that I used <code>np.ravel()</code> to flatten a list of lists. to flatten it, I had to stack arrays of apparently inequal lengths which resulted in the error.</li>\n</ol>",
      "rawMarkdown": "I am writing this to save time for others, there are 2 hidden reasons that could cause these nasty errors although the commit will succeed on training and test parquets\n1. In the real inference, it seems there is a very faint image that raises an error if you used the pre-processing in [image-preprocessing-128x128](https://www.kaggle.com/iafoss/image-preprocessing-128x128) without first normalizing the photo.\n`img = (img0*(255.0/img0.max())).astype(np.uint8)`\n\n2.  inference parquets *possibly* contain different number of images, for the public train/test parquets, both have equal number images in each parquet making the use of various numpy array tricks possible but in inference you can not as it results in  jagged array.\nmy issue is that I used `np.ravel()` to flatten a list of lists. to flatten it, I had to stack arrays of apparently inequal lengths which resulted in the error.",
      "votes": null
    },
    {
      "id": "740965",
      "postDate": "02/10/2020 03:01:39",
      "content": "<p>Thanks for sharing, really appreciate <a href=\"/ma7555\">@ma7555</a> !</p>",
      "rawMarkdown": "Thanks for sharing, really appreciate @ma7555 !",
      "votes": null
    },
    {
      "id": "741336",
      "postDate": "02/10/2020 13:52:57",
      "content": "<p>Many thanks.</p>",
      "rawMarkdown": "Many thanks.",
      "votes": null
    },
    {
      "id": "742179",
      "postDate": "02/11/2020 06:05:04",
      "content": "<p>Thanks . This saved me . <a href=\"/ma7555\">@ma7555</a> </p>",
      "rawMarkdown": "Thanks . This saved me . @ma7555",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 740965,
      "author_name": "rohitagarwal",
      "author_url": "",
      "post_date": "02/10/2020 03:01:39",
      "content": "<p>Thanks for sharing, really appreciate <a href=\"/ma7555\">@ma7555</a> !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 741336,
      "author_name": "shijinzhang",
      "author_url": "",
      "post_date": "02/10/2020 13:52:57",
      "content": "<p>Many thanks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 742179,
      "author_name": "virajbagal",
      "author_url": "",
      "post_date": "02/11/2020 06:05:04",
      "content": "<p>Thanks . This saved me . <a href=\"/ma7555\">@ma7555</a> </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "740870": "I am writing this to save time for others, there are 2 hidden reasons that could cause these nasty errors although the commit will succeed on training and test parquets\n1. In the real inference, it seems there is a very faint image that raises an error if you used the pre-processing in [image-preprocessing-128x128](https://www.kaggle.com/iafoss/image-preprocessing-128x128) without first normalizing the photo.\n`img = (img0*(255.0/img0.max())).astype(np.uint8)`\n\n2.  inference parquets *possibly* contain different number of images, for the public train/test parquets, both have equal number images in each parquet making the use of various numpy array tricks possible but in inference you can not as it results in  jagged array.\nmy issue is that I used `np.ravel()` to flatten a list of lists. to flatten it, I had to stack arrays of apparently inequal lengths which resulted in the error.",
    "740965": "Thanks for sharing, really appreciate @ma7555 !",
    "741336": "Many thanks.",
    "742179": "Thanks . This saved me . @ma7555"
  },
  "source": "meta"
}