{
  "id": 297135,
  "title": "Faced the following error while training FasterRCNN ",
  "url": "/competitions/tensorflow-great-barrier-reef/discussion/297135",
  "author_name": "",
  "post_date": "2021-12-25T13:24:51.326685500Z",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p><code>Expected y_max for bbox (0.6953125, 0.9541666666666667, 0.7171875, 1.0013888888888889, tensor(1)) to be in the range [0.0, 1.0], got 1.0013888888888889\n</code></p>\n<p>I tried clipping off ,rounding off my y_max in the custom torch dataset pipeline but still there wasn't any help to resolve this issue Maybe there is some problem with Albumentations but I haven't tried object detection before. this error pops up in training loop</p>\n<p>I even tried something else as well , I tried the notebook by <a href=\"https://www.kaggle.com/julian3833\" target=\"_blank\">@julian3833</a> where he made a <a href=\"https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-416l\" target=\"_blank\">fasterrcnn baseline</a> the notebook works completely fine when you <code>Copy and Edit</code> it but when I tried implementing the same nb on with same code I ran into the problem of                 <code>Found dtype Double expected float</code> in the training loop, for debugging this issue </p>\n<p>I tried the following:</p>\n<p>1.moving my model to float, double <br>\n2.moving my target,images to float,double</p>\n<p>with both these approaches I ended up getting new errors and a lot of frustration(trying to implement this since a week :( </p>\n<p>Despite running the same code torch is running into this issue, any kind of help would be highly appereciated </p>",
  "messages": [
    {
      "id": "1628959",
      "postDate": "12/25/2021 13:24:51",
      "content": "<p><code>Expected y_max for bbox (0.6953125, 0.9541666666666667, 0.7171875, 1.0013888888888889, tensor(1)) to be in the range [0.0, 1.0], got 1.0013888888888889\n</code></p>\n<p>I tried clipping off ,rounding off my y_max in the custom torch dataset pipeline but still there wasn't any help to resolve this issue Maybe there is some problem with Albumentations but I haven't tried object detection before. this error pops up in training loop</p>\n<p>I even tried something else as well , I tried the notebook by <a href=\"https://www.kaggle.com/julian3833\" target=\"_blank\">@julian3833</a> where he made a <a href=\"https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-416l\" target=\"_blank\">fasterrcnn baseline</a> the notebook works completely fine when you <code>Copy and Edit</code> it but when I tried implementing the same nb on with same code I ran into the problem of                 <code>Found dtype Double expected float</code> in the training loop, for debugging this issue </p>\n<p>I tried the following:</p>\n<p>1.moving my model to float, double <br>\n2.moving my target,images to float,double</p>\n<p>with both these approaches I ended up getting new errors and a lot of frustration(trying to implement this since a week :( </p>\n<p>Despite running the same code torch is running into this issue, any kind of help would be highly appereciated </p>",
      "rawMarkdown": "```Expected y_max for bbox (0.6953125, 0.9541666666666667, 0.7171875, 1.0013888888888889, tensor(1)) to be in the range [0.0, 1.0], got 1.0013888888888889\n```\n\n\nI tried clipping off ,rounding off my y_max in the custom torch dataset pipeline but still there wasn't any help to resolve this issue Maybe there is some problem with Albumentations but I haven't tried object detection before. this error pops up in training loop\n\nI even tried something else as well , I tried the notebook by @julian3833 where he made a [fasterrcnn baseline](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-416l) the notebook works completely fine when you ```Copy and Edit``` it but when I tried implementing the same nb on with same code I ran into the problem of                 ```Found dtype Double expected float``` in the training loop, for debugging this issue \n\n\nI tried the following:\n\n1.moving my model to float, double \n2.moving my target,images to float,double\n\nwith both these approaches I ended up getting new errors and a lot of frustration(trying to implement this since a week :( \n\nDespite running the same code torch is running into this issue, any kind of help would be highly appereciated",
      "votes": null
    },
    {
      "id": "1629092",
      "postDate": "12/25/2021 17:10:41",
      "content": "<p>This is Albumentations message. It means that your one bbox coordinate is outside the image - ymax (it depends on notation) 1.0013888888888889. There are many reasons …:</p>\n<ol>\n<li>We know that some bboxes in training dataset are outside image (you have to check it during annotation creation and fix this issue) </li>\n<li>In your Albumentation pipeline you have bboxes with 0 width or height (eg. you transformed it incorrectly earlier)</li>\n<li>Annotations were created not correctly</li>\n<li>You provide not correct annotation format to Albumentations bbox_params=A.BboxParams(format='coco') - for format see here: <a href=\"https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/\" target=\"_blank\">https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/</a></li>\n</ol>\n<p>I suppose you have not cleaned oryginal dataset bboxes which are outside frames.</p>",
      "rawMarkdown": "This is Albumentations message. It means that your one bbox coordinate is outside the image - ymax (it depends on notation) 1.0013888888888889. There are many reasons ...:\n1. We know that some bboxes in training dataset are outside image (you have to check it during annotation creation and fix this issue) \n2. In your Albumentation pipeline you have bboxes with 0 width or height (eg. you transformed it incorrectly earlier)\n3. Annotations were created not correctly\n4. You provide not correct annotation format to Albumentations bbox_params=A.BboxParams(format='coco') - for format see here: https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/\n\nI suppose you have not cleaned oryginal dataset bboxes which are outside frames.",
      "votes": null
    },
    {
      "id": "1629737",
      "postDate": "12/26/2021 13:46:28",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> for pointing that out can you tell me do I have to use COCO format or Pascal format for this dataset.</p>\n<p>And are there any solutions on how can I check if bboxes are outside the images</p>",
      "rawMarkdown": "Thanks @remekkinas for pointing that out can you tell me do I have to use COCO format or Pascal format for this dataset.\n\nAnd are there any solutions on how can I check if bboxes are outside the images",
      "votes": null
    },
    {
      "id": "1629751",
      "postDate": "12/26/2021 14:02:17",
      "content": "<p>I am sure they are … <br>\nJust in the dataloader implement code like this</p>\n<pre><code>  if (bbox[0] + bbox[2] &gt; 1280):\n                b_width = bbox[0] - 1280 \n            if (bbox[1] + bbox[3] &gt; 720):\n                b_height = bbox[1] - 720\n</code></pre>\n<p>Or just instead correction raise error.</p>",
      "rawMarkdown": "I am sure they are … \nJust in the dataloader implement code like this\n  ```\n  if (bbox[0] + bbox[2] > 1280):\n                b_width = bbox[0] - 1280 \n            if (bbox[1] + bbox[3] > 720):\n                b_height = bbox[1] - 720\n```\n\nOr just instead correction raise error.",
      "votes": null
    },
    {
      "id": "1631934",
      "postDate": "12/29/2021 00:06:07",
      "content": "<p>Thanks for the help <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> but it seems that there was some prevalent issues with dtypes in the dataframe ,</p>\n<p>after 3 days of debugging I found that and changed two lines of code </p>\n<pre><code>images = list(image.float().to(device) for image in images)\n        targets = [{k: v.to(torch.float32).to(device) if \"box\" in k else v.to(device) for k, v in t.items()} for t in targets]\n</code></pre>\n<p>the above code resolved my issue , anyways thanks a ton for your help anyways I ended up cleaning and exploring my dataframe from your valuable input</p>",
      "rawMarkdown": "Thanks for the help @remekkinas but it seems that there was some prevalent issues with dtypes in the dataframe ,\n\nafter 3 days of debugging I found that and changed two lines of code \n\n```\nimages = list(image.float().to(device) for image in images)\n        targets = [{k: v.to(torch.float32).to(device) if \"box\" in k else v.to(device) for k, v in t.items()} for t in targets]\n        \n```\n\nthe above code resolved my issue , anyways thanks a ton for your help anyways I ended up cleaning and exploring my dataframe from your valuable input",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1629092,
      "author_name": "remekkinas",
      "author_url": "",
      "post_date": "12/25/2021 17:10:41",
      "content": "<p>This is Albumentations message. It means that your one bbox coordinate is outside the image - ymax (it depends on notation) 1.0013888888888889. There are many reasons …:</p>\n<ol>\n<li>We know that some bboxes in training dataset are outside image (you have to check it during annotation creation and fix this issue) </li>\n<li>In your Albumentation pipeline you have bboxes with 0 width or height (eg. you transformed it incorrectly earlier)</li>\n<li>Annotations were created not correctly</li>\n<li>You provide not correct annotation format to Albumentations bbox_params=A.BboxParams(format='coco') - for format see here: <a href=\"https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/\" target=\"_blank\">https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/</a></li>\n</ol>\n<p>I suppose you have not cleaned oryginal dataset bboxes which are outside frames.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1629737,
          "author_name": "harshris21",
          "author_url": "",
          "post_date": "12/26/2021 13:46:28",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> for pointing that out can you tell me do I have to use COCO format or Pascal format for this dataset.</p>\n<p>And are there any solutions on how can I check if bboxes are outside the images</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1629751,
          "author_name": "remekkinas",
          "author_url": "",
          "post_date": "12/26/2021 14:02:17",
          "content": "<p>I am sure they are … <br>\nJust in the dataloader implement code like this</p>\n<pre><code>  if (bbox[0] + bbox[2] &gt; 1280):\n                b_width = bbox[0] - 1280 \n            if (bbox[1] + bbox[3] &gt; 720):\n                b_height = bbox[1] - 720\n</code></pre>\n<p>Or just instead correction raise error.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1631934,
          "author_name": "harshris21",
          "author_url": "",
          "post_date": "12/29/2021 00:06:07",
          "content": "<p>Thanks for the help <a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> but it seems that there was some prevalent issues with dtypes in the dataframe ,</p>\n<p>after 3 days of debugging I found that and changed two lines of code </p>\n<pre><code>images = list(image.float().to(device) for image in images)\n        targets = [{k: v.to(torch.float32).to(device) if \"box\" in k else v.to(device) for k, v in t.items()} for t in targets]\n</code></pre>\n<p>the above code resolved my issue , anyways thanks a ton for your help anyways I ended up cleaning and exploring my dataframe from your valuable input</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1628959": "```Expected y_max for bbox (0.6953125, 0.9541666666666667, 0.7171875, 1.0013888888888889, tensor(1)) to be in the range [0.0, 1.0], got 1.0013888888888889\n```\n\n\nI tried clipping off ,rounding off my y_max in the custom torch dataset pipeline but still there wasn't any help to resolve this issue Maybe there is some problem with Albumentations but I haven't tried object detection before. this error pops up in training loop\n\nI even tried something else as well , I tried the notebook by @julian3833 where he made a [fasterrcnn baseline](https://www.kaggle.com/julian3833/reef-starter-torch-fasterrcnn-train-lb-0-416l) the notebook works completely fine when you ```Copy and Edit``` it but when I tried implementing the same nb on with same code I ran into the problem of                 ```Found dtype Double expected float``` in the training loop, for debugging this issue \n\n\nI tried the following:\n\n1.moving my model to float, double \n2.moving my target,images to float,double\n\nwith both these approaches I ended up getting new errors and a lot of frustration(trying to implement this since a week :( \n\nDespite running the same code torch is running into this issue, any kind of help would be highly appereciated",
    "1629092": "This is Albumentations message. It means that your one bbox coordinate is outside the image - ymax (it depends on notation) 1.0013888888888889. There are many reasons ...:\n1. We know that some bboxes in training dataset are outside image (you have to check it during annotation creation and fix this issue) \n2. In your Albumentation pipeline you have bboxes with 0 width or height (eg. you transformed it incorrectly earlier)\n3. Annotations were created not correctly\n4. You provide not correct annotation format to Albumentations bbox_params=A.BboxParams(format='coco') - for format see here: https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/\n\nI suppose you have not cleaned oryginal dataset bboxes which are outside frames.",
    "1629737": "Thanks @remekkinas for pointing that out can you tell me do I have to use COCO format or Pascal format for this dataset.\n\nAnd are there any solutions on how can I check if bboxes are outside the images",
    "1629751": "I am sure they are … \nJust in the dataloader implement code like this\n  ```\n  if (bbox[0] + bbox[2] > 1280):\n                b_width = bbox[0] - 1280 \n            if (bbox[1] + bbox[3] > 720):\n                b_height = bbox[1] - 720\n```\n\nOr just instead correction raise error.",
    "1631934": "Thanks for the help @remekkinas but it seems that there was some prevalent issues with dtypes in the dataframe ,\n\nafter 3 days of debugging I found that and changed two lines of code \n\n```\nimages = list(image.float().to(device) for image in images)\n        targets = [{k: v.to(torch.float32).to(device) if \"box\" in k else v.to(device) for k, v in t.items()} for t in targets]\n        \n```\n\nthe above code resolved my issue , anyways thanks a ton for your help anyways I ended up cleaning and exploring my dataframe from your valuable input"
  },
  "source": "meta"
}