{ "id": 207136, "title": "Normalizing Image twice?", "url": "/competitions/cassava-leaf-disease-classification/discussion/207136", "author_name": "", "post_date": "2020-12-28T12:03:43.219614400Z", "votes": 1, "comment_count": 5, "views": 0, "content": "

Edit: actually it is common practice… it’s my bad for asking such a question without doing my homework.

\n

I made a post on Cross Validated Exchange and no one answered yet. So I guess I may find some success here.

\n

I made a 'mistake' while training: In my Dataset class from PyTorch, I defined a flag

\n
    if self.transform_norm is False:\n        image = image.astype(np.float32) / 255.0\n
\n

and this would signify that if my augmentations pipeline does not have a normalization technique, then we set this flag to False. One example that I would set the above flag to be True is if normalization augmentation appear in my pipeline:

\n

albumentations.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0)

\n

I forgot to set the flag to True and thus, the images first went through a standardization from [0,255] to [0,1] and then normalized using mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]. I thought I did wrong but the training results were actually good. So I dug deeper and found PyTorch's documentation and realized that it may be ok to do so? However, I am not using PyTorch's pretrained model out of the box, usually, I go to Ross's timm/geffnet for the models. Do let me know if we should follow the rule that PyTorch mentioned that **'pretrained models from torchvision' ** that has been trained on images with values in [0, 1] and therefore expects the same format.

\n

To quote the link from PyTorch:

\n
\n

All pre-trained models expect input images normalized in the same way,
\n i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where
\n H and W are expected to be at least 224. The images have to be loaded
\n in to a range of [0, 1] and then normalized using mean = [0.485,
\n 0.456, 0.406] and std = [0.229, 0.224, 0.225].

\n
", "messages": [ { "id": "1129519", "postDate": "12/28/2020 12:03:43", "content": "

Edit: actually it is common practice… it’s my bad for asking such a question without doing my homework.

\n

I made a post on Cross Validated Exchange and no one answered yet. So I guess I may find some success here.

\n

I made a 'mistake' while training: In my Dataset class from PyTorch, I defined a flag

\n
    if self.transform_norm is False:\n        image = image.astype(np.float32) / 255.0\n
\n

and this would signify that if my augmentations pipeline does not have a normalization technique, then we set this flag to False. One example that I would set the above flag to be True is if normalization augmentation appear in my pipeline:

\n

albumentations.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0)

\n

I forgot to set the flag to True and thus, the images first went through a standardization from [0,255] to [0,1] and then normalized using mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]. I thought I did wrong but the training results were actually good. So I dug deeper and found PyTorch's documentation and realized that it may be ok to do so? However, I am not using PyTorch's pretrained model out of the box, usually, I go to Ross's timm/geffnet for the models. Do let me know if we should follow the rule that PyTorch mentioned that **'pretrained models from torchvision' ** that has been trained on images with values in [0, 1] and therefore expects the same format.

\n

To quote the link from PyTorch:

\n
\n

All pre-trained models expect input images normalized in the same way,
\n i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where
\n H and W are expected to be at least 224. The images have to be loaded
\n in to a range of [0, 1] and then normalized using mean = [0.485,
\n 0.456, 0.406] and std = [0.229, 0.224, 0.225].

\n
", "rawMarkdown": "Edit: actually it is common practice... it’s my bad for asking such a question without doing my homework. \n\nI made a post on [Cross Validated Exchange](https://stats.stackexchange.com/questions/502603/normalized-image-twice-by-accident-and-realised-it-may-be-the-correct-method) and no one answered yet. So I guess I may find some success here.\n\nI made a 'mistake' while training: In my `Dataset` class from `PyTorch`, I defined a flag \n\n```\n if self.transform_norm is False:\n image = image.astype(np.float32) / 255.0\n```\n\nand this would signify that if my augmentations pipeline does not have a normalization technique, then we set this flag to `False`. One example that I would set the above flag to be `True` is if normalization augmentation appear in my pipeline:\n\n` albumentations.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0)`\n\nI forgot to set the flag to `True` and thus, the images first went through a standardization from [0,255] to [0,1] and then normalized using `mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]`. I thought I did wrong but the training results were actually good. So I dug deeper and found [PyTorch's documentation](https://pytorch.org/docs/stable/torchvision/models.html) and realized that it may be ok to do so? However, I am not using `PyTorch's` pretrained model out of the box, usually, I go to [Ross's timm/geffnet](https://github.com/rwightman) for the models. Do let me know if we should follow the rule that PyTorch mentioned that **'pretrained models from torchvision' ** that has been trained on images with values in [0, 1] and therefore expects the same format.\n\nTo quote the link from PyTorch: \n\n> All pre-trained models expect input images normalized in the same way,\n> i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where\n> H and W are expected to be at least 224. The images have to be loaded\n> in to a range of [0, 1] and then normalized using mean = [0.485,\n> 0.456, 0.406] and std = [0.229, 0.224, 0.225].", "votes": null }, { "id": "1129813", "postDate": "12/28/2020 15:07:11", "content": "

This is actually standard, if you look in the docs of albumentations Normalize() divides each pixel by 255 and then applies the mean and std. So the values are between [0,1] and normalized to imagenet values

", "rawMarkdown": "This is actually standard, if you look in the docs of albumentations Normalize() divides each pixel by 255 and then applies the mean and std. So the values are between [0,1] and normalized to imagenet values", "votes": null }, { "id": "1129900", "postDate": "12/28/2020 16:17:29", "content": "

Oh dear. How could I have missed that on the source code documentation. You are right.

", "rawMarkdown": "Oh dear. How could I have missed that on the source code documentation. You are right.", "votes": null }, { "id": "1129932", "postDate": "12/28/2020 16:50:44", "content": "

Happens to the best, good luck chief!

", "rawMarkdown": "Happens to the best, good luck chief!", "votes": null }, { "id": "1130260", "postDate": "12/28/2020 21:59:21", "content": "

Far from it! Good luck amigos!

", "rawMarkdown": "Far from it! Good luck amigos!", "votes": null }, { "id": "1130850", "postDate": "12/29/2020 10:47:59", "content": "

albumentations Normalize checks for datatype of image and if it's float it subtracts mean and divides by std. If it's int it first divides image values by 255

", "rawMarkdown": "albumentations Normalize checks for datatype of image and if it's float it subtracts mean and divides by std. If it's int it first divides image values by 255", "votes": null } ], "comments": [ { "id": 1129813, "author_name": "yannmajewski", "author_url": "", "post_date": "12/28/2020 15:07:11", "content": "

This is actually standard, if you look in the docs of albumentations Normalize() divides each pixel by 255 and then applies the mean and std. So the values are between [0,1] and normalized to imagenet values

", "votes": null, "replies": [ { "id": 1129900, "author_name": "reighns", "author_url": "", "post_date": "12/28/2020 16:17:29", "content": "

Oh dear. How could I have missed that on the source code documentation. You are right.

", "votes": null, "replies": [] }, { "id": 1129932, "author_name": "yannmajewski", "author_url": "", "post_date": "12/28/2020 16:50:44", "content": "

Happens to the best, good luck chief!

", "votes": null, "replies": [] }, { "id": 1130260, "author_name": "reighns", "author_url": "", "post_date": "12/28/2020 21:59:21", "content": "

Far from it! Good luck amigos!

", "votes": null, "replies": [] }, { "id": 1130850, "author_name": "thepowerfuldeez", "author_url": "", "post_date": "12/29/2020 10:47:59", "content": "

albumentations Normalize checks for datatype of image and if it's float it subtracts mean and divides by std. If it's int it first divides image values by 255

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1129519": "Edit: actually it is common practice... it’s my bad for asking such a question without doing my homework. \n\nI made a post on [Cross Validated Exchange](https://stats.stackexchange.com/questions/502603/normalized-image-twice-by-accident-and-realised-it-may-be-the-correct-method) and no one answered yet. So I guess I may find some success here.\n\nI made a 'mistake' while training: In my `Dataset` class from `PyTorch`, I defined a flag \n\n```\n if self.transform_norm is False:\n image = image.astype(np.float32) / 255.0\n```\n\nand this would signify that if my augmentations pipeline does not have a normalization technique, then we set this flag to `False`. One example that I would set the above flag to be `True` is if normalization augmentation appear in my pipeline:\n\n` albumentations.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225], max_pixel_value=255.0, p=1.0)`\n\nI forgot to set the flag to `True` and thus, the images first went through a standardization from [0,255] to [0,1] and then normalized using `mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]`. I thought I did wrong but the training results were actually good. So I dug deeper and found [PyTorch's documentation](https://pytorch.org/docs/stable/torchvision/models.html) and realized that it may be ok to do so? However, I am not using `PyTorch's` pretrained model out of the box, usually, I go to [Ross's timm/geffnet](https://github.com/rwightman) for the models. Do let me know if we should follow the rule that PyTorch mentioned that **'pretrained models from torchvision' ** that has been trained on images with values in [0, 1] and therefore expects the same format.\n\nTo quote the link from PyTorch: \n\n> All pre-trained models expect input images normalized in the same way,\n> i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where\n> H and W are expected to be at least 224. The images have to be loaded\n> in to a range of [0, 1] and then normalized using mean = [0.485,\n> 0.456, 0.406] and std = [0.229, 0.224, 0.225].", "1129813": "This is actually standard, if you look in the docs of albumentations Normalize() divides each pixel by 255 and then applies the mean and std. So the values are between [0,1] and normalized to imagenet values", "1129900": "Oh dear. How could I have missed that on the source code documentation. You are right.", "1129932": "Happens to the best, good luck chief!", "1130260": "Far from it! Good luck amigos!", "1130850": "albumentations Normalize checks for datatype of image and if it's float it subtracts mean and divides by std. If it's int it first divides image values by 255" }, "source": "meta" }