\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4630396%2Fecc23148ecbc394237d240e64c35de30%2Fcomparison.png?generation=1610780740528070&alt=media\")
I recently learned that using Custom Dataset Mean and Std Normalization can help your model to learn and perform better. Hence I have tried to compare the images by normalizing with Normal Stats and ImageNet Stats
\nMeans and STD's used -
\nImageNet - \n Mean - [0.485, 0.456, 0.406] \n STD - [0.229, 0.224, 0.225]\n\nDataset - \n Mean - [0.4303, 0.4967, 0.3134]\n STD - [0.2142, 0.2191, 0.1954]\nI have taken the values from this notebook.
\n
Interested to know your thoughts on these images.
", "messages": [ { "id": "1155060", "postDate": "01/16/2021 07:06:47", "content": "I recently learned that using Custom Dataset Mean and Std Normalization can help your model to learn and perform better. Hence I have tried to compare the images by normalizing with Normal Stats and ImageNet Stats
\nMeans and STD's used -
\nImageNet - \n Mean - [0.485, 0.456, 0.406] \n STD - [0.229, 0.224, 0.225]\n\nDataset - \n Mean - [0.4303, 0.4967, 0.3134]\n STD - [0.2142, 0.2191, 0.1954]\nI have taken the values from this notebook.
\n
Interested to know your thoughts on these images.
", "rawMarkdown": "I recently learned that using Custom Dataset Mean and Std Normalization can help your model to learn and perform better. Hence I have tried to compare the images by normalizing with Normal Stats and ImageNet Stats\n\nMeans and STD's used - \n```\nImageNet - \n Mean - [0.485, 0.456, 0.406] \n STD - [0.229, 0.224, 0.225]\n\nDataset - \n Mean - [0.4303, 0.4967, 0.3134]\n STD - [0.2142, 0.2191, 0.1954]\n```\nI have taken the values from [this](https://www.kaggle.com/woshifym/cassava-mean-std-calculation/notebook) notebook.\n\n\nInterested to know your thoughts on these images.", "votes": null }, { "id": "1155075", "postDate": "01/16/2021 07:24:34", "content": "i tried with and without normalization in the past and never observed much difference,let me know if you get boost in CV with dataset normalization in this competition,thank you
", "rawMarkdown": "i tried with and without normalization in the past and never observed much difference,let me know if you get boost in CV with dataset normalization in this competition,thank you", "votes": null }, { "id": "1155078", "postDate": "01/16/2021 07:29:36", "content": "Definitely. I will report back my results here.
", "rawMarkdown": "Definitely. I will report back my results here.", "votes": null }, { "id": "1155249", "postDate": "01/16/2021 10:21:04", "content": "Here are my Model training results -
\nI changed the Mean and STD of my Notebook here
Best Accuracy with ImageNet Normalization - 0.89319934\nBest Accuracy with Custom Normalization - 0.89390044 \nUsing Custom Dataset Normalization Helps with the CV.
", "rawMarkdown": "Here are my Model training results - \nI changed the Mean and STD of my Notebook [here](https://www.kaggle.com/yerramvarun/cassava-taylorce-loss-label-smoothing-combo)\n\n```\nBest Accuracy with ImageNet Normalization - 0.89319934\nBest Accuracy with Custom Normalization - 0.89390044 \n```\nUsing Custom Dataset Normalization Helps with the CV.", "votes": null }, { "id": "1155382", "postDate": "01/16/2021 11:22:02", "content": "now train without any normalization and observe the difference?
", "rawMarkdown": "now train without any normalization and observe the difference?", "votes": null }, { "id": "1155720", "postDate": "01/16/2021 15:54:38", "content": "@mobassir I have run the training without any normalization and it performs slightly worse than the other two.
\nBest Accuracy without any Normalization - 0.8903949\nAre you sure that a difference at fourth decimal place is significant for this competition ??
", "rawMarkdown": "Are you sure that a difference at fourth decimal place is significant for this competition ??", "votes": null }, { "id": "1155827", "postDate": "01/16/2021 17:45:22", "content": "I understand that it might seem insignificant, but do consider that the above values are for a single fold and using a loss which is not used generally, I am currently training for remaining folds and using a common loss and then I can say for sure if they matter or not.
", "rawMarkdown": "I understand that it might seem insignificant, but do consider that the above values are for a single fold and using a loss which is not used generally, I am currently training for remaining folds and using a common loss and then I can say for sure if they matter or not.", "votes": null }, { "id": "1155876", "postDate": "01/16/2021 18:31:55", "content": "Frankly, I am having a hard time understanding why the change in the normalization would have any effect at all. After all, renormalizing the images is just a simple linear transformation. Our neural net with millions trainable parameters should have no problem learning such a transformation if it is indeed beneficial. Please let me know if I am missing something.
", "rawMarkdown": "Frankly, I am having a hard time understanding why the change in the normalization would have any effect at all. After all, renormalizing the images is just a simple linear transformation. Our neural net with millions trainable parameters should have no problem learning such a transformation if it is indeed beneficial. Please let me know if I am missing something.", "votes": null }, { "id": "1155934", "postDate": "01/16/2021 19:33:10", "content": "Agreed
\nI read many stuffs in discussion topics here that I am sure have, at best, very marginal impact on model performance.
\nBut it may be a misinterpretation on my part to reassure myself because I don't have GPU/Time to try them out ^^
i tried with and without normalization in the past and never observed much difference,let me know if you get boost in CV with dataset normalization in this competition,thank you
", "votes": null, "replies": [ { "id": 1155078, "author_name": "yerramvarun", "author_url": "", "post_date": "01/16/2021 07:29:36", "content": "Definitely. I will report back my results here.
", "votes": null, "replies": [] } ] }, { "id": 1155249, "author_name": "yerramvarun", "author_url": "", "post_date": "01/16/2021 10:21:04", "content": "Here are my Model training results -
\nI changed the Mean and STD of my Notebook here
Best Accuracy with ImageNet Normalization - 0.89319934\nBest Accuracy with Custom Normalization - 0.89390044 \nUsing Custom Dataset Normalization Helps with the CV.
", "votes": null, "replies": [ { "id": 1155382, "author_name": "mobassir", "author_url": "", "post_date": "01/16/2021 11:22:02", "content": "now train without any normalization and observe the difference?
", "votes": null, "replies": [] }, { "id": 1155720, "author_name": "yerramvarun", "author_url": "", "post_date": "01/16/2021 15:54:38", "content": "@mobassir I have run the training without any normalization and it performs slightly worse than the other two.
\nBest Accuracy without any Normalization - 0.8903949\nAre you sure that a difference at fourth decimal place is significant for this competition ??
", "votes": null, "replies": [] }, { "id": 1155827, "author_name": "yerramvarun", "author_url": "", "post_date": "01/16/2021 17:45:22", "content": "I understand that it might seem insignificant, but do consider that the above values are for a single fold and using a loss which is not used generally, I am currently training for remaining folds and using a common loss and then I can say for sure if they matter or not.
", "votes": null, "replies": [] } ] }, { "id": 1155876, "author_name": "graf10a", "author_url": "", "post_date": "01/16/2021 18:31:55", "content": "Frankly, I am having a hard time understanding why the change in the normalization would have any effect at all. After all, renormalizing the images is just a simple linear transformation. Our neural net with millions trainable parameters should have no problem learning such a transformation if it is indeed beneficial. Please let me know if I am missing something.
", "votes": null, "replies": [ { "id": 1155934, "author_name": "serigne", "author_url": "", "post_date": "01/16/2021 19:33:10", "content": "Agreed
\nI read many stuffs in discussion topics here that I am sure have, at best, very marginal impact on model performance.
\nBut it may be a misinterpretation on my part to reassure myself because I don't have GPU/Time to try them out ^^