I have big gap between metrics on my validation dataset and private dataset:
\nCV: 0.91
\nLB: 0.85
\nI don't know why? What the reason? Please help.
\nMy ideas:
I have big gap between metrics on my validation dataset and private dataset:
\nCV: 0.91
\nLB: 0.85
\nI don't know why? What the reason? Please help.
\nMy ideas:
are you using any augementations ? if not then model is probably overfitting , I trained effnetb7 ns encoder with and without augs , without augs :- cv :- 0.93x , lb :- 0.89x , with low augs :- cv :- 0.92x , lb :- 0.91x , initial learning rate :- 1e-4 ,if i started with 1e-3 the model was converging very fast to 0.9x validation dice score within 3-4 epochs but after that my model was improving inconsistently , training started with 1e-4 my model was able to converge to 0.9x in 7-8 epochs and the validation score after every epochs was constantly increasing , although i trained for 40 epochs , training longer / using psuedolabels / ensembling different model architectures / post processing may be the key to success in this comp . also i found out that using large batch sizes improves my model .
", "rawMarkdown": "are you using any augementations ? if not then model is probably overfitting , I trained effnetb7 ns encoder with and without augs , without augs :- cv :- 0.93x , lb :- 0.89x , with low augs :- cv :- 0.92x , lb :- 0.91x , initial learning rate :- 1e-4 ,if i started with 1e-3 the model was converging very fast to 0.9x validation dice score within 3-4 epochs but after that my model was improving inconsistently , training started with 1e-4 my model was able to converge to 0.9x in 7-8 epochs and the validation score after every epochs was constantly increasing , although i trained for 40 epochs , training longer / using psuedolabels / ensembling different model architectures / post processing may be the key to success in this comp . also i found out that using large batch sizes improves my model .", "votes": null }, { "id": "1269978", "postDate": "04/11/2021 06:32:09", "content": "Thank you for the answer. I do use the augmentation: random rotation, scale, contrast, gamma, hue shift. Should I add a blur? Anyway my gap is match bigger than your: 0.06 vs 0.01. How do you read images with rasterio crop by crop?
", "rawMarkdown": "Thank you for the answer. I do use the augmentation: random rotation, scale, contrast, gamma, hue shift. Should I add a blur? Anyway my gap is match bigger than your: 0.06 vs 0.01. How do you read images with rasterio crop by crop?", "votes": null }, { "id": "1270057", "postDate": "04/11/2021 08:29:20", "content": "I just increased my augmentation and my LB score decreased to 0.811
", "rawMarkdown": "I just increased my augmentation and my LB score decreased to 0.811", "votes": null }, { "id": "1270074", "postDate": "04/11/2021 08:59:10", "content": "my update :- single model effnetb7ns 3 fold cv : 0.9213 lb :- 0.920 , btw i am using the trick that @iafoss has shared here https://www.kaggle.com/iafoss/256x256-images to generate images .
", "rawMarkdown": "my update :- single model effnetb7ns 3 fold cv : 0.9213 lb :- 0.920 , btw i am using the trick that @iafoss has shared here https://www.kaggle.com/iafoss/256x256-images to generate images .", "votes": null }, { "id": "1270136", "postDate": "04/11/2021 10:30:08", "content": "I don't have folds, so my CV is not pricise. I see in the 256x256 code:
\nhsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
\nIs rasterio return BGR?
I think it returns RGB, but BGR2HSV and RGB2HSV give the same results
", "rawMarkdown": "I think it returns RGB, but BGR2HSV and RGB2HSV give the same results", "votes": null }, { "id": "1271209", "postDate": "04/12/2021 12:00:48", "content": "You can try augmentations. We used more than 20 types of methods to augment the datasets. And you can add dropout layer in your model.😄
", "rawMarkdown": "You can try augmentations. We used more than 20 types of methods to augment the datasets. And you can add dropout layer in your model.😄", "votes": null }, { "id": "1286909", "postDate": "04/28/2021 14:20:13", "content": "I am using the segmentation_models library popular here. But how to add dropout layer to backbone of the Unet model?
", "rawMarkdown": "I am using the segmentation_models library popular here. But how to add dropout layer to backbone of the Unet model?", "votes": null }, { "id": "1287431", "postDate": "04/29/2021 04:42:45", "content": "Usually add dropout layer behind the backbone just the CNN
", "rawMarkdown": "Usually add dropout layer behind the backbone just the CNN", "votes": null } ], "comments": [ { "id": 1269455, "author_name": "trooperog", "author_url": "", "post_date": "04/10/2021 14:46:11", "content": "are you using any augementations ? if not then model is probably overfitting , I trained effnetb7 ns encoder with and without augs , without augs :- cv :- 0.93x , lb :- 0.89x , with low augs :- cv :- 0.92x , lb :- 0.91x , initial learning rate :- 1e-4 ,if i started with 1e-3 the model was converging very fast to 0.9x validation dice score within 3-4 epochs but after that my model was improving inconsistently , training started with 1e-4 my model was able to converge to 0.9x in 7-8 epochs and the validation score after every epochs was constantly increasing , although i trained for 40 epochs , training longer / using psuedolabels / ensembling different model architectures / post processing may be the key to success in this comp . also i found out that using large batch sizes improves my model .
", "votes": null, "replies": [ { "id": 1269978, "author_name": "vedenev", "author_url": "", "post_date": "04/11/2021 06:32:09", "content": "Thank you for the answer. I do use the augmentation: random rotation, scale, contrast, gamma, hue shift. Should I add a blur? Anyway my gap is match bigger than your: 0.06 vs 0.01. How do you read images with rasterio crop by crop?
", "votes": null, "replies": [] }, { "id": 1270057, "author_name": "vedenev", "author_url": "", "post_date": "04/11/2021 08:29:20", "content": "I just increased my augmentation and my LB score decreased to 0.811
", "votes": null, "replies": [] }, { "id": 1270074, "author_name": "trooperog", "author_url": "", "post_date": "04/11/2021 08:59:10", "content": "my update :- single model effnetb7ns 3 fold cv : 0.9213 lb :- 0.920 , btw i am using the trick that @iafoss has shared here https://www.kaggle.com/iafoss/256x256-images to generate images .
", "votes": null, "replies": [] }, { "id": 1270136, "author_name": "vedenev", "author_url": "", "post_date": "04/11/2021 10:30:08", "content": "I don't have folds, so my CV is not pricise. I see in the 256x256 code:
\nhsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
\nIs rasterio return BGR?
I think it returns RGB, but BGR2HSV and RGB2HSV give the same results
", "votes": null, "replies": [] } ] }, { "id": 1271209, "author_name": "jasonhuangcn", "author_url": "", "post_date": "04/12/2021 12:00:48", "content": "You can try augmentations. We used more than 20 types of methods to augment the datasets. And you can add dropout layer in your model.😄
", "votes": null, "replies": [ { "id": 1286909, "author_name": "vsharkeev", "author_url": "", "post_date": "04/28/2021 14:20:13", "content": "I am using the segmentation_models library popular here. But how to add dropout layer to backbone of the Unet model?
", "votes": null, "replies": [] }, { "id": 1287431, "author_name": "jasonhuangcn", "author_url": "", "post_date": "04/29/2021 04:42:45", "content": "Usually add dropout layer behind the backbone just the CNN
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1269269": "I have big gap between metrics on my validation dataset and private dataset:\nCV: 0.91\nLB: 0.85\nI don't know why? What the reason? Please help.\nMy ideas:\n- glomeruli of another type more frequent in the private dataset\n- pale or blured images in the private dataset\n- some images in the private dataset not in RGB colorsapce\n- problems with image reading in the private dataset", "1269455": "are you using any augementations ? if not then model is probably overfitting , I trained effnetb7 ns encoder with and without augs , without augs :- cv :- 0.93x , lb :- 0.89x , with low augs :- cv :- 0.92x , lb :- 0.91x , initial learning rate :- 1e-4 ,if i started with 1e-3 the model was converging very fast to 0.9x validation dice score within 3-4 epochs but after that my model was improving inconsistently , training started with 1e-4 my model was able to converge to 0.9x in 7-8 epochs and the validation score after every epochs was constantly increasing , although i trained for 40 epochs , training longer / using psuedolabels / ensembling different model architectures / post processing may be the key to success in this comp . also i found out that using large batch sizes improves my model .", "1269978": "Thank you for the answer. I do use the augmentation: random rotation, scale, contrast, gamma, hue shift. Should I add a blur? Anyway my gap is match bigger than your: 0.06 vs 0.01. How do you read images with rasterio crop by crop?", "1270057": "I just increased my augmentation and my LB score decreased to 0.811", "1270074": "my update :- single model effnetb7ns 3 fold cv : 0.9213 lb :- 0.920 , btw i am using the trick that @iafoss has shared here https://www.kaggle.com/iafoss/256x256-images to generate images .", "1270136": "I don't have folds, so my CV is not pricise. I see in the 256x256 code:\nhsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\nIs rasterio return BGR?", "1270424": "I think it returns RGB, but BGR2HSV and RGB2HSV give the same results", "1271209": "You can try augmentations. We used more than 20 types of methods to augment the datasets. And you can add dropout layer in your model.😄", "1286909": "I am using the segmentation_models library popular here. But how to add dropout layer to backbone of the Unet model?", "1287431": "Usually add dropout layer behind the backbone just the CNN" }, "source": "meta" }