I use 4 data augmentation functions, flip, brightness, contrast and noise. Unlike the implement of torchvision, i set a prob for each function for if or not use it in a train img's pre process. At first, i set a very small prob for each function ({'flip_prob': 0.15, 'brightness': (0.4, 0.9), 'brightness_prob': 0.35, 'contrast': (1.5, 2.5), 'contrast_prob': 0.35, 'noise_prob': 0.4}), then i increase these prob (to{'flip_prob': 0.3, 'brightness': (0.4, 0.9), 'brightness_prob': 0.6, 'contrast': (1.5, 2.5), 'contrast_prob': 0.6, 'noise_prob': 0.6}), and i found the mAP also increase (LB increase too).
But I think if i set the probability very high, or even set it to 1, it will make the distribution of training set and test set very different.
\n\nSo, how do you set the probs and other parameters of data augmentation .
", "messages": [ { "id": "716675", "postDate": "01/12/2020 05:00:10", "content": "I use 4 data augmentation functions, flip, brightness, contrast and noise. Unlike the implement of torchvision, i set a prob for each function for if or not use it in a train img's pre process. At first, i set a very small prob for each function ({'flip_prob': 0.15, 'brightness': (0.4, 0.9), 'brightness_prob': 0.35, 'contrast': (1.5, 2.5), 'contrast_prob': 0.35, 'noise_prob': 0.4}), then i increase these prob (to{'flip_prob': 0.3, 'brightness': (0.4, 0.9), 'brightness_prob': 0.6, 'contrast': (1.5, 2.5), 'contrast_prob': 0.6, 'noise_prob': 0.6}), and i found the mAP also increase (LB increase too).
But I think if i set the probability very high, or even set it to 1, it will make the distribution of training set and test set very different.
\n\nSo, how do you set the probs and other parameters of data augmentation .
", "rawMarkdown": "I use 4 data augmentation functions, flip, brightness, contrast and noise. Unlike the implement of torchvision, i set a prob for each function for if or not use it in a train img's pre process. At first, i set a very small prob for each function (`{'flip_prob': 0.15, 'brightness': (0.4, 0.9), 'brightness_prob': 0.35, 'contrast': (1.5, 2.5), 'contrast_prob': 0.35, 'noise_prob': 0.4}`), then i increase these prob (to` {'flip_prob': 0.3, 'brightness': (0.4, 0.9), 'brightness_prob': 0.6, 'contrast': (1.5, 2.5), 'contrast_prob': 0.6, 'noise_prob': 0.6}`), and i found the mAP also increase (LB increase too). \n\nBut I think if i set the probability very high, or even set it to 1, it will make the distribution of training set and test set very different. \n\nSo, how do you set the probs and other parameters of data augmentation .", "votes": null }, { "id": "716750", "postDate": "01/12/2020 07:45:58", "content": "Look at albumentation for image augmentation
", "rawMarkdown": "Look at albumentation for image augmentation", "votes": null }, { "id": "717020", "postDate": "01/12/2020 16:12:04", "content": "How does flip image work when we have x, y, z values to predict which are related to the 2d coordinates in the image? I've tried to flip the image but the processing the label value seems to be cumbersome.
\nif flip:\n x=-x\n pitch=-pitch\n roll=-roll\n\nx is the value in train.csv
I've seen this. However, if you use the modified 3d x, y, z values and translate 3d to 2d, the 2d coordinates don't match the flipped image so well. So I just discard this modification. I'm surprised this actually works.
Actually, this does not affect the result, cnn could learn these differences. If you think this may make some mistakes, you can add an offset to the value (the y value in picture , y represents the horizontal coordinate in the picture) after flipping. i set offset =11pixel
", "rawMarkdown": "Actually, this does not affect the result, cnn could learn these differences. If you think this may make some mistakes, you can add an offset to the value (the y value in picture , y represents the horizontal coordinate in the picture) after flipping. i set offset =11pixel", "votes": null }, { "id": "717759", "postDate": "01/13/2020 15:24:48", "content": "Yes. I noticed that the offset is a constant value. Thanks for your advice. I will have a try.
", "rawMarkdown": "Yes. I noticed that the offset is a constant value. Thanks for your advice. I will have a try.", "votes": null }, { "id": "719363", "postDate": "01/15/2020 12:34:35", "content": "I believe we should simply flip across the camera principal point. Then, the projected xyz-values will be correct. Here is a small script: https://www.kaggle.com/gebbissimo/correct-horizontal-flipping-during-augmentation
", "rawMarkdown": "I believe we should simply flip across the camera principal point. Then, the projected xyz-values will be correct. Here is a small script: https://www.kaggle.com/gebbissimo/correct-horizontal-flipping-during-augmentation", "votes": null } ], "comments": [ { "id": 716750, "author_name": "econdata", "author_url": "", "post_date": "01/12/2020 07:45:58", "content": "Look at albumentation for image augmentation
", "votes": null, "replies": [] }, { "id": 717020, "author_name": "syoya1997", "author_url": "", "post_date": "01/12/2020 16:12:04", "content": "How does flip image work when we have x, y, z values to predict which are related to the 2d coordinates in the image? I've tried to flip the image but the processing the label value seems to be cumbersome.
\nif flip:\n x=-x\n pitch=-pitch\n roll=-roll\n\nx is the value in train.csv
I've seen this. However, if you use the modified 3d x, y, z values and translate 3d to 2d, the 2d coordinates don't match the flipped image so well. So I just discard this modification. I'm surprised this actually works.
Actually, this does not affect the result, cnn could learn these differences. If you think this may make some mistakes, you can add an offset to the value (the y value in picture , y represents the horizontal coordinate in the picture) after flipping. i set offset =11pixel
", "votes": null, "replies": [] }, { "id": 717759, "author_name": "syoya1997", "author_url": "", "post_date": "01/13/2020 15:24:48", "content": "Yes. I noticed that the offset is a constant value. Thanks for your advice. I will have a try.
", "votes": null, "replies": [] }, { "id": 719363, "author_name": "gebbissimo", "author_url": "", "post_date": "01/15/2020 12:34:35", "content": "I believe we should simply flip across the camera principal point. Then, the projected xyz-values will be correct. Here is a small script: https://www.kaggle.com/gebbissimo/correct-horizontal-flipping-during-augmentation
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "716675": "I use 4 data augmentation functions, flip, brightness, contrast and noise. Unlike the implement of torchvision, i set a prob for each function for if or not use it in a train img's pre process. At first, i set a very small prob for each function (`{'flip_prob': 0.15, 'brightness': (0.4, 0.9), 'brightness_prob': 0.35, 'contrast': (1.5, 2.5), 'contrast_prob': 0.35, 'noise_prob': 0.4}`), then i increase these prob (to` {'flip_prob': 0.3, 'brightness': (0.4, 0.9), 'brightness_prob': 0.6, 'contrast': (1.5, 2.5), 'contrast_prob': 0.6, 'noise_prob': 0.6}`), and i found the mAP also increase (LB increase too). \n\nBut I think if i set the probability very high, or even set it to 1, it will make the distribution of training set and test set very different. \n\nSo, how do you set the probs and other parameters of data augmentation .", "716750": "Look at albumentation for image augmentation", "717020": "How does flip image work when we have `x, y, z` values to predict which are related to the 2d coordinates in the image? I've tried to flip the image but the processing the label value seems to be cumbersome.", "717039": "```\nif flip:\n x=-x\n pitch=-pitch\n roll=-roll\n```\nx is the value in train.csv", "717068": "I've seen this. However, if you use the modified 3d `x, y, z` values and translate 3d to 2d, the 2d coordinates don't match the flipped image so well. So I just discard this modification. I'm surprised this actually works.", "717335": "Actually, this does not affect the result, cnn could learn these differences. If you think this may make some mistakes, you can add an offset to the value (the y value in picture , y represents the horizontal coordinate in the picture) after flipping. i set offset =11pixel", "717759": "Yes. I noticed that the offset is a constant value. Thanks for your advice. I will have a try.", "719363": "I believe we should simply flip across the camera principal point. Then, the projected xyz-values will be correct. Here is a small script: https://www.kaggle.com/gebbissimo/correct-horizontal-flipping-during-augmentation" }, "source": "meta" }