Mixup seems to work very well in this competition.
\nBut I don't know how to choose alpha properly because this is the first time I use mixup.
\nCan I assume some reasonable values of alpha based on the type of task or data? or, do I have to repeat the experiment endlessly?
\nFYI: I tried only 0.2, 0.5 and 1.0. 1.0 works well for me.
", "messages": [ { "id": 1331370, "postDate": "2021-06-01T12:29:05.913Z", "content": "Mixup seems to work very well in this competition.
\nBut I don't know how to choose alpha properly because this is the first time I use mixup.
\nCan I assume some reasonable values of alpha based on the type of task or data? or, do I have to repeat the experiment endlessly?
\nFYI: I tried only 0.2, 0.5 and 1.0. 1.0 works well for me.
", "rawMarkdown": "Mixup seems to work very well in this competition.\n\nBut I don't know how to choose alpha properly because this is the first time I use mixup.\n\nCan I assume some reasonable values of alpha based on the type of task or data? or, do I have to repeat the experiment endlessly?\n\nFYI: I tried only 0.2, 0.5 and 1.0. 1.0 works well for me.", "votes": 19 }, { "id": 1334039, "postDate": "2021-06-03T08:30:43.757Z", "content": "chris use alpha = np.random.uniform(0.19,0.31) for cutmix https://www.kaggle.com/c/bengaliai-cv19/discussion/136025, I tried this method in mixup，The results will improve slightly in my experiment。
", "rawMarkdown": "chris use alpha = np.random.uniform(0.19,0.31) for cutmix https://www.kaggle.com/c/bengaliai-cv19/discussion/136025, I tried this method in mixup，The results will improve slightly in my experiment。", "votes": 7 }, { "id": 1332727, "postDate": "2021-06-02T09:06:17.040Z", "content": "Try use alpha 5 to make both sample get around 0.5 percentage for mixup, to avoid single sample got too less weights for mixup.
", "rawMarkdown": "Try use alpha 5 to make both sample get around 0.5 percentage for mixup, to avoid single sample got too less weights for mixup.", "votes": 1 }, { "id": 1332626, "postDate": "2021-06-02T08:11:27.757Z", "content": "You can think of mixup as a regularizer and higher alpha leads to higher regularization. (Since we ask the network to predict points harder samples i.e. far away points from the original examples being mixed.)
\nYou can choose values to experiment with by applying similar heuristics we apply for any other regularizer. i.e. amount of underfitting/overfitting for the base model without mixup, model capacity, training budget, etc.
\nI am, for now, using smaller models and have been using alpha=1.0, with a 50% mixup probability.
", "rawMarkdown": "You can think of mixup as a regularizer and higher alpha leads to higher regularization. (Since we ask the network to predict points harder samples i.e. far away points from the original examples being mixed.)\n\nYou can choose values to experiment with by applying similar heuristics we apply for any other regularizer. i.e. amount of underfitting/overfitting for the base model without mixup, model capacity, training budget, etc.\n\nI am, for now, using smaller models and have been using alpha=1.0, with a 50% mixup probability.", "votes": 1, "replies": [ { "id": 1347156, "postDate": "2021-06-13T03:13:09.817Z", "content": "Could you tell me how to use code to achieve 50% mixup，I also thought about doing this, but I don’t know how to implement it in code.
", "rawMarkdown": "Could you tell me how to use code to achieve 50% mixup，I also thought about doing this, but I don’t know how to implement it in code." }, { "id": 1348805, "postDate": "2021-06-14T09:26:46.553Z", "content": "I am just calling the mixup function based on sampled random value… something like this:
\nimport numpy as np
\nmixup_prob = 0.5
p = np.random.rand() # uniform random sampling between 0 and 1
\nif p <= mixup_prob:
\n perform_mixup()
Hello, I have the same question and for now, I find that according to the paper, page 5:
\n\n\nFor mixup, we find that α ∈ [0.1, 0.4] leads to improved performance over ERM, whereas for large α, mixup leads to underfitting
\n
I haven't read the full article so maybe there are some tips depending on the nature of the data.
", "rawMarkdown": "Hello, I have the same question and for now, I find that according to the [paper, page 5](https://arxiv.org/pdf/1710.09412.pdf): \n> For mixup, we find that α ∈ [0.1, 0.4] leads to improved performance over ERM, whereas for large α, mixup leads to underfitting\n\nI haven't read the full article so maybe there are some tips depending on the nature of the data.", "votes": 1, "replies": [ { "id": 1332128, "postDate": "2021-06-01T23:51:24.120Z", "content": "Thanks.
\nI read the paper roughly and started from alpha=0.2 because 0.2 was chosen relatively many times.
\nAs you say, the difference between natural images(ImageNet, CIFAR) and this competition data may have an impact.
", "rawMarkdown": "Thanks.\n\nI read the paper roughly and started from alpha=0.2 because 0.2 was chosen relatively many times.\n\nAs you say, the difference between natural images(ImageNet, CIFAR) and this competition data may have an impact.", "votes": 1 } ] }, { "id": 1331562, "postDate": "2021-06-01T14:48:39.730Z", "content": "Not related to this but how did you implement mixup ?
", "rawMarkdown": "Not related to this but how did you implement mixup ?", "votes": 1, "replies": [ { "id": 1331587, "postDate": "2021-06-01T15:04:52.663Z", "content": "I referenced the implementation in facebook research's repository:
\nhttps://github.com/facebookresearch/mixup-cifar10/blob/master/train.py#L119
I think this implementation is same as one in this notebook:
\nhttps://www.kaggle.com/micheomaano/efficientnet-b4-mixup-cv-0-98-lb-0-97
Here is for tensorflow and keras:
\nhttps://keras.io/examples/vision/mixup/
if you are using timm, then you can maybe checkout timm.data.Mixup
", "rawMarkdown": "if you are using timm, then you can maybe checkout timm.data.Mixup", "votes": 1 } ] }, { "id": 1342435, "postDate": "2021-06-09T12:51:22.010Z", "content": "I tried 2,5,10 when I read https://www.kaggle.com/c/global-wheat-detection/discussion/153257 , but alpha=1 is best for me.
", "rawMarkdown": "I tried 2,5,10 when I read https://www.kaggle.com/c/global-wheat-detection/discussion/153257 , but alpha=1 is best for me.", "votes": 2 } ], "comments": [ { "id": 1334039, "author_name": "MeisterMorxrc", "author_url": "", "post_date": "2021-06-03T08:30:43.757000", "content": "chris use alpha = np.random.uniform(0.19,0.31) for cutmix https://www.kaggle.com/c/bengaliai-cv19/discussion/136025, I tried this method in mixup，The results will improve slightly in my experiment。
", "votes": 7, "replies": [] }, { "id": 1332727, "author_name": "Hao", "author_url": "", "post_date": "2021-06-02T09:06:17.040000", "content": "Try use alpha 5 to make both sample get around 0.5 percentage for mixup, to avoid single sample got too less weights for mixup.
", "votes": 1, "replies": [] }, { "id": 1332626, "author_name": "jvm", "author_url": "", "post_date": "2021-06-02T08:11:27.757000", "content": "You can think of mixup as a regularizer and higher alpha leads to higher regularization. (Since we ask the network to predict points harder samples i.e. far away points from the original examples being mixed.)
\nYou can choose values to experiment with by applying similar heuristics we apply for any other regularizer. i.e. amount of underfitting/overfitting for the base model without mixup, model capacity, training budget, etc.
\nI am, for now, using smaller models and have been using alpha=1.0, with a 50% mixup probability.
", "votes": 1, "replies": [ { "id": 1347156, "author_name": "Gainover", "author_url": "", "post_date": "2021-06-13T03:13:09.817000", "content": "Could you tell me how to use code to achieve 50% mixup，I also thought about doing this, but I don’t know how to implement it in code.
", "votes": 0, "replies": [] }, { "id": 1348805, "author_name": "jvm", "author_url": "", "post_date": "2021-06-14T09:26:46.553000", "content": "I am just calling the mixup function based on sampled random value… something like this:
\nimport numpy as np
\nmixup_prob = 0.5
p = np.random.rand() # uniform random sampling between 0 and 1
\nif p <= mixup_prob:
\n perform_mixup()
Hello, I have the same question and for now, I find that according to the paper, page 5:
\n\n\nFor mixup, we find that α ∈ [0.1, 0.4] leads to improved performance over ERM, whereas for large α, mixup leads to underfitting
\n
I haven't read the full article so maybe there are some tips depending on the nature of the data.
", "votes": 1, "replies": [ { "id": 1332128, "author_name": "Tawara", "author_url": "", "post_date": "2021-06-01T23:51:24.120000", "content": "Thanks.
\nI read the paper roughly and started from alpha=0.2 because 0.2 was chosen relatively many times.
\nAs you say, the difference between natural images(ImageNet, CIFAR) and this competition data may have an impact.
", "votes": 1, "replies": [] } ] }, { "id": 1331562, "author_name": "Mithil Salunkhe", "author_url": "", "post_date": "2021-06-01T14:48:39.730000", "content": "Not related to this but how did you implement mixup ?
", "votes": 1, "replies": [ { "id": 1331587, "author_name": "Tawara", "author_url": "", "post_date": "2021-06-01T15:04:52.663000", "content": "I referenced the implementation in facebook research's repository:
\nhttps://github.com/facebookresearch/mixup-cifar10/blob/master/train.py#L119
I think this implementation is same as one in this notebook:
\nhttps://www.kaggle.com/micheomaano/efficientnet-b4-mixup-cv-0-98-lb-0-97
Here is for tensorflow and keras:
\nhttps://keras.io/examples/vision/mixup/
if you are using timm, then you can maybe checkout timm.data.Mixup
", "votes": 1, "replies": [] } ] }, { "id": 1342435, "author_name": "patriot", "author_url": "", "post_date": "2021-06-09T12:51:22.010000", "content": "I tried 2,5,10 when I read https://www.kaggle.com/c/global-wheat-detection/discussion/153257 , but alpha=1 is best for me.
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "1331370": "Mixup seems to work very well in this competition.\n\nBut I don't know how to choose alpha properly because this is the first time I use mixup.\n\nCan I assume some reasonable values of alpha based on the type of task or data? or, do I have to repeat the experiment endlessly?\n\nFYI: I tried only 0.2, 0.5 and 1.0. 1.0 works well for me.", "1334039": "chris use alpha = np.random.uniform(0.19,0.31) for cutmix https://www.kaggle.com/c/bengaliai-cv19/discussion/136025, I tried this method in mixup，The results will improve slightly in my experiment。", "1332727": "Try use alpha 5 to make both sample get around 0.5 percentage for mixup, to avoid single sample got too less weights for mixup.", "1332626": "You can think of mixup as a regularizer and higher alpha leads to higher regularization. (Since we ask the network to predict points harder samples i.e. far away points from the original examples being mixed.)\n\nYou can choose values to experiment with by applying similar heuristics we apply for any other regularizer. i.e. amount of underfitting/overfitting for the base model without mixup, model capacity, training budget, etc.\n\nI am, for now, using smaller models and have been using alpha=1.0, with a 50% mixup probability.", "1332048": "Hello, I have the same question and for now, I find that according to the [paper, page 5](https://arxiv.org/pdf/1710.09412.pdf): \n> For mixup, we find that α ∈ [0.1, 0.4] leads to improved performance over ERM, whereas for large α, mixup leads to underfitting\n\nI haven't read the full article so maybe there are some tips depending on the nature of the data.", "1331562": "Not related to this but how did you implement mixup ?", "1342435": "I tried 2,5,10 when I read https://www.kaggle.com/c/global-wheat-detection/discussion/153257 , but alpha=1 is best for me." } }