In my experiments, mixup will make models less confident, maybe overconfident. From this point, I think it's similar to label smoothing. Also, it makes sense to me when thinking of its loss representation. please share your insight as well.
", "messages": [ { "id": 1350200, "postDate": "2021-06-15T10:33:37.053Z", "content": "It is somehow like label smoothing but more complex. In label smoothing you enforce the label 1 to be less than 1 (eg 0.98) but on tehniques like mixup and cutmix you help generalize better creating a smooth transition between hard labels like 0 and 1, now a image with mixup/cutmix can have the label 0.3 or 0.7 depending on the criteria.
\nIt is also helps the model to learn from incomplete information by providing less clear images for training.
\nAlso, creating a more clear interpolations between labels mixup techniques are used against attacks with adversarial examples.
From my observation, mixup does not work for me. By adding that my model droped from 0.965 to 0.964. Do not know what the problems are.
", "rawMarkdown": "From my observation, mixup does not work for me. By adding that my model droped from 0.965 to 0.964. Do not know what the problems are.", "votes": 1, "replies": [ { "id": 1357367, "postDate": "2021-06-19T16:17:09.507Z", "content": "@coincheung What alpha parameter of the beta distribution are you using ? Also, what type of model and resolution ? Are you using other hard core augment techniques?
", "rawMarkdown": "@coincheung What alpha parameter of the beta distribution are you using ? Also, what type of model and resolution ? Are you using other hard core augment techniques? " } ] }, { "id": 1349874, "postDate": "2021-06-15T06:38:34.297Z", "content": "In my experiments, mixup will make models less confident, maybe overconfident. From this point, I think it's similar to label smoothing. Also, it makes sense to me when thinking of its loss representation. please share your insight as well.
", "rawMarkdown": "In my experiments, mixup will make models less confident, maybe overconfident. From this point, I think it's similar to label smoothing. Also, it makes sense to me when thinking of its loss representation. please share your insight as well.", "votes": 1 } ], "comments": [ { "id": 1350200, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2021-06-15T10:33:37.053000", "content": "It is somehow like label smoothing but more complex. In label smoothing you enforce the label 1 to be less than 1 (eg 0.98) but on tehniques like mixup and cutmix you help generalize better creating a smooth transition between hard labels like 0 and 1, now a image with mixup/cutmix can have the label 0.3 or 0.7 depending on the criteria.
\nIt is also helps the model to learn from incomplete information by providing less clear images for training.
\nAlso, creating a more clear interpolations between labels mixup techniques are used against attacks with adversarial examples.
From my observation, mixup does not work for me. By adding that my model droped from 0.965 to 0.964. Do not know what the problems are.
", "votes": 1, "replies": [ { "id": 1357367, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2021-06-19T16:17:09.507000", "content": "@coincheung What alpha parameter of the beta distribution are you using ? Also, what type of model and resolution ? Are you using other hard core augment techniques?
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1350200": "It is somehow like label smoothing but more complex. In label smoothing you enforce the label 1 to be less than 1 (eg 0.98) but on tehniques like mixup and cutmix you help generalize better creating a smooth transition between hard labels like 0 and 1, now a image with mixup/cutmix can have the label 0.3 or 0.7 depending on the criteria. \nIt is also helps the model to learn from incomplete information by providing less clear images for training.\nAlso, creating a more clear interpolations between labels mixup techniques are used against attacks with adversarial examples.\n", "1353318": "From my observation, mixup does not work for me. By adding that my model droped from 0.965 to 0.964. Do not know what the problems are.", "1349874": "In my experiments, mixup will make models less confident, maybe overconfident. From this point, I think it's similar to label smoothing. Also, it makes sense to me when thinking of its loss representation. please share your insight as well." } }