![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F113660%2Fadeb3f274e3aa6d55d9f135da5c8e72f%2FSelection_120.png?generation=1580999468651681&alt=media\")
i am thinking of making some tutorial and code for \"fast autoagument from scratch\". it will be based on cifar10 ( wide-resnet-28-10 ). I spend some time to read and understand the paper and opensource github code.
\n\ni would like to know if my understanding below is correct or no?
\n\n```\nfast autoaugmentation algorithm:
\n\ndivide dataset in to K fold. for each fold we have Dm_k and Da_k.
train K models for each Dm_k us usual. once trained these model will not be modified.
for each trained model (say model_k), determine the best augmentation T_k for dataset Da_k as follow:\n for k in 1 to K:\n T_k = empty\n for t in 1 to B:\n estimated_best_augment = bayes_optimisation_with_hyperopt(\n candidate_augments, T_k, model_k, Da_k, \n model_loss_function )\n estimated_best_augment = select_top_augment (estimated_best_augment)\n T_k = union (T_k, estimated_best_augment)
combine all T_k into T (learned augmentation). Retrain a new model using T
```
\n\nIf it is correct, i will finalize and code and release it later. thanks
", "messages": [ { "id": "738321", "postDate": "02/06/2020 11:38:14", "content": "i am thinking of making some tutorial and code for \"fast autoagument from scratch\". it will be based on cifar10 ( wide-resnet-28-10 ). I spend some time to read and understand the paper and opensource github code.
\n\ni would like to know if my understanding below is correct or no?
\n\n```\nfast autoaugmentation algorithm:
\n\ndivide dataset in to K fold. for each fold we have Dm_k and Da_k.
train K models for each Dm_k us usual. once trained these model will not be modified.
for each trained model (say model_k), determine the best augmentation T_k for dataset Da_k as follow:\n for k in 1 to K:\n T_k = empty\n for t in 1 to B:\n estimated_best_augment = bayes_optimisation_with_hyperopt(\n candidate_augments, T_k, model_k, Da_k, \n model_loss_function )\n estimated_best_augment = select_top_augment (estimated_best_augment)\n T_k = union (T_k, estimated_best_augment)
combine all T_k into T (learned augmentation). Retrain a new model using T
```
\n\nIf it is correct, i will finalize and code and release it later. thanks
", "rawMarkdown": "i am thinking of making some tutorial and code for \"fast autoagument from scratch\". it will be based on cifar10 ( wide-resnet-28-10 ). I spend some time to read and understand the paper and opensource github code.\n\ni would like to know if my understanding below is correct or no?\n\n```\nfast autoaugmentation algorithm:\n\n1. divide dataset in to K fold. for each fold we have Dm_k and Da_k.\n\n2. train K models for each Dm_k us usual. once trained these model will not be modified.\n\n3. for each trained model (say model_k), determine the best augmentation T_k for dataset Da_k as follow:\n for k in 1 to K:\n T_k = empty\n for t in 1 to B:\n estimated_best_augment = bayes_optimisation_with_hyperopt(\n candidate_augments, T_k, model_k, Da_k, \n model_loss_function )\n estimated_best_augment = select_top_augment (estimated_best_augment)\n T_k = union (T_k, estimated_best_augment)\n\n4. combine all T_k into T (learned augmentation). Retrain a new model using T\n\n```\n\nIf it is correct, i will finalize and code and release it later. thanks", "votes": null }, { "id": "738420", "postDate": "02/06/2020 13:53:03", "content": "I will shamelessly tag @ildoonet as he is one of the paper's authors
", "rawMarkdown": "I will shamelessly tag @ildoonet as he is one of the paper's authors", "votes": null }, { "id": "738445", "postDate": "02/06/2020 14:31:11", "content": "proposed study plan:
\n\n
pre-release version:
\n\nhttps://drive.google.com/drive/folders/1-IZzvSAfDloDRYEWlB-U8P7aZW8TIvdO
\n\n2020-0206\n- baseline implementation of wide-resnet-28-10 , achieve 2.6 to 2.9 error on cifar10 using opensource learned policy augmentation. code is refactor from https://github.com/kakaobrain/fast-autoaugment, for my own clarity and future experiment.
\n\nnext plan:\n- get baseline results without opensource learned policy augmentation.\n- see if i can learn the policy myself.
", "rawMarkdown": "pre-release version:\n\nhttps://drive.google.com/drive/folders/1-IZzvSAfDloDRYEWlB-U8P7aZW8TIvdO\n\n---\n\n2020-0206\n- baseline implementation of wide-resnet-28-10 , achieve 2.6 to 2.9 error on cifar10 using opensource learned policy augmentation. code is refactor from https://github.com/kakaobrain/fast-autoaugment, for my own clarity and future experiment.\n\n- this confirm implementation of wide-resnet-28-10 is correct. It also confirm the training code and hyper-parameters are correct. the training loss curve will be used as reference in future comparison\n\nnext plan:\n- get baseline results without opensource learned policy augmentation.\n- see if i can learn the policy myself.", "votes": null }, { "id": "757097", "postDate": "02/26/2020 12:33:36", "content": "\n\njoin this slack group for experiment on cifar10 on auto augment (discussion on \nBengali.AI dataset is not allowed since it is prohibited share discussion among non-team members)
", "rawMarkdown": "https://join.slack.com/t/cifar10-autoaugment/shared_invite/enQtOTcyMDkwNzk4MDA2LTBlODhlYjM4ZjA5YjkxYjQxNTZhNzU0NjM5ZTc4MTE1N2I5ZmNmOGJkZTA3NzhmNWYwNTdiOTY0ZjNiYTViOGQ\n\njoin this slack group for experiment on cifar10 on auto augment (discussion on \nBengali.AI dataset is not allowed since it is prohibited share discussion among non-team members)", "votes": null }, { "id": "757116", "postDate": "02/26/2020 13:05:19", "content": "@hengck23 I believe it is alright if the discussion is is in the forum? So long as it is accessible to all of us. Thanks for sharing
", "rawMarkdown": "hengck23 I believe it is alright if the discussion is is in the forum? So long as it is accessible to all of us. Thanks for sharing", "votes": null }, { "id": "757127", "postDate": "02/26/2020 13:12:49", "content": "it is not allowed. in my first discussion in Amazon Forest Challenge, the kaggle moderator mentions that all discussion for the kaggle challenge must be in the forum (even external discussion accessible to all kagglers are not allowed). This is to enable the moderator to moderates all discussions.
", "rawMarkdown": "it is not allowed. in my first discussion in Amazon Forest Challenge, the kaggle moderator mentions that all discussion for the kaggle challenge must be in the forum (even external discussion accessible to all kagglers are not allowed). This is to enable the moderator to moderates all discussions.", "votes": null }, { "id": "758319", "postDate": "02/27/2020 16:29:53", "content": "2020-0226 release:\n- wide-resnet-28-10 is takes a bit of memory. So wide-resnet-40-2 is used instead. Cifar10 is replaced by cifar100 as this is closer to our case (lack of samples per class).
\n\nsome preliminary training results is given but not finalised. i am working for free gpu resources, especially for pbt training
", "rawMarkdown": "2020-0226 release:\n- wide-resnet-28-10 is takes a bit of memory. So wide-resnet-40-2 is used instead. Cifar10 is replaced by cifar100 as this is closer to our case (lack of samples per class).\n\n- implement training for the following cases:\n 1. basic augmentation only (flip + shift)\n 2. basic+cutout\n 3. complex augmentation without augmentation hyperparameters optimization (i,e, random)\n 4. complex augmentation with augmentation hyperparameters optimization using population based training (pbt) implemented using ray framework\n 5. learned policy augmentation from fast-autoagument github website\n\n----\n\nsome preliminary training results is given but not finalised. i am working for free gpu resources, especially for pbt training", "votes": null }, { "id": "771405", "postDate": "03/14/2020 05:45:09", "content": "DADA: Differentiable Automatic Data Augmentation\nhttps://arxiv.org/pdf/2003.03780.pdf
", "rawMarkdown": "DADA: Differentiable Automatic Data Augmentation\nhttps://arxiv.org/pdf/2003.03780.pdf", "votes": null } ], "comments": [ { "id": 738420, "author_name": "moewie94", "author_url": "", "post_date": "02/06/2020 13:53:03", "content": "I will shamelessly tag @ildoonet as he is one of the paper's authors
", "votes": null, "replies": [] }, { "id": 738445, "author_name": "hengck23", "author_url": "", "post_date": "02/06/2020 14:31:11", "content": "proposed study plan:
\n\n
pre-release version:
\n\nhttps://drive.google.com/drive/folders/1-IZzvSAfDloDRYEWlB-U8P7aZW8TIvdO
\n\n2020-0206\n- baseline implementation of wide-resnet-28-10 , achieve 2.6 to 2.9 error on cifar10 using opensource learned policy augmentation. code is refactor from https://github.com/kakaobrain/fast-autoaugment, for my own clarity and future experiment.
\n\nnext plan:\n- get baseline results without opensource learned policy augmentation.\n- see if i can learn the policy myself.
", "votes": null, "replies": [] }, { "id": 757097, "author_name": "hengck23", "author_url": "", "post_date": "02/26/2020 12:33:36", "content": "\n\njoin this slack group for experiment on cifar10 on auto augment (discussion on \nBengali.AI dataset is not allowed since it is prohibited share discussion among non-team members)
", "votes": null, "replies": [ { "id": 757116, "author_name": "roguekk007", "author_url": "", "post_date": "02/26/2020 13:05:19", "content": "@hengck23 I believe it is alright if the discussion is is in the forum? So long as it is accessible to all of us. Thanks for sharing
", "votes": null, "replies": [] }, { "id": 757127, "author_name": "hengck23", "author_url": "", "post_date": "02/26/2020 13:12:49", "content": "it is not allowed. in my first discussion in Amazon Forest Challenge, the kaggle moderator mentions that all discussion for the kaggle challenge must be in the forum (even external discussion accessible to all kagglers are not allowed). This is to enable the moderator to moderates all discussions.
", "votes": null, "replies": [] } ] }, { "id": 758319, "author_name": "hengck23", "author_url": "", "post_date": "02/27/2020 16:29:53", "content": "2020-0226 release:\n- wide-resnet-28-10 is takes a bit of memory. So wide-resnet-40-2 is used instead. Cifar10 is replaced by cifar100 as this is closer to our case (lack of samples per class).
\n\nsome preliminary training results is given but not finalised. i am working for free gpu resources, especially for pbt training
", "votes": null, "replies": [] }, { "id": 771405, "author_name": "hengck23", "author_url": "", "post_date": "03/14/2020 05:45:09", "content": "DADA: Differentiable Automatic Data Augmentation\nhttps://arxiv.org/pdf/2003.03780.pdf
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "738321": "i am thinking of making some tutorial and code for \"fast autoagument from scratch\". it will be based on cifar10 ( wide-resnet-28-10 ). I spend some time to read and understand the paper and opensource github code.\n\ni would like to know if my understanding below is correct or no?\n\n```\nfast autoaugmentation algorithm:\n\n1. divide dataset in to K fold. for each fold we have Dm_k and Da_k.\n\n2. train K models for each Dm_k us usual. once trained these model will not be modified.\n\n3. for each trained model (say model_k), determine the best augmentation T_k for dataset Da_k as follow:\n for k in 1 to K:\n T_k = empty\n for t in 1 to B:\n estimated_best_augment = bayes_optimisation_with_hyperopt(\n candidate_augments, T_k, model_k, Da_k, \n model_loss_function )\n estimated_best_augment = select_top_augment (estimated_best_augment)\n T_k = union (T_k, estimated_best_augment)\n\n4. combine all T_k into T (learned augmentation). Retrain a new model using T\n\n```\n\nIf it is correct, i will finalize and code and release it later. thanks", "738420": "I will shamelessly tag @ildoonet as he is one of the paper's authors", "738445": "proposed study plan:\n\n", "738455": "pre-release version:\n\nhttps://drive.google.com/drive/folders/1-IZzvSAfDloDRYEWlB-U8P7aZW8TIvdO\n\n---\n\n2020-0206\n- baseline implementation of wide-resnet-28-10 , achieve 2.6 to 2.9 error on cifar10 using opensource learned policy augmentation. code is refactor from https://github.com/kakaobrain/fast-autoaugment, for my own clarity and future experiment.\n\n- this confirm implementation of wide-resnet-28-10 is correct. It also confirm the training code and hyper-parameters are correct. the training loss curve will be used as reference in future comparison\n\nnext plan:\n- get baseline results without opensource learned policy augmentation.\n- see if i can learn the policy myself.", "757097": "https://join.slack.com/t/cifar10-autoaugment/shared_invite/enQtOTcyMDkwNzk4MDA2LTBlODhlYjM4ZjA5YjkxYjQxNTZhNzU0NjM5ZTc4MTE1N2I5ZmNmOGJkZTA3NzhmNWYwNTdiOTY0ZjNiYTViOGQ\n\njoin this slack group for experiment on cifar10 on auto augment (discussion on \nBengali.AI dataset is not allowed since it is prohibited share discussion among non-team members)", "757116": "hengck23 I believe it is alright if the discussion is is in the forum? So long as it is accessible to all of us. Thanks for sharing", "757127": "it is not allowed. in my first discussion in Amazon Forest Challenge, the kaggle moderator mentions that all discussion for the kaggle challenge must be in the forum (even external discussion accessible to all kagglers are not allowed). This is to enable the moderator to moderates all discussions.", "758319": "2020-0226 release:\n- wide-resnet-28-10 is takes a bit of memory. So wide-resnet-40-2 is used instead. Cifar10 is replaced by cifar100 as this is closer to our case (lack of samples per class).\n\n- implement training for the following cases:\n 1. basic augmentation only (flip + shift)\n 2. basic+cutout\n 3. complex augmentation without augmentation hyperparameters optimization (i,e, random)\n 4. complex augmentation with augmentation hyperparameters optimization using population based training (pbt) implemented using ray framework\n 5. learned policy augmentation from fast-autoagument github website\n\n----\n\nsome preliminary training results is given but not finalised. i am working for free gpu resources, especially for pbt training", "771405": "DADA: Differentiable Automatic Data Augmentation\nhttps://arxiv.org/pdf/2003.03780.pdf" }, "source": "meta" }