{
  "id": 143236,
  "title": "AugMix implementation",
  "url": "/competitions/flower-classification-with-tpus/discussion/143236",
  "author_name": "",
  "post_date": "2020-04-14T12:43:04.624301600Z",
  "votes": 12,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I want to share with you the implementation of <a href=\"https://arxiv.org/pdf/1912.02781.pdf\">AugMix </a> data augmentation which can be run on TPU.  Here is the notebook: <a href=\"https://www.kaggle.com/szacho/augmix-data-augmentation-on-tpu\">https://www.kaggle.com/szacho/augmix-data-augmentation-on-tpu</a></p>\n\n<p>I did not include Jensen-Shannon Loss, which is recommended in the original paper, because I encountered some memory issues and I'm still trying to fit it in 16GB RAM. However, AugMix may be useful even without this loss. </p>\n\n<p>Also, please have in mind that I'm relatively new in machine learning and kaggle, so any feedback would be greatly appreciated. </p>",
  "messages": [
    {
      "id": "807155",
      "postDate": "04/14/2020 12:43:04",
      "content": "<p>I want to share with you the implementation of <a href=\"https://arxiv.org/pdf/1912.02781.pdf\">AugMix </a> data augmentation which can be run on TPU.  Here is the notebook: <a href=\"https://www.kaggle.com/szacho/augmix-data-augmentation-on-tpu\">https://www.kaggle.com/szacho/augmix-data-augmentation-on-tpu</a></p>\n\n<p>I did not include Jensen-Shannon Loss, which is recommended in the original paper, because I encountered some memory issues and I'm still trying to fit it in 16GB RAM. However, AugMix may be useful even without this loss. </p>\n\n<p>Also, please have in mind that I'm relatively new in machine learning and kaggle, so any feedback would be greatly appreciated. </p>",
      "rawMarkdown": "I want to share with you the implementation of [AugMix ](https://arxiv.org/pdf/1912.02781.pdf) data augmentation which can be run on TPU.  Here is the notebook: https://www.kaggle.com/szacho/augmix-data-augmentation-on-tpu\n\nI did not include Jensen-Shannon Loss, which is recommended in the original paper, because I encountered some memory issues and I'm still trying to fit it in 16GB RAM. However, AugMix may be useful even without this loss. \n\nAlso, please have in mind that I'm relatively new in machine learning and kaggle, so any feedback would be greatly appreciated.",
      "votes": null
    },
    {
      "id": "807178",
      "postDate": "04/14/2020 12:59:00",
      "content": "<p>Thanks, man. You're a genius. Thanks for sharing this.</p>",
      "rawMarkdown": "Thanks, man. You're a genius. Thanks for sharing this.",
      "votes": null
    },
    {
      "id": "807712",
      "postDate": "04/14/2020 20:24:57",
      "content": "<p>Thanks a lot for sharing!</p>",
      "rawMarkdown": "Thanks a lot for sharing!",
      "votes": null
    },
    {
      "id": "822332",
      "postDate": "04/26/2020 20:35:47",
      "content": "<p>I decided to share this special loss implementation as well, even if it's not memory-efficient now. It works flawlessly with small models like MobileNet, but for larger models, one should probably go for memory-efficient optimizers like Adafactor, SM3 to save memory. </p>\n\n<p>Here is the link to a git repo, it contains runnable on Kaggle notebook and basically the same code split into files. \n<a href=\"https://github.com/szacho/flower-classification\">https://github.com/szacho/flower-classification</a></p>\n\n<p>And the bad news is that it does not help in this competition. AugMix in the best case performs as good as no augmentation at all. </p>",
      "rawMarkdown": "I decided to share this special loss implementation as well, even if it's not memory-efficient now. It works flawlessly with small models like MobileNet, but for larger models, one should probably go for memory-efficient optimizers like Adafactor, SM3 to save memory. \n\nHere is the link to a git repo, it contains runnable on Kaggle notebook and basically the same code split into files. \nhttps://github.com/szacho/flower-classification\n\nAnd the bad news is that it does not help in this competition. AugMix in the best case performs as good as no augmentation at all.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 807178,
      "author_name": "akshat0007",
      "author_url": "",
      "post_date": "04/14/2020 12:59:00",
      "content": "<p>Thanks, man. You're a genius. Thanks for sharing this.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 807712,
      "author_name": "ilu000",
      "author_url": "",
      "post_date": "04/14/2020 20:24:57",
      "content": "<p>Thanks a lot for sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 822332,
      "author_name": "szacho",
      "author_url": "",
      "post_date": "04/26/2020 20:35:47",
      "content": "<p>I decided to share this special loss implementation as well, even if it's not memory-efficient now. It works flawlessly with small models like MobileNet, but for larger models, one should probably go for memory-efficient optimizers like Adafactor, SM3 to save memory. </p>\n\n<p>Here is the link to a git repo, it contains runnable on Kaggle notebook and basically the same code split into files. \n<a href=\"https://github.com/szacho/flower-classification\">https://github.com/szacho/flower-classification</a></p>\n\n<p>And the bad news is that it does not help in this competition. AugMix in the best case performs as good as no augmentation at all. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "807155": "I want to share with you the implementation of [AugMix ](https://arxiv.org/pdf/1912.02781.pdf) data augmentation which can be run on TPU.  Here is the notebook: https://www.kaggle.com/szacho/augmix-data-augmentation-on-tpu\n\nI did not include Jensen-Shannon Loss, which is recommended in the original paper, because I encountered some memory issues and I'm still trying to fit it in 16GB RAM. However, AugMix may be useful even without this loss. \n\nAlso, please have in mind that I'm relatively new in machine learning and kaggle, so any feedback would be greatly appreciated.",
    "807178": "Thanks, man. You're a genius. Thanks for sharing this.",
    "807712": "Thanks a lot for sharing!",
    "822332": "I decided to share this special loss implementation as well, even if it's not memory-efficient now. It works flawlessly with small models like MobileNet, but for larger models, one should probably go for memory-efficient optimizers like Adafactor, SM3 to save memory. \n\nHere is the link to a git repo, it contains runnable on Kaggle notebook and basically the same code split into files. \nhttps://github.com/szacho/flower-classification\n\nAnd the bad news is that it does not help in this competition. AugMix in the best case performs as good as no augmentation at all."
  },
  "source": "meta"
}