{ "id": 199631, "title": "Mixed Precision Training", "url": "/competitions/cassava-leaf-disease-classification/discussion/199631", "author_name": "", "post_date": "2020-11-26T14:22:09.470570500Z", "votes": 12, "comment_count": 2, "views": 0, "content": "

These are some resources on what Mixed Precision Training is and how it might be useful.
\nThis will help you in faster training and decrease memory requirements.

\n
    \n
  1. Henry AI Labs
    2. \n
  2. Abhishek Thakur's video on using AMP with pytorch
    3. \n
  3. NVIDIA Blog
    4. \n
  4. Original Paper
    5. \n
\n

Hope this helps!

", "messages": [ { "id": "1092076", "postDate": "11/26/2020 14:22:09", "content": "

These are some resources on what Mixed Precision Training is and how it might be useful.
\nThis will help you in faster training and decrease memory requirements.

\n
    \n
  1. Henry AI Labs
    2. \n
  2. Abhishek Thakur's video on using AMP with pytorch
    3. \n
  3. NVIDIA Blog
    4. \n
  4. Original Paper
    5. \n
\n

Hope this helps!

", "rawMarkdown": "These are some resources on what Mixed Precision Training is and how it might be useful.\nThis will help you in faster training and decrease memory requirements.\n1. [Henry AI Labs](https://www.youtube.com/watch?v=pKZs4hllCvI)\n2. [Abhishek Thakur's video on using AMP with pytorch](https://www.youtube.com/watch?v=X7iOkhGePXg)\n3. [NVIDIA Blog](https://developer.nvidia.com/automatic-mixed-precision)\n4. [Original Paper](https://arxiv.org/pdf/1710.03740.pdf) \n\nHope this helps!", "votes": null }, { "id": "1094934", "postDate": "11/29/2020 05:27:08", "content": "

Thanks for sharing! For anyone looking for a quick and simple definition of Mixed Precision Training, it's the use of both 16 and 32 bit floating point values during training. This means that your hardware has the flexibility to use less memory when less memory is required.
\nHere's a link to Tensorflow's information on the subject.

", "rawMarkdown": "Thanks for sharing! For anyone looking for a quick and simple definition of Mixed Precision Training, it's the use of both 16 and 32 bit floating point values during training. This means that your hardware has the flexibility to use less memory when less memory is required.\n[Here's a link to Tensorflow's information on the subject.](https://www.tensorflow.org/guide/mixed_precision#:~:text=Mixed%20precision%20is%20the%20use,faster%20and%20use%20less%20memory.&text=However%2C%20there%20are%20two%20lower,16%20bits%20of%20memory%20instead.)", "votes": null }, { "id": "1112757", "postDate": "12/14/2020 21:40:43", "content": "

Will we get speedup if using the GPU? This guide seems to suggest that the Kaggle GPU (P100) will not benefit from mixed precision?

\n

https://www.tensorflow.org/guide/mixed_precision

", "rawMarkdown": "Will we get speedup if using the GPU? This guide seems to suggest that the Kaggle GPU (P100) will not benefit from mixed precision?\n\nhttps://www.tensorflow.org/guide/mixed_precision", "votes": null } ], "comments": [ { "id": 1094934, "author_name": "maltonji", "author_url": "", "post_date": "11/29/2020 05:27:08", "content": "

Thanks for sharing! For anyone looking for a quick and simple definition of Mixed Precision Training, it's the use of both 16 and 32 bit floating point values during training. This means that your hardware has the flexibility to use less memory when less memory is required.
\nHere's a link to Tensorflow's information on the subject.

", "votes": null, "replies": [] }, { "id": 1112757, "author_name": "kagglethomas88", "author_url": "", "post_date": "12/14/2020 21:40:43", "content": "

Will we get speedup if using the GPU? This guide seems to suggest that the Kaggle GPU (P100) will not benefit from mixed precision?

\n

https://www.tensorflow.org/guide/mixed_precision

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1092076": "These are some resources on what Mixed Precision Training is and how it might be useful.\nThis will help you in faster training and decrease memory requirements.\n1. [Henry AI Labs](https://www.youtube.com/watch?v=pKZs4hllCvI)\n2. [Abhishek Thakur's video on using AMP with pytorch](https://www.youtube.com/watch?v=X7iOkhGePXg)\n3. [NVIDIA Blog](https://developer.nvidia.com/automatic-mixed-precision)\n4. [Original Paper](https://arxiv.org/pdf/1710.03740.pdf) \n\nHope this helps!", "1094934": "Thanks for sharing! For anyone looking for a quick and simple definition of Mixed Precision Training, it's the use of both 16 and 32 bit floating point values during training. This means that your hardware has the flexibility to use less memory when less memory is required.\n[Here's a link to Tensorflow's information on the subject.](https://www.tensorflow.org/guide/mixed_precision#:~:text=Mixed%20precision%20is%20the%20use,faster%20and%20use%20less%20memory.&text=However%2C%20there%20are%20two%20lower,16%20bits%20of%20memory%20instead.)", "1112757": "Will we get speedup if using the GPU? This guide seems to suggest that the Kaggle GPU (P100) will not benefit from mixed precision?\n\nhttps://www.tensorflow.org/guide/mixed_precision" }, "source": "meta" }