Member-only story
How To Train Your PyTorch Models With Less Memory
Strategies I regularly use to reduce GPU memory consumption by almost 20x
My articles are free to read for everyone. If you don’t have a Medium subscription, read this article by following this link.
One of the most common bottlenecks when training large deep learning models (yeah, including those fancy LLMs and vision transformers) is reaching peak memory consumption. Since most people don’t have access to fancy GPU clusters, or deep learning rigs with seemingly unlimited GPU memory, in this article I’ll be outlining some techniques and strategies to reduce memory consumption by almost 20x without sacrificing modeling performance and prediction accuracy. Keep in mind — most of these techniques aren’t mutually exclusive, and can be easily combined for greater memory efficiency!
1. Automatic Mixed-Precision Training
As described in one of my previous articles, one of the easiest way to reduce memory footprint is mixed precision training. If you haven’t checked that article out yet, give it a read!
