Member-only story
How To Train Your PyTorch Models (Much) Faster
Tips and tricks I learnt while working with the best in the industry
My articles are free to read for everyone. If you don’t have a Medium subscription, read this article by following this link.
Training deep learning models can sometimes feel like watching paint dry. Sometimes iterations are so slow that you find yourself banging your head against the keyboard, frustrated as you watch each epoch crawl by on your terminal. And that’s when you wonder:
Is there a better way to do this?
Well, fret not! I’m back with another article sharing some tips/tricks making your deep learning workflow much more agile. I’ll share straightforward and clever coding tweaks that help you harness your computational resources more effectively. We’ll also dive into a host of industry-tested tips, tricks and tweaks that can extract every bit of performance from your hardware, so you can focus less on waiting and more on faster iterations.
1. Enable Automatic Mixed Precision Training
If your GPU supports mixed precision training (think AMD/NVidia GPUs), PyTorch makes it incredibly easy and straightforward to enable it in your training schedule. Mixed precision training uses a mix of…
