Deep Learning with PyTorch Lightning

Kunal Sawarkar

• Verlag: Packt Publishing
• Jahr: 2022
• ISBN:
  9781800569270
• Auflage: 1
• Thema: Computer Science
• Thema: Data Processing

Build, train, deploy, and scale deep learning models quickly and accurately, improving your productivity using the lightweight PyTorch WrapperKey Features• Become well-versed with PyTorch Lightning architecture and learn how it can be implemented in various industry domains• Speed up your research using PyTorch Lightning by creating new loss functions, networks, and architectures• Train and build new algorithms for massive data using distributed trainingBook DescriptionPyTorch Lightning lets researchers build their own Deep Learning (DL) models without having to worry about the boilerplate. With the help of this book, you'll be able to maximize productivity for DL projects while ensuring full flexibility from model formulation through to implementation. You'll take a hands-on approach to implementing PyTorch Lightning models to get up to speed in no time.You'll start by learning how to configure PyTorch Lightning on a cloud platform, understand the architectural components, and explore how they are configured to build various industry solutions. Next, you'll build a network and application from scratch and see how you can expand it based on your specific needs, beyond what the framework can provide. The book also demonstrates how to implement out-of-box capabilities to build and train Self-Supervised Learning, semi-supervised learning, and time series models using PyTorch Lightning. As you advance, you'll discover how generative adversarial networks (GANs) work. Finally, you'll work with deployment-ready applications, focusing on faster performance and scaling, model scoring on massive volumes of data, and model debugging.By the end of this PyTorch book, you'll have developed the knowledge and skills necessary to build and deploy your own scalable DL applications using PyTorch Lightning.What you will learn• Customize models that are built for different datasets, model architectures, and optimizers• Understand how a variety of Deep Learning models from image recognition and time series to GANs, semi-supervised and self-supervised models can be built• Use out-of-the-box model architectures and pre-trained models using transfer learning• Run and tune DL models in a multi-GPU environment using mixed-mode precisions• Explore techniques for model scoring on massive workloads• Discover troubleshooting techniques while debugging DL modelsWho this book is forThis deep learning book is for citizen data scientists and expert data scientists transitioning from other frameworks to PyTorch Lightning. This book will also be useful for deep learning researchers who are just getting started with coding for deep learning models using PyTorch Lightning. Working knowledge of Python programming and an intermediate-level understanding of statistics and deep learning fundamentals is expected.

Section 1: Kickstarting with PyTorch Lightning

This is the section for beginners that introduces the basics of PyTorch Lightning, starting with how to install it and build simple models. It will also show you how to get going quickly with the library of PyTorch Lightning Bolt state-of-the-art models out of the box.

This section comprises the following chapters:

• Chapter 1, PyTorch Lightning Adventure
• Chapter 2, Getting off the Ground with the First Deep Learning Model
• Chapter 3, Transfer Learning Using Pre-Trained Models
• Chapter 4, Ready-to-Cook Models from Lightning Flash

Chapter 1: PyTorch Lightning Adventure

Welcome to the world of PyTorch Lightning!!

We are witnessing what is popularly referred to as the Fourth Industrial Revolution, driven by Artificial Intelligence (AI). Since the creation of the steam engine some 350 years ago, which set humanity on the path to industrialization we saw another two industrial revolutions. We saw electricity bringing a sea change roughly 100 years ago, followed by the digital age some 50 years later revolutionizing the way we live our lives today. There is an equally transformative power in AI. Everything that we know about the world is changing fast and will continue to change at a pace that no one imagined before and certainly no one planned for. We are seeing transformational changes in how we contact customer services, with the advent of AI-powered chatbots; in how we watch movies/videos, with AI recommending what we should watch; in how we shop, using algorithms optimized for supply chains; in how cars are driven, using self-driving technology; in how new drugs are developed, by applying AI to complex problems such as protein folding; in how medical diagnoses are being carried out, by finding hidden patterns in massive amounts of data. Underpinning each of the preceding technologies is the power of AI. The impact of AI on our world is more than just the technology that we use; rather, it is much more transformational in terms of how we interact with society, how we work, and how we live. As many have said, AI is the new electricity, powering the engine of the 21st century.

And this monumental impact of AI on our lives and psyche is the result of a recent breakthrough in the field of Deep Learning (DL). It had long been the dream of scientists to create something that mimics the brain. The brain is a fascinating natural evolutionary phenomenon. A human brain has more Synapses than stars in the universe, and it is those neural connections that make us intelligent and allow us to do things such as think, analyze, recognize objects, reason with logic, and describe our understanding. While Artificial Neural Networks (ANNs) do not really work in the same way as biological neurons, they do serve as inspiration.

In the evolution of species, the earliest creatures were unicellular (such as amoeba), first appearing around 4 billion years ago, followed by small multi-cellular species that navigated blindly with no sense of direction for about 3.5 billion years. When everyone around you is blind, the first species that developed vision had a significant advantage over all other species by becoming the most intelligent species, and in evolutionary biology, this step (which happened some 500 million years ago) is known as the Cambrian explosion. This single event led to remarkable growth in the evolution of species, resulting in everything that we currently see on earth today. In other words, though Earth is about 4.5 billion years old, all the complex forms of life, including human brains, evolved in just the last 500 million years (which is in just 10% of Earth's lifetime), led by that single evolutionary event, which in turn led to the ability of organisms to "see" things.

In fact in humans as much 1/3 of our brain is linked to visual cortex; which is far more than any other senses. Perhaps explaining how our brain evolved to be most intelligence by first mastering "vision" ability.

With DL models of image recognition, we can finally make machines "see" things (Fei Fei Li has described this as the Cambrian explosion of Machine Learning (ML)), an event that will put AI on a different trajectory altogether, where one day it may really be comparable to human intelligence.

In 2012, a DL model achieved near-human accuracy in image recognition, and since then, numerous frameworks have been created to make it easy for data scientists to train complex models. Creating Feature Engineering (FE) steps, complex transformations, training feedback loops, and optimization requires a lot of manual coding. Frameworks help to abstract certain modules and make coding easier as well standardized. PyTorch Lightning is not just the newest framework, but it is also arguably the best framework that strikes the perfect balance between the right levels of abstraction and power to perform complex research. It is an ideal framework for a beginner in DL, as well as for professional data scientists looking to productionalize a model. In this chapter, we will see why that is the case and how we can harness the power of PyTorch Lightning to build impactful AI applications quickly and easily.

In this chapter, we will cover the following topics:

• What makes PyTorch Lightning so special?
• <pip install>—My Lightning adventure
• Understanding the key components of PyTorch Lightning
• Crafting AI applications using PyTorch Lightning

What makes PyTorch Lightning so special?

So, if you are a novice data scientist, the question on your mind would be this: Which DL framework should I start with? And if you are curious about PyTorch Lightning, then you may well be asking yourself: Why should I learn this rather than something else? On the other hand, if you are an expert data scientist who has been building DL models for some time, then you will already be familiar with other popular frameworks such as TensorFlow, Keras, and PyTorch. The question then becomes: If you are already working in this area, why switch to a new framework? Is it worth making the effort to learn something different when you already know another tool? These are fair questions, and we will try to answer all of them in this section.

Let's start with a brief history of DL frameworks to establish where PyTorch Lightning fits in this context.

The first one….

The first DL model was executed in 1993 in Massachusetts Institute of Technology (MIT) labs by the godfather of DL, Ya...