Why You Should Start Your Deep Learning Journey With PyTorch
There’s no denying the fact that Deep Learning as we know it, how awesome it is when we can see that with minimal or no human-intervention a job can be done. It’s almost creating your own self but for a super-specific task. Since, Machine Learning, Deep Learning is dubbed to be one of the sexiest jobs of the 21st century(hyped?) so there has to be some starting point, a sort of a roadmap that you can follow to reach to the other side. Luckily, we can now approach it relatively easier with modern frameworks like Tensorflow, PyTorch which gives you a high-level interface to build awesome stuff! Let’s discuss why you should start with PyTorch. 1. The Pythonic Nature One of the USP’s of the PyTorch is its pythonic nature. That means line-by-line execution of the code and simultaneous building of the computation graphs just like in python. This makes it quite readable and easy to debug which is what all Pythonistas want. 2. The Conceptual Help Although these frameworks are there in the first place to make your life easier, however, it’s also equally important to know the basics of Deep Learning, how it works, the secret ingredient, Backpropagation!! which tremendously increase your knowledge about Deep Neural Networks. With PyTorch, if your concepts aren’t crystal clear you’ll not be able to build things pretty well so by choosing PyTorch, not only you’ll learn it the hard way but with a stronger foundation. You must be thinking that Tensorflow can also do the same and I’m not opposing you but trust me you’ll wind up with Keras much often if you google stuff and you don’t want Keras at the start of your learning as it might spoil you with its easiness. 3. Dynamic Computation Graphs As discussed in the “Pythonic” section as well, PyTorch creates dynamic computation graphs just like python. This helps to comprehend the process better. Source: http://pytorch.org/about/ 4. Easy switch to Accelerators PyTorch comes with this awesome feature of utilizing CUDA cores and switching to available accelerators pretty quickly and easily. This will transfer the Tensor calculation to the GPU and make your computations faster. device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”) model = models.resnet18(pretrained=True) num_ftrs = model_ft.fc.in_features # Here the size of each output sample is set to 2. # Alternatively, it can be generalized to nn.Linear(num_ftrs, len(class_names)). model.fc = nn.Linear(num_ftrs, 2) # switching to Accelerator if available model = model.to(device) 5. Support for Data Parallelism PyTorch was built on the idea of using multi-core systems efficiently and it proudly delivers that. With the DataLoader mechanism in place which will handle large datasets with ease without hogging your system memory. PyTorch’s heart is in Parallelism!! data_transforms = { ‘train’: transforms.Compose([ transforms.ToTensor(), transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ]), ‘val’: transforms.Compose([ transforms.ToTensor(), transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ]),}data_dir = ‘data/hymenoptera_data’image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in [‘train’, ‘val’]}dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=4, shuffle=True, num_workers=4) for x in [‘train’, ‘val’]}dataset_sizes = {x: len(image_datasets[x]) for x in [‘train’, ‘val’]}class_names = image_datasets[‘train’].classes You can put your model to the distributed model like the snippet below and this also proves helpful for multi-GPU setup. import torch.distributed as distfrom torch.nn.parallel import DistributedDataParalleldist.init_process_group(backend=’gloo’)model = DistributedDataParallel(model) 6. The Ever Amazing TorchScript Python is known for many things like its readability, easiness to learn, and interpretable execution process but not performance as compared to compiled languages like C++, Java. Don’t worry PyTorch got you covered in this area as well!. Torch engineers suspected this way before and made a TorchScript which is PyTorch but in C++ and made it super-modular. Inferences in C++ makes 10x performance boosts You can build and train models in PyTorch employing the readability and relatively easy debugging process of Python and convert it to a TorchScript and make inferences in C++ which will be a significant boost in performance. You can build and train models in PyTorch as well. Let’s look at some examples of both: Converting PyTorch Model to TorchScript: # Reference: #TODO- Add Link from torchvision import models # Download and load the pre-trained model model = models.resnet18(pretrained=True) # Set upgrading the gradients to False for param in model.parameters(): param.requires_grad = False # Save the model except the final FC Layer resnet18 = torch.nn.Sequential(*list(resnet18.children())[:-1]) example_input = torch.rand(1, 3, 224, 224) script_module = torch.jit.trace(resnet18, example_input) script_module.save(‘resnet18_without_last_layer.pt’) Inference in TorchScript: torch::jit::script::Module module; // argv[1] should be the path to the model module = torch::jit::load(argv[1]); /* We need to convert last layer input and output features from (512, 1000) to (512, 2) since we only have 2 classes */ torch::nn::Linear linear_layer(512, 2); // Define the optimizer on parameters of linear_layer with learning_rate = 1e-3 torch::optim::Adam optimizer(linear_layer->parameters(), torch::optim::AdamOptions(1e-3)) 7. Deployment Serving Models has never been easier thanks to all the cloud services and a much simpler setup process. PyTorch supports a large band of devices like mobile (android) and web. A service as basic as Flask to using some sort of packaging mechanism like a Docker container. Then that service is often made network accessible via some sort of service, either using JSON over HTTP or an RPC technology like gRPC. PyTorch is Mobile ready! PyTorch comes with a support of ONNX (Open Neural Network Exchange) which helps you to export your models built in PyTorch to other mobile-friendly formats like Caffe2. Let’s see how you can export your model using ONNX support import torch.onnx import torchvision dummy_input = torch.randn(1, 3, 224, 224) model = torchvision.models.alexnet(pretrained=True) torch.onnx.export(model, dummy_input, “alexnet.onnx”) The following snippet is an example of utilizing torch models in android and ios. ## Save your model torch.jit.script(model).save(“my_mobile_model.pt”) ## iOS prebuilt binary pod ‘LibTorch’ ## Android prebuilt binary implementation ‘org.pytorch:pytorch_android:1.3.0’ ## Run your model (Android example) Tensor input = Tensor.fromBlob(data, new long[]{1, data.length}); IValue output = module.forward(IValue.tensor(input)); float[] scores = output.getTensor().getDataAsFloatArray(); You’ve reached the end!! Making your concepts crystal clear in one of the stepping stones to your successful career in Deep Learning. Despite any level of abstraction you got to learn the things from the ground level and PyTorch can help you understand gradient calculation like an expert, catering till the end of the development pipeline as well. If you are a beginner try focussing on math and concepts and simultaneously applying to small using PyTorch. This will help you get a good command of Tensors as well. Rest, the internet is filled with great tutorials and anyone can learn anything. Happy Learning!
There’s no denying the fact that Deep Learning as we know it, how awesome it is when we can see that with minimal or no human-intervention a job can be done. It’s almost creating your own self but for a super-specific task. Since, Machine Learning, Deep Learning is dubbed to be one of the sexiest […]
There’s no denying the fact that Deep Learning as we know it, how awesome it is when we can see that with minimal or no human-intervention a job can be
["trust your data"]
["machine learning", "Keras", "TPU", "AI", "neural network", "PyTorch", "docker", "Ray", "deep learning", "trust your data", "TensorFlow"]
["AI", "machine learning", "deep learning", "neural network", "Ray", "TensorFlow", "PyTorch", "Keras", "docker", "TPU"]
I share my journey of putting sense into data and understanding the bare bones of Machine Learning.