PyTorch Releases v1.9: All New Features &#038; Updates
Open-source machine learning framework PyTorch has released its version 1.9 just two months after the release of version 1.8. The new release consists of 3,400 new commits made by 398 contributors. Highlights: Improvements to functions such as torch.linalg, torch.special, and Complex Autograd to support scientific computingBeta release of Mobile Interpreter to execute PyTorch programs in edge devicesUpstreaming of TorchElastic into Pytorch CoreMajor updates to the PyTorch RPC frameworkNew APIs for performance optimizationSupport for distributed training, GPU utilization, and SM efficiency in the PyTorch Profiler. https://twitter.com/PyTorch/status/1404838218277740549?s=20 Increased support for scientific computing In the new version, torch.linalg has been moved to a stable release. This module extends PyTorch support for linear algebra by implementing several functions such as NumPy’s linear algebra module, torch.linalg.matrix_norm and torch.linalg.householder_product. The Complex Autograd feature has also been promoted to a stable state after being released as a beta in v1.8. In the latest release, the team has extended support for Complex Autograd for 98 percent operators, added more OpInfos, and TorchAudio has been migrated to native complex tensors. Complex Autograd provides users the ability to calculate complex gradients and work with complex variables. It is an important feature for users who need to work with complex numbers in PyTorch toolkits such as TorchAudio, ESPNet, FastMRI, and Asteroid. PyTorch 1.9 also includes torch.use_determinstic_algorithms option for debugging and writing reproducible programs. The torch.special module, which contains functions for scientific computing and helps in working with distributions like iv, ive, erfcx, logerfc, and logerfcx, has been made available in beta. Along with torch.special module, nn.Model parameterization is also now available in beta. It helps to reduce the space required for parameters without using any special optimization methods. PyTorch Mobile Mobile Interpreter in beta to execute PyTorch programs in edge devices is one of the major announcements. Mobile Interpreter will significantly reduce the binary size as compared to the current on-device runtime. “The current pt size with MobileNetV2 in arm64-v8a Android is 8.6 MB compressed and 17.8 MB uncompressed. Using Mobile Interpreter, we are targeting at the compressed size below 4 MB and uncompressed size below 8MB,” the team said in the blog. PyTorch is releasing a new PyTorch Video library-based video app and an update on the speech recognition app based on torchaudio and wave2vec model. With the addition of these two apps, PyTorch now offers a full demo apps suite that cover image, text, audio, and video. Distributed training TorchElastic is a coordinator for PyTorch worker processes that handles scaling events. It was open-sourced more than a year ago and has been used in various distributed torch use-cases such as deepspeech.pytorch, pytorch-lighting, and Kubernetes CRD. In v1.9, TorchElastic has been made part of PyTorch core. Other beta features made available in the distributed package include–CUDA support in RPC, ZeroRedundancyOptimizer, and support for profiling distributed collectives. Performance optimization Module Freezing is a process for optimising and specialising programs; freezing is a recommended practice for model deployment. It inlines module parameters and attribute values as constants into the TorchScript internal representation. PyTorch Profiler, Inference mode API, and torch.package have been graduated to beta. Other updates Along with the release of PyTorch v1.9, the team also announced updates to a couple of PyTorch libraries which include new releases for domain libraries such as TorchVision, TorchText, and TorchAudio. Highlights: TorchVision: Addition of new SSD and SSDLite models, quantized kernels for object detection, and iOS supportTorchAudio: Several new additions including wav2vec 2.0 model, options for quality control in sampling, and performance improvements in spectral operations and resampling.TorchText – Introduction of a new high-performance Vocab module for common functional APIs for NLP workflows.
Open-source machine learning framework PyTorch has released its version 1.9 just two months after the release of version 1.8. The new release consists of 3,400 new commits made by 398 contributors.  Highlights: Improvements to functions such as torch.linalg, torch.special, and Complex Autograd to support scientific computing Beta release of Mobile Interpreter to execute PyTorch programs […]
The Complex Autograd feature has been promoted to a stable state after being released as a beta in v1.8.
["APIs", "Pytorch", "updates"]
["CUDA", "Pytorch", "NumPy", "Go", "machine learning", "AI", "PyTorch", "APIs", "NLP", "object detection", "updates", "R", "kubernetes"]
["AI", "machine learning", "NLP", "PyTorch", "NumPy", "object detection", "kubernetes", "CUDA", "R", "Go"]
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at shraddha.goled@analyticsindiamag.com.