Everything You Need To Know About Facebook’s Deep Learning Library PyTorchVideo
Deep video understanding is one of the most challenging tasks in computer vision. With the rise of computing power and the amount of video data on the internet, the demand for new-age machine learning models and tools continues to grow. As per Stanford University, technologies used to develop object detection from videos are maturing rapidly. Facebook AI recently unveiled a new deep learning library for video understanding called PyTorchVideo. The source code is available on GitHub. With PyTorchVideo, Facebook aims to help researchers develop cutting-edge machine learning models and tools to enhance video understanding capabilities, alongside providing a unified repository of reproducible and efficient video understanding components for research and production applications. In addition to this, Facebook is looking to standardise video-focused libraries that serve various video use cases in one place. “This has created a barrier for developers looking to work with videos for the first time,” said Facebook AI, stating that lack of standardisation makes it difficult to collaborate and spur innovation. In the coming months, Facebook will improve the PyTorchVideo library to enable and support more groundbreaking research in video understanding. “We welcome contributions from the entire community. All our efforts will be directed at supporting the rich open-source community committed to pushing the boundaries of video research,” said Facebook. PyTorchVideo: In a nutshell Today, the PyTorchVideo library supports components that can be used for various video understanding applications, including video classification, self-supervised learning, detection, and optical flow, among others. The video understanding library supports other modalities, including audio and text. It is not just limited to desktop devices; its Accelerator package also provides mobile hardware-specific optimization and model deployment flow. Some of the core features of PyTorchVideo include: Enables researchers to build new video architectures using its video models and pretrained weights with customizable components It consists of a set of downstream tasks like action classification, action detection, acoustic event detection and self-supervised learning (SSL)Supports a wide range of datasets and tasks for benchmarking various video models under different evaluation protocols Promotes hardware-aware model design and full-speed on-device model execution using efficient building blocks and deployment flow optimized for inference on hardware like mobile devices, Intel NNPI, etc. Offers access to a growing toolkit of standard scripts for video processing, including tracking, decoding and optimal flow extracting At present, PyTorch Video is being used by Facebook AI for various research works, including: SlowFast networks for video recognition Audiovisual SloFast networks for video recognitionX3D: Expanding architectures for efficient video recognitionVideo classification with channel-separated convolutional networks Also, it has been used to fuel recent advances in video transformers and self-supervised learning, including: Multiscale vision transformersA large-scale study on unsupervised spatiotemporal representation learningMultiview pseudo-labelling for semi-supervised learning from videoUnidentified video objects: A benchmark for dense, open-world segmentation There is a shortage of open-source codes and libraries for developing video understanding tools and models, which makes it difficult for researchers to exchange ideas, compare notes and accelerate innovation in the space. Facebook AI’s PyTorchVideo can bolster innovation in the video understanding space, going beyond the realm of deep fakes and synthetic media propaganda.
Deep video understanding is one of the most challenging tasks in computer vision. With the rise of computing power and the amount of video data on the internet, the demand for new-age machine learning models and tools continues to grow. As per Stanford University, technologies used to develop object detection from videos are maturing rapidly. […]
Facebook looks to enhance the PyTorchVideo library to enable and support more groundbreaking research in video understanding.
[]
["Go", "machine learning", "AI", "PyTorch", "Transformers", "computer vision", "Aim", "deep learning", "object detection", "R"]
["AI", "machine learning", "deep learning", "computer vision", "Aim", "PyTorch", "Transformers", "object detection", "R", "Go"]
Amit Raja Naik, known as the 'AI Human,' serves as the editor at AIM Media House, where he leads a team of talented tech journalists who are driving and shaping technology conversations across India and around the world.