Kornia: An OpenCV-inspired PyTorch Framework
Kornia is an open-source Python library inspired by OpenCV designed to handle generic Computer Vision tasks. It was introduced by Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee and Gary Bradski in October, 2019 (research paper). Kornia leverages PyTorch library at its backend in terms of model’s efficiency and reverse-mode auto-differentiation for defining and computing complex functions’ gradients. It comprises a subset of packages having operators that act as an input to neural networks for performing a wide range of tasks such as image transformations, depth estimation, epipolar geometry, filtering and edge-detection applicable on high-dimensional tensors. Unlike conventional CPU-based CV libraries such as torchvision and scikit-image, several standard deep learning functions can be implemented on GPUs using Kornia. Kornia bridges the gap between two simple yet powerful libraries namely, OpenCV and PyTorch. Though solely based on traditional CV solutions like torchvision, tf.image, PIL and skimage, it enables differentiable programming for CV applications by utilizing the crucial properties of PyTorch like GPU hardware acceleration, differentiability and distributed data-flows. The following table compares Kornia with some important CV libraries/modules: Library/ModuleCPU supportedGPU supportedBatch ProcessingDifferentia-bleDistributedSupports multidimensional arraytorchvisionYesNoNoNo NoNoscikit-imageYesNoNoNoNoNoopencvYesYesNoNoNoNotf.imageYesYesYesYesYesYesKorniaYesYesYesYesYesYes Source of above summary: Research paper Components of the library kornia.augmentation – module for GPU-based image augmentation kornia.utils – image to tensor conversion utilities and metrics kornia.color – module for color space conversions kornia.morphology – module for morphological image transformations kornia.losses – collection of several loss functions kornia.geometry – for geometric CV tasks like image conversion using various camera models kornia.enhance – module for intensity transformations and normalization kornia.filters – for edge detection and image filtering use cases kornia.feature – for feature setection Practical implementation Here, we demonstrate three use cases of Kornia – blurring a custom image, changing its color space, and adjusting its colors. The code is test on Google colab with Python 3.7.10 and Kornia 0.4.1 versions. We have used the following image for demonstration: Image source Step-wise explanation of the code is as follows: First, install the library using pip command. !pip install kornia Import required libraries and modules import torch import torchvision import kornia import cv2 import numpy as np import matplotlib.pyplot as plt Image blurring using Kornia Read the input image #Read the image using OpenCV and convert it to a numpy array input_image: np.ndarray = cv2.imread('img2.jpg') #Convert the image color space from BGR to RGB input_image = cv2.cvtColor(input_image, cv2.COLOR_BGR2RGB) Convert the image to a torch 4D tensor tensor_image: torch.tensor = kornia.image_to_tensor(input_image, keepdim=False) Create an operator that blurs the tensor image using Gaussian filter gaussian = kornia.filters.GaussianBlur2d((11, 11), (10.5, 10.5)) Where, (11,11) is the size of the kernel and (1.05,10.5) is the standard deviation of the kernel Convert the tensor image to float type and apply the gaussian operator defined in previous step to blur the image blur_image: torch.tensor = gaussian(tensor_image.float()) Convert the blurred tensor image back to a numpy array final_blur_image: np.ndarray = kornia.tensor_to_image(blur_image.byte()) Plot the input and output images fig, ax = plt.subplots(1, 2, figsize=(16, 10)) ax = ax.ravel() #flatten the axis to a 1D array #Original image ax[0].axis('off')  #turn off the axis lines and labels ax[0].set_title('Original image')  #Title of image ax[0].imshow(input_image)   #Display the image #Blurred image ax[1].axis('off')    #turn off the axis lines and labels ax[1].set_title('Blurred image')   #Title of image ax[1].imshow(final_blur_image)  #Display the image Output: Color space conversion using Kornia Read the input colored image and convert it to a numpy array bgr_image: np.ndarray = cv2.imread('img2.jpg', cv2.IMREAD_COLOR) Convert the image to a torch tensor tensor_bgr: torch.Tensor = kornia.image_to_tensor(bgr_image, keepdim=False) Define functions to flip the image horizontally and vertically and rotate it by 180 degrees. def horizontal_flip(input: torch.Tensor) -> torch.Tensor: return torch.flip(input, [-1]) #torch.flip() reverses the order of the input tensor image along given axis def vertical_flip(input: torch.Tensor) -> torch.Tensor: return torch.flip(input, [-2]) def rotate_180(input: torch.Tensor) -> torch.Tensor: return torch.flip(input, [-2, -1]) Define a function to plot a batch of images def imshow(input: torch.Tensor): #Create a grid with 2 rows output: torch.Tensor = torchvision.utils.make_grid(input, nrow=2, padding=5) #Convert the grid of images to numpy ndarray output_np: np.ndarray = kornia.tensor_to_image(output) plt.imshow(output_np)  #Plot the grid plt.axis('off')  #Turn off the axis lines and labels plt.show()   #Display the grid Create a batch of images from BGR image #Apply the functions defined in step (3) to the input tensor image and then #concatenate resulting tensors batch_bgr = torch.cat([tensor_bgr, horizontal_flip(tensor_bgr), vertical_flip(tensor_bgr), rotate_180(tensor_bgr)]) #Display the batch of images imshow(batch_bgr) Output: BGR to RGB color space conversion batch_rgb = kornia.bgr_to_rgb(batch_bgr)  #conversion imshow(batch_rgb) #display the resulting Output: RGB to Grayscale color space conversion batch_gray = kornia.rgb_to_grayscale(batch_rgb.float() / 255.) imshow(batch_gray)  #display the grayscale image Output: RGB to HSV (Hue, Saturation, Value) color space conversion batch_hsv = kornia.rgb_to_hsv(batch_rgb.float() / 255.) imshow(batch_hsv[:, 2:3])  #display HSV image Output: Color adjustment Read the input image in BGR format bgr_image: np.ndarray = cv2.imread('img2.jpg', cv2.IMREAD_COLOR) Convert input image to torch tensor tensor_bgr: torch.Tensor = kornia.image_to_tensor(bgr_image) Convert the image from BGR to RGB tensor_rgb: torch.Tensor = kornia.bgr_to_rgb(tensor_bgr) Expand the image tensor_rgb = tensor_rgb.expand(4, -1, -1, -1) # 4 x Channels x Height x Width tensor_rgb = tensor_rgb.float() / 255. #Normalize the expanded image Define a function to create a batch of images def imshow(input: torch.Tensor): #create grid having 2 rows of images output: torch.Tensor = torchvision.utils.make_grid(input, nrow=2, padding=5) #Convert tensor images to numpy ndarray output_np: np.ndarray = kornia.tensor_to_image(output) #Plot the grid of images plt.imshow(output_np) plt.axis('off') plt.show() Display the batch of RGB images imshow(tensor_rgb) Output: Adjust brightness of the batch images tensor_brightness: torch.Tensor = kornia.adjust_brightness(tensor_rgb, 0.6) #Where, 0.6 is the factor to adjust brightness of each element in the batch imshow(tensor_brightness)  #display the resulting batch Output: Adjust contrast of the batch images tensor_contrast: torch.Tensor = kornia.adjust_contrast(tensor_rgb, 0.2) #0.2 is the contrast adjustment factor per batch element imshow(tensor_contrast)  #dispaly the resulting batch Output: Adjust gamma correction of the images tensor_gamma: torch.Tensor = kornia.adjust_gamma(tensor_rgb, gamma=3., gain=1.5) #where, ‘gamma’ is a non-negative number and ‘gain’ is the constant multiplier imshow(tensor_gamma)  #display resulting batch Output: Adjust saturation level of the batch images tensor_saturated: torch.Tensor = kornia.adjust_saturation(tensor_rgb, 0.2) #where 0.2 is the saturation factor imshow(tensor_saturated)  #display the resulting batch Output: Adjust hue of the images tensor_hue: torch.Tensor = kornia.adjust_hue(tensor_rgb, 0.5) #where 0.5 gives measure of how much to shift the hue channel imshow(tensor_hue)  #display the resulting batch Output: Google colab notebook of the above-explained implementation is available here. References To get in-depth understanding of the Kornia library, refer to the following web links: Research paper Official documentation GitHub repository
Kornia is an open-source Python library inspired by OpenCV designed to handle generic Computer Vision tasks. It was introduced by Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee and Gary Bradski in October, 2019 (research paper).  Kornia leverages PyTorch library at its backend in terms of model’s efficiency and reverse-mode auto-differentiation for defining and computing […]
Kornia is an open-source Python library inspired by OpenCV designed to handle generic Computer Vision tasks.
["Computer Vision", "Guide"]
["NumPy", "AI", "neural network", "PyTorch", "computer vision", "OpenCV", "Colab", "Ray", "deep learning", "Computer Vision", "Matplotlib", "Guide"]
["AI", "deep learning", "neural network", "computer vision", "Ray", "PyTorch", "Colab", "OpenCV", "NumPy", "Matplotlib"]
A zealous learner aspiring to advance in the domain of AI/ML. Eager to grasp emerging techniques to get insights from data and hence explore realistic Data Science applications as well.