Guide To TensorLy: A Python Library For Tensor Learning
TensorLy is an open-source Python library that eases the task of performing tensor operations. It provides a high-level API for dealing with deep tensorized neural networks and tensor methods. It was created in 2015 by a senior research scientist at NVIDIA Research Group named Jean Kossaifi. It was presented at the NeurIPS workshop and was later published byJean Kossaifi, Yannis Panagakis, Anima Anandkumar and Maja Pantic in a JMLR paper in February 2019. The project is supported by the following universities and organizations: NVIDIAINRIAImperial College, LondonCaltechNational and Kapodistrian University of Athens TensorLy makes it easy to handle tensor decomposition, tensor learning and tensor algebra. It’s robust backend system enables users to perform computations with NumPy, TensorFlow, PyTorch, MXNet, CuPy or JAX. It can run methods that can scale on multiple CPUs or GPUs. TensorLy was initially built on the top of NumPy and SciPy with a soft dependency on plotting library Matplotlib. A backend system was added to it later on for combining tensor methods with deep learning and for enabling transparent switching between various libraries and platforms.. It provides a simple and efficient API such as that of scikit-learn. However, scikit-learn deals with observations in terms of vectors whereas TensorLy represent them as higher-order arrays called tensors. The methods, functionalities and libraries supported be summarized in the following diagrams: Installation of TensorLy NOTE: TensorLy has been developed and tested only on Python 3 version. Install the library using pip command: pip install -U tensorly (option -U above is optional; use it for packages updates) Install it using conda from tensorly channel: conda install -c tensorly tensorly Practical implementation Here, we demonstrate how to use this library for basic tensor operations, tensor decomposition and tensor regression. Step-wise explanation for code of each of the use cases is as follows: Some basic tensor operations Import required libraries import numpy as np import tensorly as tl from tensorly.testing import assert_array_equal #assert_array_equal() raises an error if two array-like objects are unequal Create a tensor (it is nothing but a numpy array) orig_tensor = tl.tensor(np.arange(40).reshape((5, 2, 4))) A 3D array containing elements from 0 to 39 will be created’ Print the created tensor orig_tensor = tl.tensor(np.arange(40).reshape((5, 2, 4))) print('Original tensor:\n{}'.format(orig_tensor)) Output: Unfolding the tensor for m in range(tensor.ndim):  #for each dimension of the tensor print('Mode-{} unfolding:\n{}'.format(m, tl.unfold(tensor, m))) unfold() method performs unfolding starting from mode 0, and unfolds till (n-1)th dimension (where, ‘n’ is the dimension of the tensor). Refolding the tensor for mode in range(tensor.ndim):  #for each dimension of the tensor unf = tl.unfold(tensor, mode) #unfold the tensor first f = tl.fold(unf, mode, tensor.shape) #re-fold the tensor #give an error if the refolded tensor and original tensor are unequal assert_array_equal(f, tensor) The unfolded tensor after executing above lines of code: The re-folded tensor will be as follows: Tensor decomposition example Here, we demonstrate compression of colored image of a raccoon using parafac decomposition and Tucker decomposition techniques. Install required libraries import matplotlib.pyplot as plt import tensorly as tl import numpy as np from scipy.misc import face #for the raccoon’s face image #to zoom the image represented in the form of an array from scipy.ndimage import zoom from tensorly.decomposition import parafac from tensorly.decomposition import tucker from math import ceil Load the raccoon face image img = face() #Represent the image as a tensor img = tl.tensor(zoom(face(), (0.3, 0.3, 1)), dtype='float64') Define a function to convert image tensor’s datatype from float to unsigned integer def to_img(tensor): image = tl.to_numpy(tensor) #Convert the tensor to a numpy array #Subtract minimum element from the array from each of the elements image -= image.min() #Divide the modified elements by the maximum array element image /= image.max() #Multiply the elements by 255 (0-255 is pixel range for colored images) image *= 255 #Change the datatype to unit8 (unsigned integer) return image.astype(np.uint8) Define ranks of decomposition # Rank of CP decomposition cp_rank = 25 # Rank of Tucker decomposition tucker_rank = [100, 100, 2] ‘Rank’ here refers to the number of directions required to describe it. Perform CP decomposition weights, factors = parafac(img, rank=cp_rank, init='random', tol=10e-6) # Reconstruct the image from the factors cp_rec = tl.cp_to_tensor((weights, factors)) Perform Tucker decomposition core, factors = tucker(img, rank=tucker_rank, init='random', tol=10e-5, random_state=12345) #Convert the tucker tensor into a full tensor tucker_rec = tl.tucker_to_tensor((core, factors)) Plot the original image fig = plt.figure() ax = fig.add_subplot(1, 3, 1) #arguments represent (number of rows, number of columns, index) ax.set_axis_off() #Turn the X and Y axes off ax.imshow(to_img(img))  #Display the plot ax.set_title('Original Image') #Title of the plot Similarly, plot the image compressed using CP decomposition and Tucker decomposition. ax = fig.add_subplot(1, 3, 2) ax.set_axis_off() ax.imshow(to_img(cp_rec)) ax.set_title('CP decomposition') ax = fig.add_subplot(1, 3, 3) ax.set_axis_off() ax.imshow(to_img(tucker_rec)) ax.set_title('Tucker Decomposition') #Adjust the padding between and around the subplots plt.tight_layout() #Display the plots plt.show() Output: Tensor regression example Import required libraries import matplotlib.pyplot as plt from tensorly.base import tensor_to_vec, partial_tensor_to_vec from tensorly.datasets.synthetic import gen_image from tensorly.random import check_random_state from tensorly.regression.cp_regression import CPRegressor import tensorly as tl Set image parameters img_ht = 25 #height of the image img_width = 25 #width of the image # shape of the images patterns = ['swiss', 'circle'] # ranks for testing ranks = [1, 2, 3, 4, 5] Generate random samples r = check_random_state(1) #Tensor representation of the samples X = tl.tensor(r.normal(size=(1000, img_ht, img_width), loc=0, scale=1)) #numpy.random.RandomState.normal() draws random samples from a normal distribution Parameters of the plot n_rows = len(patterns) n_columns = len(ranks) + 1 Plot the images before and after applying regression fig = plt.figure() for i, pattern in enumerate(patterns): # Generate the original image for regression testing wt_img = gen_image(region=pattern, image_height=img_ht, image_width=img_width) wt_img = tl.tensor(wt_img) #Convert the image into tensor # Generate the labels y = tl.dot(partial_tensor_to_vec(X, skip_begin=1), tensor_to_vec(wt_img)) # Plot the original weights ax = fig.add_subplot(n_rows, n_columns, i*n_columns + 1) ax.imshow(tl.to_numpy(wt_img), cmap=plt.cm.OrRd, interpolation='nearest') ax.set_axis_off() if i == 0: ax.set_title('Original\nweights') for j, rank in enumerate(ranks): # Create a tensor Regressor estimator est = CPRegressor(weight_rank=rank, tol=10e-7, n_iter_max=100, reg_W=1, verbose=0) # Fit the estimator to the data est.fit(X, y) #Plot the images after regression ax = fig.add_subplot(n_rows, n_columns, i*n_columns + j + 2) ax.imshow(tl.to_numpy(est.weight_tensor_), cmap=plt.cm.OrRd, interpolation='nearest') ax.set_axis_off() if i == 0: ax.set_title('Learned\nrank = {}'.format(rank)) plt.suptitle("CP tensor regression") #Add a centered title to the figure plt.show() #Display the plot Output: Google colab notebooks of the above implementations: Basic operationsDecompositionRegression References Refer to the following sources to get an in-depth understanding of TensorLy Python library: Research paperOfficial websiteGitHub repository
TensorLy is an open-source Python library that eases the task of performing tensor operations. It provides a high-level API for dealing with deep tensorized neural networks and tensor methods. It was created in 2015 by a senior research scientist at NVIDIA Research Group named Jean Kossaifi. It was presented at the NeurIPS workshop and was […]
TensorLy is an open-source Python library which provides high-level API for dealing with deep tensorized neural networks and tensor methods.
["Python Library"]
["scikit-learn", "AI", "neural network", "PyTorch", "ML", "Ray", "Aim", "deep learning", "JAX", "Python Library", "TensorFlow"]
["AI", "ML", "deep learning", "neural network", "Aim", "Ray", "TensorFlow", "PyTorch", "JAX", "scikit-learn"]
A zealous learner aspiring to advance in the domain of AI/ML. Eager to grasp emerging techniques to get insights from data and hence explore realistic Data Science applications as well.