Step-by-step guide to build a simple neural network in PyTorch from scratch
PyTorch is one of the most used libraries for building deep learning models, especially neural network-based models. In many tasks related to deep learning, we find the use of PyTorch because of its features and capabilities like production-ready, distributed training, robust ecosystem, and cloud support. In this article, we will learn how we can build a simple neural network using the PyTorch library in just a few steps. For this purpose, we will demonstrate a hands-on implementation where we will build a simple neural network for a classification problem. We will accomplish this implementation in the following steps:- Step 1: Creating the dataStep 2: Loading the data using data loaderStep 3: Building a neural network model Defining the neural net classInstantiating the classifierStep 4: Training the neural network modelOptimizing loss curveDefining decision boundariesStep 5: Making Predictions Let’s start with the first step, where we will create a dataset for implementation. Step 1: Creating the data In this article, we will understand the intuition behind defining and working the neural networks in PyTorch. For this purpose, we will build a neural network from scratch and apply it to a classification problem. For this purpose, we will create a dataset using make_classification under sklearn.datasets which we use in making a classification data set. Using the following lines of code we can create the dataset for classification: from sklearn.datasets import make_classification X, Y = make_classification( n_features=4, n_redundant=0, n_informative=3, n_clusters_per_class=2, n_classes=3 ) Using the above code we have created a dataset for classification wherein the dataset we have 4 features with 3 informative features and 3 classes. Let’s visualize the dataset. import matplotlib.pyplot as plt plt.title("Multi-class data, 4 informative features, 3 classes", fontsize="large") plt.scatter(X[:, 0], X[:, 1], marker="o", c=Y, s=25, edgecolor="k") Here we can see our dataset in a two-dimensional space and points are clearly visible from three classes. After making the dataset, we are ready to build a classification model. Since in this article, we are discussing a simple implementation of a neural network using the PyTorch, we will use a two-layer neural network where we can use sigmoid as our activation function. Data points in the above graph will be our input coordinates and classes related to the dots are the ground truth. Step 2: Loading the data using data loader Before defining the model, we are required to split our dataset into tests and train sets. That can be done using the following lines of codes. from sklearn.model_selection import train_test_split X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.33, random_state=42) Let’s check the shape of the split dataset. X_train.shape, X_test.shape, Y_train.shape, Y_test.shape Output: After this, we are required to define the train data as a PyTorch tensor. import torch Y_test = torch.from_numpy(X_test) Y_test = torch.from_numpy(np.asarray(Y_test)) To use the batches of the dataset we are required to put the data through the Dataloader module of PyTorch. Using the below lines of code we can do that. from torch.utils.data import Dataset, DataLoader class Data(Dataset): def __init__(self): self.X=torch.from_numpy(X_train) self.Y=torch.from_numpy(Y_train) self.len=self.X.shape[0] def __getitem__(self,index): return self.X[index], self.Y[index] def __len__(self): return self.len data=Data() loader=DataLoader(dataset=data,batch_size=64) Checking the shape of the data under the tensor. print(data.X[0:5]) print(data.X.shape) print(data.Y[0:5]) print(data.Y.shape) Output: Step 3: Building a neural network model First of all, we will define the dimensions of the network. input_dim=4 # how many Variables are in the dataset hidden_dim = 25 # hidden layers output_dim=3 # number of classes Step 3.1: Defining the neural net class A class for creating simple networks can be defined using the following lines of codes. import torch.nn as nn class Net(nn.Module): def __init__(self,input,H,output): super(Net,self).__init__() self.linear1=nn.Linear(input,H) self.linear2=nn.Linear(H,output) def forward(self,x): x=torch.sigmoid(self.linear1(x)) x=self.linear2(x) return x Step 3.2: Instantiating the classifier Now with the above-defined dimensions, we can instantiate a model instance using the class defined in the last step. clf=Net(input_dim,hidden_dim,output_dim) Let’s check the information about the model print(clf.parameters) Output: Here we can see that the model which we have prepared will take 4 features as input and will give 4 features as output. Step 4: Training the neural network model Before training, we are required to define the criterion to calculate gradients of parameters and optimizers to update the parameters. criterion=nn.CrossEntropyLoss() optimizer=torch.optim.SGD(clf.parameters(), lr=0.1) After defining the criterion and optimizer, we are ready to train our model. Using the following lines of codes we can train it. learning_rate = 1e-1 loss_list = [] for t in range(1000): y_pred = clf(x) loss = criterion(y_pred, y) loss_list.append(loss.item()) clf.zero_grad() loss.backward() with torch.no_grad(): for param in clf.parameters(): param -= learning_rate * param.grad In this training, we have defined methods that will get the accumulated gradient to zero, append the loss on the loss_list, and will get a new gradient, and update parameters using the backward propagation. Step 4.1: Optimizing loss curve The loss curve will let us know about the performance of the model. Visualizing loss curve step = np.linspace(0,1000,1000) plt.plot(step,np.array(loss_list)) Output: In the above output, we can see that our loss curve converged, and using this model we can also define our decision boundaries. Step 4.2: Defining decision boundaries params = list(clf.parameters()) w = params[0].detach().numpy()[0] b = params[1].detach().numpy()[0] t= params[3].detach().numpy()[0] plt.scatter(X[:, 0], X[:, 1], c=Y,cmap='jet') u = np.linspace(X[:, 0].min(), X[:, 0].max(), 2) plt.plot(u, (0.5-b-w[0]*u)/w[1]) plt.plot(u, (0.5-t-w[0]*u)/w[1]) plt.xlim(X[:, 0].min()-0.5, X[:, 0].max()+0.5) plt.ylim(X[:, 1].min()-0.5, X[:, 1].max()+0.5) Output: In the above output, we can see that we have defined our decision boundaries and it is looking good. Now we can use the model for making predictions. Making predictions Let’s make the predictions with the trained neural network. x_val = torch.from_numpy(X_test) z=clf(x_val) yhat=torch.max(z.data,1) yhat[1] Output: In the above output, we can see the predictions of the class labels of the given input patterns. Using a few of the modifications like the number of layers or number of epochs we can improve the performance of our model. Final words In this article, we have learnt how to define a neural net in PyTorch in just a few steps. We have defined a dataset on which we have trained a double-layer neural network. We have used this network for classification and implemented that in PyTorch. This can be an example of a simple multiclass classification neural network also. References: PyTorch documentationsLink for the codes
In this article, we will learn how we can build a simple neural network using the PyTorch library in just a few steps. For this purpose, we will demonstrate a hands-on implementation where we will build a simple neural network for a classification problem.
We will demonstrate a Step-by-step Guide to Build a Simple Neural Network in PyTorch for multiclass classification from scratch
["AI (Artificial Intelligence)", "Data Science", "Data Scientist", "Deep Learning", "Machine Learning", "multiclass classification", "Pytorch"]
["Pytorch", "NumPy", "Go", "TPU", "AI", "neural network", "PyTorch", "Machine Learning", "Ray", "multiclass classification", "deep learning", "Matplotlib", "Deep Learning", "Data Science", "Data Scientist", "R", "AI (Artificial Intelligence)"]
["AI", "deep learning", "neural network", "Ray", "PyTorch", "NumPy", "Matplotlib", "TPU", "R", "Go"]
Yugesh is a graduate in automobile engineering and worked as a data analyst intern. He completed several Data Science projects. He has a strong interest in Deep Learning and writing blogs on data science and machine learning.