What is Pytorch ?
Pytorch is an open source machine learning framework which is based on the Torch library. This framework is widely used in deep learning applications like Convolutional Neural Networks for Computer vision, RNN, etc. Even tesla uses this in their cars. It was created by Facebook or META AI specifically for python users.
Seeing pytorch in action
So without further ado, lets see how we can use it in python for creating our neural networks. I hope you are familiar with machine learning basics. If not here is something you might wanna see
Importing libraries and pytorch
import torch as t import matplotlib.pyplot as mp import numpy
NOTE: The library torch is pytorch itself . We are also going to matplotlib and numpy. Some of you might know, these are widely used in the context of machine learning. Numpy is used for handling numerical values and matplotlib for visualizing the accuracy of our model.
Data preprocessing
Try to go through each line carefully. If you don’t understand what functions we calling, just refer to the pytorch site
# data
data = datasets.load_wine()
input, target = data.data, data.target
# data
# data preprocessing
input_ = t.from_numpy(input.astype(np.float32))
target_ = t.from_numpy(target.astype(np.float32))
print(target_.shape)
target_ = target_.view(target_.shape[0], -1)
from torch.utils.data import TensorDataset, DataLoader, random_split
ds = TensorDataset(input_, target_)
train, val, test_ = random_split(ds, [107, 36, 35])
batch_size = 15 #h1
train = DataLoader(train, batch_size=batch_size, shuffle=True)
val = DataLoader(val, batch_size=batch_size)
test = DataLoader(test_, batch_size=batch_size)
We are going to use the classic wine dataset by importing it with a library called sklearn. Hence it is a classification problem
As you might notice we used different things like TensorDataset, DataLoader, random_split. These are used to transform our data into somewhat a more usable format. Notice how we are spitting our data into test training and validation sets. And then also making batches out of them of size 15. here batch_size can be considered a hyperparameter.
Model
from torch.nn.functional import softmax
class Logistic_regression(t.nn.Module):
def __init__(self):
super().__init__()
self.linear = t.nn.Linear(input_.shape[1], 3)
def forward(self, x):
x = self.linear(x)
return x
def predict(self, x):
x = self.forward(x) # layer 1 to layer 2
x = t.nn.functional.softmax(x, dim=0) # activation function 1 / layer 2
max_probs, preds = t.max(x, dim=0)
return preds
def multi_predict(self, x):
x = t.nn.functional.softmax(self.forward(x), dim=1)
# layer 2 # for multi predction
max_probs, preds = t.max(x, dim=1)
return preds
def accuracy(self, x, y):
x = self.multi_predict(x)
y = y.reshape([y.shape[0]]).to(t.int64)
return t.sum(x == y).item() / len(x)
model = Logistic_regression()
loss_fn = t.nn.functional.cross_entropy
lr = 1e-3 # h2
opt = t.optim.SGD(model.parameters(), lr=lr)
Since it is a classification problem we are going to use something called softmax function as our loss function. We have made different functions for different purposes.
Training
We are going to store each training loop number and its accuracy in list x and y respectively.
x_axis = []
y_axis = []
def fit(num_epochs):
for epoch in range(num_epochs):
for i, j in train:
pred = model.forward(i)
j = j.reshape([j.shape[0]]).to(t.int64)
loss = loss_fn(pred, j)
loss.backward()
opt.step()
opt.zero_grad()
if (epoch+1) % 10 == 0:
x_axis.append(epoch+1)
acc_per_batch = []
for a, b in val:
acc_ = model.accuracy(a, b)
acc_per_batch.append(acc_)
y_axis.append(sum(acc_per_batch)/len(acc_per_batch))
fit(100)
mp.plot(x_axis, y_axis)
mp.show()
Here we are training our model for 100 times. Lets see how our accuracy graph looks like.
Weird ! That is not how it is supposed to like right? Well it seems that model did its best around 60’th training loop. Also try changing the values of hyperparameters , to see if it changes the accuracy.
Testing
Finally, lets see our model’s accuracy in the test data set.
a = model.predict(test_[0][0])
print(a)
print(test_[0][1])