A Coder’s Guide to Neural Networks — Chapter 4: Pima Indians Diabetes Dataset

The question you must be asking is, “how do we do that?” Simple. The answer is that we turn the first row into the first column, and the second row into the second column, etc. That changing columns into rows and vice versa is called transpose, hence the “T” in “.T”.

Which matrix do we transpose? If we do the one on the left, then we change it from [1 x 3] to [3 x 1]. That doesn’t really help us, because the amount of columns of the matrix on the left still doesn’t match up with the amount of rows of the matrix on the right. If we transpose the matrix on the right, we go from [2 x 3] to [3 x 2]. That works perfectly! Our amount of columns of the matrix on the left is now equal to the amount of rows of the matrix on the right, and now, we can perform our equation, without the errors.

Let’s run our equation again, now that we just covered matrix multiplication for multiple features.

weights_E@X_E.T

tensor([70, 47])

Before we apply logistic regression to our dataset, we’re finally going to go over why I keep adding “1” to all of our graphs. We haven’t added any 1s to any of our datasets, so why do I keep putting it there in our graphs? Before I explain, let’s quickly go back to our equation of a line and some simple rules of multiplication.

Our equation of a line was defined as y = slope * X + bias, right? Makes sense so far. Now, what if I showed you the following equation,
y = slope * X + bias * 1 . It’s mathematically the same, because anything multiplied by 1 is itself. If it’s the same, then why do I add a 1 to it? It seems better to just take the simpler equation.

Let’s go over some code to make better sense of this.

bias_E = torch.tensor([5])

I just made a rank 1 tensor by the name of bias_E. It contains only 1 number within it. Let’s print out our linear equation below (with and without the 1).

print(weights_E@X_E.T + bias_E)
print(weights_E@X_E.T + bias_E*1)

tensor([75, 52])
tensor([75, 52])

No surprise there. It’s the exact same result when multiplying with and without the 1.

Now, we’re going to do something a little different. We’re going to create 2 new variables using the variables we already have. The tensor bias_E is concatenated to weights_E and a tensor of 1s is concatenated to the tensor X_E. Let’s take a look.

weights_E_new = torch.cat((weights_E, bias_E), dim=0)
X_E_new = torch.cat((X_E, torch.ones(2, 1, dtype=int)), dim=1)
print('weights_E_new:')
print(weights_E_new)
print()
print('X_E_new:')
print(X_E_new)

weights_E_new:
tensor([2, 3, 4, 5])

X_E_new:
tensor([[9, 8, 7, 1],
[4, 5, 6, 1]])

There’s nothing extraordinary done to those variables, but this is where the magic of math happens.

print(weights_E@X_E.T + bias_E*1)
print(weights_E_new@X_E_new.T)

tensor([75, 52])
tensor([75, 52])

We were able to get the same result, but without adding the bias. How? Let me show you in math…

Our equations are the following (with the bias):
Row 1:
2(9) + 3(8) + 4(7) + 5(1)

Row 2:
2(4) + 3(5) + 4(6) + 5(1)

When you look at it like that, the bias just looks like it can be part of the weights tensor, and the 1, which is multiplied in each equation, looks like they can be added to the X tensor. That’s exactly what the concatenation did. That’s the magic of math!

Just before we run logistic regression, let’s split our dataset into 2 parts. 1 part holding the features and the other holding the target. Thereafter, we’ll turn those into tensors.

X = df[df.columns[:-1]]
y = df['Outcome']
X = torch.tensor(X.values)
y = torch.tensor(y.values)
X = torch.cat((X, torch.ones(X.shape[0], dtype=float).reshape(-1,1)), dim=1)

Going over the code, X = df[df.columns[:-1]] takes all the columns of the dataset, except for the last column (i.e. Outcome) and assigns it to X, while y = df[‘Outcome’] only takes the Outcome column and assigns it to y.

The torch.ones function is being used exactly the same way, except this time, we’re adding a 1 for every row that exists in X (X.shape[0] returns the amount of rows present in X). Don’t worry about the reshape for now. We’ll return to it at the end of this chapter.

Finally, time for logistic regression. Let’s stop writing words and start writing code. Time to see logistic regression in action!

def log_loss(y, y_pred): ##log loss error (binary cross entropy)
return -torch.sum((y*torch.log(y_pred) + (1-y)*torch.log(1-y_pred)))/y.shape[0]
def sigmoid(linear):
return 1/(1+torch.exp(-linear))
epoch_loss = []
weights = torch.zeros(9, dtype=float, requires_grad=True)
learning_rate = 1e-4
n = X.shape[0]
for epoch in range(8000+1): 
linear = weights.reshape(1,-1)@X.T ##one equation for all
y_pred =  sigmoid(linear) ##logistic
loss = log_loss(y, y_pred)
epoch_loss.append(loss.item())
###backpropagation###
loss.backward()
###step###
with torch.no_grad():
weights -= learning_rate * weights.grad
weights.grad.zero_()
if(epoch % 1000 == 0):
acc = torch.sum(((y_pred>=0.5)+0 == y)+0).item()/y.shape[0]
print('epoch: {0} - loss: {1:.5f}; acc: {2:.3f}'.format(epoch, epoch_loss[-1], acc))