Logistic Regression is the most basic and widely used classification algorithm in machine learning. It is used in many real-world scenarios like spam detected, cancer detection, IRIS dataset, etc. Mostly it is used in binary classification problems. But it can also be used in multiclass classification.

Logistic Regression predicts the probability that the given data point belongs to a certain class or not. In this article, I will be using the famous heart disease dataset from Kaggle:

In this dataset, the main goal is to predict whether the given person has heart disease or not. In this case, logistic regression will predict the probability of whether the given person has heart disease or not.

For understanding logistic regression, first, you need to understand linear regression, as it forms the base for logistic regression.

As you probably know earlier, linear Regression is a linear model that assumes that a relationship exists between the given input variables(independent variables) and the output variable(dependent variable).

It suggests that the output value can be predicted as a linear combination of input variables. It basically tries to fit a straight line to the data:

If the variables are x1, x2, x3, and x4 then the linear regression suggests that the y value can be predicted as:

`y = w0 + w1*x1 + w2*x2 + w3*x3 + w4*x4`

The main goal of the linear regression model is to predict the optimal values of `w0, w1, w2, w3 and w4`

such that the cost function value is as low as possible.

What is the cost function?Cost Function quantifies the error between predicted values and expected values. For linear regression, the cost function used is the mean squared error.

The formula for mean squared error is given by: