Proof of Logistic Regression

Nerd Cafe

Step 1: What Are We Trying to Do?

We want to predict a binary outcome, like:

• Will it rain today? (Yes or No)

• Is this email spam? (Yes or No)

So, for every input (let's say features like humidity or email content), we want to predict whether the answer is 1 (yes) or 0 (no).

Step 2: The Idea: Use a Function to Predict Probabilities

Instead of predicting 0 or 1 directly, logistic regression predicts a probability between 0 and 1.

We use this function:

$$Probability=\frac{1}{1+e^{-z}}$$

Where:

$$z=θ_{0}+θ_{1}x_{1}+θ_{2}x_{2}+⋯+θ_{n}x_{n}$$

This function is called the sigmoid function. It turns any number into a value between 0 and 1.

• If the output is > 0.5 → we predict 1

• If the output is ≤ 0.5 → we predict 0

Step 3: Training the Model = Finding Best Parameters (θ)

We want to find the values for θ (theta parameters) that make the predictions as accurate as possible.

We do this by minimizing the error (cost).