1. Perceptrons

Perceptron

The foundation of neural networks is the perceptron, the simplest possible neural unit. A single perceptron is a linear discriminative classifier that learns the parameters of a decision boundary directly from data. Like other linear models, it defines a discriminant function
$y(x)=w^{T}x+w_0,$
and assigns classes based on its sign:
$\hat{y}=\{\begin{array}{ll}+1,& y(x)>0\\ -1,& y(x)<0\end{array}.$

The key difference is that the perceptron includes a learning rule that iteratively adjusts the weights $w$ and bias $w_0$ whenever a sample is misclassified. For each labeled training pair $(x,y)$, where $y\in \{-1,+1\}$:

1. Compute the prediction $\hat{y}=\text{sign}(w^{T}x+w_0)$.
2. If $\hat{y}\ne y$, update: $w\leftarrow w+yx,w_0\leftarrow w_0+y$.

Each update shifts the decision boundary so that the misclassified point moves toward the correct side.