P e x c e r a

The Anatomy of an Artificial Perceptron

The perceptron, introduced by Frank Rosenblatt in 1958, is the fundamental building block of artificial neural networks. It computes a weighted sum of inputs, adds a bias, and applies a step activation function to make a binary decision.

Mathematical Formulation

The perceptron transforms a vector of inputs into a binary classification output using a linear combination followed by a threshold function.

The Linear Summation

An artificial perceptron takes inputs $x_i$, scales them by weights $w_i$, and adds a bias $b$ to produce the pre-activation value $z$:

$$z = \sum_{i=1}^n w_i x_i + b = \mathbf{w}^T \mathbf{x} + b$$

The weights determine the relative importance of each input, while the bias shifts the decision boundary away from the origin. The dot product represents the alignment of the input vector with the weight vector in high-dimensional space.

The Step Activation Function

The pre-activation value $z$ is passed through a binary step function $f(z)$ to generate the final prediction $y$:

$$f(z) = \begin{cases} 1 & \text{if } z \ge 0 \ 0 & \text{if } z < 0 \text{ (or } -1 \text{)} \end{cases}$$

This outputs a binary class label directly based on whether the weighted input crosses the zero threshold. Rosenblatt's original perceptron used a step function, which creates a sharp decision boundary but makes the function non-differentiable at zero.

Geometrical Interpretation

The perceptron equation represents a decision boundary that divides the input space into two distinct regions.

The Decision Hyperplane

The equation $\mathbf{w}^T \mathbf{x} + b = 0$ defines a decision hyperplane. In two dimensions, this is a line; in three dimensions, it is a plane; and in higher dimensions, it is a hyperplane. Points on one side of this boundary yield a positive activation ($y=1$), while points on the other side yield a negative or zero activation ($y=0$).

The weight vector $\mathbf{w}$ is perpendicular to this hyperplane, pointing toward the positive class. The bias $b$ determines the perpendicular distance from the origin to the decision boundary, controlling how easy it is for the perceptron to fire.

Linear Separability Constraint

Because the decision boundary is linear, a single perceptron can only classify datasets that are linearly separable. This means that a single straight line or hyperplane must be able to completely divide the classes.

If the classes overlap or have a non-linear relationship (such as concentric circles or checkerboard patterns), the perceptron learning algorithm will fail to converge, highlighting the fundamental limitation of single-layer models.

PyTorch Implementation

We can implement the linear and threshold operations of a perceptron using PyTorch tensors and basic comparison operations.

Coding a Perceptron Forward Pass

Here is how to calculate a perceptron's forward pass manually in PyTorch:

<pre><code class="language-python">import torch import torch.nn as nn class Perceptron(nn.Module): def __init__(self, input_dim): super().__init__() # Initialize weights and bias manually self.weights = nn.Parameter(torch.randn(input_dim, 1)) # [input_dim, 1] self.bias = nn.Parameter(torch.zeros(1)) # [1] def forward(self, x): # x shape: [batch_size, input_dim] # Linear combination: z = xW + b z = torch.matmul(x, self.weights) + self.bias # [batch_size, 1] # Binary step activation function y = (z >= 0.0).float() # [batch_size, 1] return y # Instantiate and test with batch_size=2, input_dim=3 model = Perceptron(input_dim=3) x = torch.tensor([[1.0, 2.0, -1.0], [-2.0, 0.5, 1.5]]) print(model(x))</pre>

This implementation inherits from nn.Module and registers its weights and bias as learnable parameters. The forward pass utilizes efficient matrix multiplication, allowing it to process batches of samples simultaneously.

The Perceptron Learning Rule

The training of a perceptron is based on an iterative correction rule. For each sample, the weight update is calculated as $\Delta w_i = \eta (y - \hat{y}) x_i$, where $\eta$ is the learning rate, $y$ is the ground-truth label, and $\hat{y}$ is the prediction.

If the prediction is correct, the weights remain unchanged. If the model misclassifies a positive sample as negative, the weights are adjusted toward the input vector; if it misclassifies a negative sample as positive, the weights are adjusted away from the input vector.