The Artificial Neuron: Weights, Bias and a Decision

From brain cells to maths

A biological neuron receives signals from other neurons through its dendrites. If the combined signal in the cell body is strong enough, the neuron fires, sending a pulse down its axon to the next neurons. The junctions, called synapses, have different strengths, and learning changes those strengths.

An artificial neuron keeps only that skeleton:

Inside an artificial neuron

A worked example: will Sam enjoy this film?

The inputs describe the film. The weights describe Sam's taste. The bias describes how keen Sam is on films in general.

Passing z through the sigmoid (Level 4) gives σ(−0.49) ≈ 0.38: a 38% chance Sam enjoys it. The action helps, but at two and a half hours it is just too long.

The weighted sum is a dot product (Level 3) of the input vector and the weight vector, plus the bias: z = w · x + b. That one line of maths, repeated billions of times, is most of what a neural network computes.

A neuron draws a line

With two inputs, the neuron says yes when w₁x₁ + w₂x₂ + b > 0 and no when it is below 0. The boundary, where w₁x₁ + w₂x₂ + b = 0, is a straight line.

• The weights set the angle of the line. The weight vector (w₁, w₂) points at right angles to the line, towards the "yes" side.
• The bias slides the line without turning it.
• With three inputs the boundary is a flat plane, and with more it is a hyperplane: a single neuron can only ever separate classes with something flat.

Move the sliders and watch the line turn and slide. Then press Train and the perceptron rule below moves them for you.

Neurons as logic gates

In 1943 McCulloch and Pitts showed that simple threshold units can compute logic. With a step activation (output 1 if z > 0, otherwise 0):

Because NAND alone can build any logic circuit, a network of neurons can in principle compute anything a digital computer can. The interesting question is not whether networks can compute, but whether they can learn the right weights.

The perceptron learning rule

Frank Rosenblatt's 1958 perceptron learned its weights from examples with a beautifully simple rule. For each training example:

1. Predict: output = 1 if w · x + b > 0, otherwise 0.
2. Error = target − output, which is +1, 0 or −1.
3. Update every weight: wᵢ ← wᵢ + learning rate × error × xᵢ, and the bias: b ← b + learning rate × error.

If the prediction is right, the error is 0 and nothing changes. If it said 0 but should have said 1, each weight grows in proportion to its input, pushing z up next time. If it said 1 but should have said 0, the weights shrink.

That guarantee is also the catch. If no straight line can separate the classes, the perceptron never settles.

The XOR problem

XOR (exclusive or) outputs 1 when exactly one input is 1.

The two "1" points sit on one diagonal and the two "0" points on the other. No single straight line can put both 1s on one side and both 0s on the other, so no single neuron can compute XOR. Minsky and Papert's 1969 book Perceptrons made this famous, and it helped trigger the first AI winter.

The fix was known in principle: use two neurons to draw two lines, and a third to combine them. A hidden layer. What was missing until 1986 was a way to train the hidden neurons. Level 7 builds that network and the algorithm that trains it.

Hands-on: build and train a neuron

A neuron in four lines

import math

def neuron(inputs, weights, bias):
    z = sum(x * w for x, w in zip(inputs, weights)) + bias
    return 1 / (1 + math.exp(-z))             # sigmoid activation

film = [0.9, 0.2, 2.5]                        # action, comedy, hours
sam = [1.5, 0.8, -0.6]
print(f"Chance Sam enjoys it: {neuron(film, sam, -0.5):.2f}")
print(f"The same film cut to 1.5 hours: {neuron([0.9, 0.2, 1.5], sam, -0.5):.2f}")

Logic gates

def step_neuron(inputs, weights, bias):
    z = sum(x * w for x, w in zip(inputs, weights)) + bias
    return 1 if z > 0 else 0

gates = {"AND": ([1, 1], -1.5), "OR": ([1, 1], -0.5), "NAND": ([-1, -1], 1.5)}
for name, (weights, bias) in gates.items():
    outputs = [step_neuron([a, b], weights, bias) for a in (0, 1) for b in (0, 1)]
    print(f"{name:4}: inputs 00 01 10 11 give {outputs}")

The perceptron learns OR, then fails at XOR

def train_perceptron(data, epochs=12, lr=1.0):
    w1 = w2 = b = 0.0
    for epoch in range(1, epochs + 1):
        mistakes = 0
        for (x1, x2), target in data:
            output = 1 if w1 * x1 + w2 * x2 + b > 0 else 0
            error = target - output
            if error:
                mistakes += 1
                w1 += lr * error * x1
                w2 += lr * error * x2
                b += lr * error
        print(f"  epoch {epoch:2}: w = ({w1:+.0f}, {w2:+.0f}), b = {b:+.0f}, mistakes {mistakes}")
        if mistakes == 0:
            return "learned it"
    return "never settled"

OR = [((0, 0), 0), ((0, 1), 1), ((1, 0), 1), ((1, 1), 1)]
XOR = [((0, 0), 0), ((0, 1), 1), ((1, 0), 1), ((1, 1), 0)]
print("OR:")
print(" ", train_perceptron(OR))
print("XOR:")
print(" ", train_perceptron(XOR, epochs=6))

Try this

1. Find weights and a bias that make a neuron output 1 only when both inputs are 0 (NOR).
2. Train the perceptron on AND. How many epochs does it take?
3. Change Sam's weights so that the long film is predicted to be enjoyable. Which weight did you change, and what does it say about the new Sam?

Level check

• Label the parts of an artificial neuron and match them to a biological neuron
• Calculate a neuron's weighted sum and output for given inputs, weights and bias
• Explain how the weights and the bias change a neuron's decision boundary
• Give weights and a bias that make a neuron act as AND, OR or NOT
• Apply the perceptron learning rule to one example by hand
• Explain why a single neuron cannot compute XOR, and what fixes it

Key terms you must be able to define

Artificial neuron

A unit that computes a weighted sum of its inputs plus a bias and passes it through an activation function.

Weight

A number multiplying one input; its sign and size set how much and which way that input pushes the output.

Bias

A number added to the weighted sum, shifting how easily the neuron outputs a high value.

Weighted sum

z = w₁x₁ + w₂x₂ + … + b, the dot product of weights and inputs plus the bias.

Activation function

The function applied to the weighted sum to give the neuron's output.

Step function

An activation that outputs 1 if its input is above 0 and 0 otherwise.

Perceptron

Rosenblatt's 1958 single neuron with a step activation, trained by the perceptron learning rule.

Linearly separable

Able to be split into classes by a straight line, plane or hyperplane.

XOR problem

The fact that no single neuron can compute exclusive or, because its classes are not linearly separable.

Hidden layer

A layer of neurons between the inputs and the output, whose values are not seen directly.

Flashcards

Go further: notes for future AI engineers

The geometry behind the weights

The decision boundary of a neuron is the set of points where w · x + b = 0. Because a dot product with w measures how far a point lies along the direction of w, the boundary is perpendicular to w, at a distance of |b| ÷ |w| from the origin. Doubling every weight and the bias leaves the line exactly where it is, but makes the sigmoid output change more sharply across it: the model becomes more confident without changing its mind. That is why weight decay (Level 4) also acts as a brake on overconfidence.

Hebb's rule and modern learning

Donald Hebb's 1949 idea, often summarised as "cells that fire together wire together", strengthens a connection when both neurons are active. The perceptron rule adds the crucial ingredient Hebb lacked: an error signal. Change the weights only when the output is wrong, in the direction that reduces the error. Gradient descent and backpropagation are the grown-up version of that same idea.

Misconceptions to avoid