Neural networks

Years 9 to 13

An artificial neuron multiplies inputs by weights, adds them up, and fires if the total is big enough. Networks join many together.

A neuron: 2 × 3 + 1 × (−2) = 4. Since 4 is more than 0, it outputs 1.

For learners

An artificial neuron takes some numbers in. Each input is multiplied by a weight, which says how much that input matters.

The neuron adds up the results. If the total is big enough, it fires and outputs 1. If not, it outputs 0.

A neural network joins lots of neurons together. Training the network means adjusting all the weights, a little at a time, until its answers are right.

A worked example

Inputs 3 and 1, weights 2 and −4, bias 1. What is the total?

$3 \times 2 = 6$ and $1 \times (- 4) = - 4$ .
$6 + (- 4) = 2$ .
Add the bias: $2 + 1 = 3$ .

For grown-ups

Each neuron computes a weighted sum, adds a bias, and applies an activation function such as ReLU. A layer of neurons is a matrix multiplied by a vector. Training uses gradient descent: work out how the error changes as each weight changes, then nudge the weights to make the error smaller. This is the same slope idea as in calculus.

Common mistakes

Forgetting to add the bias before deciding.
Thinking a neuron 'understands': it only does arithmetic.
Mixing up the weight (what the input is multiplied by) and the input itself.

Why it matters

Neural networks power image recognition, translation and chatbots.

Topics in quarter that use this

A tiny artificial neuron
A neural network layer
Learning by going downhill