Neural plasticity · learning rules

Hebbian Cell Assemblies

“Neurons that fire together wire together” is the slogan. The claim underneath it is the interesting part: if a rule that only ever looks at two cells at a time is applied over and over, a group of cells that keeps being co-activated will end up wired into a unit that can be reawakened by any fragment of itself. Train two patterns here and watch the weight matrix separate into blocks.

Untrained network. Every synapse starts weak and roughly equal.

Protocol

Network

Assembly size4 cells Pattern A takes the first cells of the ring, pattern B the next. The rest are never driven and act as a control.

θ firing threshold0.60 Total synaptic input a cell needs before it fires.

Input reliability90% Chance each member of the pattern is actually driven on a given trial. Real input is never clean.

Learning rule

η learning rate0.10

ρ depression threshold0.50 Set it to 0 and the rule becomes pure “fire together, wire together”, with no way to ever weaken anything.

Homeostasis

Synaptic scaling

Hold each cell's total incoming weight constant, so synapses have to compete for a fixed budget instead of all saturating.

Cue size for probing2 cells How much of the pattern the probe supplies. The rest has to be filled in by the network.

Auto-train speed6 trials/s

The network. Line weight is synaptic strength; only connections above a quarter of maximum are drawn. Hover a cell for its inputs.

assembly A assembly B undriven strengthened

The same synapses as a matrix: row i, column j is the weight from cell j onto cell i. Hover a cell for the value.

strong weak

Mean weight within A, within B, and between the two, trial by trial. The gap that opens between the curves is the assembly.

within A within B A ↔ B

trials0

within A0.20

within B0.20

A ↔ B0.20

last recall—

What the figure is doing

Twelve cells, all-to-all recurrently connected, each synapse a single number W_ij between 0 and 1: the strength of the connection from cell j onto cell i. A cell fires when the weighted sum of its active inputs, plus whatever external drive it is getting, reaches the threshold θ.

A trial drives one pattern, A or B, and then applies the learning rule to every synapse whose pre-synaptic cell fired:

ΔW_ij = η · x_j · (x_i − ρ)

Read it from the synapse's point of view. Nothing happens unless the pre-synaptic cell fired (x_j = 1): plasticity is homosynaptic, a silent input is not punished for what the rest of the network did. Given that it fired, the synapse is strengthened if the post-synaptic cell also fired and weakened if it did not. ρ sets where that changeover sits, and so how much depression the rule produces for each unit of potentiation.

That asymmetry is what makes the rule usable. Set ρ = 0 and the rule becomes pure potentiation: it can add but never subtract. The within-assembly weights still climb to the ceiling, so the matrix still looks like it has blocks — but the connections out of each assembly are never weakened, and they sit at their starting value of 0.2. Four of those firing at once is 0.8, which is over threshold. Probe it and the cue recruits A, then A ignites B, then B ignites everything else, and the recall readout reports the entire rest of the network as an intrusion. A network that cannot weaken a synapse can store exactly one thing: all of it.

Things worth doing with it

Train A alone, twenty times. Watch the top-left block of the matrix fill in while the cells of A stop driving anything outside A. The network has not been told that A is a group. It has only ever applied a two-cell rule.
Then train B. The second block appears and the off-diagonal blocks empty out. The A ↔ B curve falling away from the other two is the moment the two patterns become separately addressable.
Probe. Press Probe A on the untrained network and nothing happens: two cells firing cannot push a third over threshold. Train, then probe again, and the cue recruits the whole assembly. That is pattern completion, and it is the payoff Hebb was after — a memory that can be addressed by its contents rather than by its address.
Break it. Push θ up and the cue can no longer recruit anything; pull it down and everything fires at once, including B, which the recall readout reports as an intrusion. A useful assembly lives in the window between those two failures, and nothing in the learning rule guarantees the network lands in it.
Turn on synaptic scaling. Each cell's incoming weights are renormalised to a fixed total after every trial, so a synapse can only gain at its neighbours' expense. The weights stop saturating and settle at graded values instead, and the matrix keeps its block structure without ever pinning anything to 1.
Lower the input reliability. The patterns are delivered with cells missing, so the rule sees a different subset each trial. The assembly still forms, because what it is extracting is the statistics of the input rather than any single presentation.

Where the model stops being true

The cells are binary and update in lockstep. Real assemblies are defined by spike timing, and the order of firing carries information this model throws away — which is exactly what the STDP figure is for.
Every cell here is excitatory. A real recurrent network this densely and strongly connected would seize; cortex holds itself just below that point with inhibition that is tuned as carefully as the excitation, and the interneurons doing it are plastic too.
Weights are bounded by fiat. In biology the ceiling comes from the number of receptors a spine can hold and the volume it can grow to, and the floor comes from silent synapses being pruned outright, which is a structural change rather than a change in one number.
Learning is instantaneous and permanent. Real potentiation has an early phase that decays within an hour or two, and only some of it is consolidated into a lasting change by protein synthesis. Nothing here forgets.