Topic: How are concepts represented by the brain?
Computers can represent concepts as programs, in a chosen programming language. But how about brains?
- How is a concept represented?
- How does a directed, weighted, dynamic graph represent/form a memory/concept?
- What is a memory?
- reproducible (with stimulus or without stimulus)
- corresponds to a sampleable distribution
Concrete requirements for memory
- Similar response for similar stimuli
- Hierarchical (concepts of concepts)
Simple Connectome Model
Neurons are modeled as a directed, weighted and dynamic graph:
= strength of edge (i, j)
= activation level of a neuron i
Suppose W is fixed and only X changes with time t. Then,
= edge weight
activation level i at time t
If we use a linear threshold function for :
At equilibrium, it converges to (eigenvector of X).
This model (with a linear threshold function) is not satisfactory because no matter which stimulus we give, it converges to the same X!
There are 2 detailed models in the literature:
- Neuroidal Model (Valiant)
- Cell Assemblies (Hebb)
Concept is stored as an “item” (a subset of neurons).
Each concept is memorized as a subset of neurons of size r, and if k out of r neurons fire, that concept is recalled.
Using this model, we can memorize up to number of concepts.
Since each concept should be distinguishable, overlaps between subsets should be small!
- Base graph is random () and support is fixed (weights will be changed).
- Output is random
- JOIN & LINK operations
How do we represent hierarchical concepts in neuroidal model?
If is a concept that is composed of and , we want to fire when and both fire.
- Use union:
- is simply an union of and
- Problem: concept size doubles for every union (not stable)!
- Create another subset of size r:
- We want to set up so that if k neurons in fire and k neurons in fire, then k neurons in also fire.
- Pick (“recruit”) neurons that is connected to both and
- P(neuron l fires when fires) = P (on r tosses of p biased coin, we get at least k heads) = q(r,p,k). Since it should happen for both and , it should be . We want
A concept is stored as an “assembly” of highly interconnected neurons. Because of such high interconnectivity, some assembly member neurons can activate the entire assembly.
- Reader neurons
- Rules (neural syntax)
- “Synapsembles”: weights are dynamically changing all the time
Suppose an external stimulus X(0) is given. Then,
How should we change weights W? We should strengthen the connection between two neurons if both keep firing:
Also, we normalize the pre-synaptic weights at each neuron by keeping the sum of all incoming weights at 1.
Note that changes depending on both and both fire.