Neuromorphic Computing
A Brief Introduction
What is Neuromorphic Computing ?
● Coined by Carver Mead in 1990 to refer to custom VLSI chips that mimicked
biological neural systems.
● Today, neuromorphic computing combines computing fields such as machine
learning and artificial intelligence with cutting-edge hardware development
and materials science, as well as ideas from neuroscience.
Neural-inspired systems for non–von Neumann computational architectures
Key Features of a Neuromorphic System
● Highly Connected
● Co-located memory and processing
● Requiring Low Power
● Simple Communication between the components
● Learning in components
Motivation
● Von-Neumann Bottleneck
“it is an intellectual bottleneck that has kept us
tied to word- at-a-time thinking” - [Backus1978]
● Plateau of Moore’s Law
● End of Dennard Scaling
Why Neuromorphic architecture ?
● Co-located memory and computation
● Neuromorphic architectures often result in lower power consumption
● Natural Implementations of Data Analysis Techniques
● Potential Emergence of Intelligent Behaviour
Models in Neuromorphic Computing
Which neural network model to use ?
● Common components of a neural network model are neurons and synapses,
taking inspiration from biological neural networks.
● When defining the neural network model, we must also define models for each
component.
● The choice of the model may depend on the application .
Neuron Models
● A biological neuron is usually composed of:
○ Axons - where neuron transits output
○ Dendrites - where neurons receive input
○ Synapse - junction between axon of a neuron and dendrites of another neuron
● Typical behaviour of a neuron:
○ Accumulates charge through change in potential across neuron’s cell membrane
○ Voltage Potential in neuron > threshold → neuron fires
○ Generate action potential to the next neuron
Neuron Models
● The neuron models are divided into 5 broad categories:
○ Biologically-plausible
○ Biologically-inspired
○ Neuron+other
○ Integrate-and-fire
○ McCulloch-Pitts
Network Model
Network models describe how different neurons and synapses are connected and how
they interact.
Algorithms & Learning
Some of the major open questions for neuromorphic systems revolve around
algorithms.
● Supervised Learning: Back-propagation is one of the most commonly utilized
algorithms for programming
● Unsupervised Learning: Hebbian- type learning rules, spike-timing dependent
plasticity.
Hardwares - Memristor
● MEMory + ResISTOR = MEMRISTOR
● Passive electronic component with two terminals having variable electrical
resistance.
● The resistance value changes depending on the
○ Polarity of voltage applied
○ Duration of application
● Resistance value does not change even when the power is turned off.
Hardwares - Memristor
● A popular device-level component in neuromorphic computing.
● Memristors can exhibit STDP-like behavior that is very similar to what occurs in
biological synapses.
● Possess Inherent fault tolerance
● Uses:
○ Entire Synapse implementation
○ Synaptic weight storage element
References
● Catherine D. Schuman, Thomas E. Potok, Robert M. Patton, J. Douglas Birdwell, Mark E.
Dean, Garrett S. Rose, & James S. Plank. (2017). A Survey of Neuromorphic Computing
and Neural Networks in Hardware. arXiv:1705.06963 [cs.NE]
● https://www.nanowerk.com/memristor.php (Memristor)
● Potok, Thomas, Schuman, Catherine, Patton, Robert, Hylton, Todd, Li, Hai, and Pino,
Robinson. Neuromorphic Computing, Architectures, Models, and Applications. A
Beyond-CMOS Approach to Future Computing, June 29-July 1, 2016, Oak Ridge, TN.
United States: N. p., 2016. Web. doi:10.2172/1341738
● Wikipedia
Thank You
