"We can't solve problems by using the same kind of thinking we used when we created them." (Albert Einstein)
Noise-based logic and computing: from Boolean logic gates to brain circuitry and its possible hardware realization L.B. Kish (1), S.M. Bezrukov (2), S. Khatri (1), Z. Gingl (3), S. Sethuraman (1) (1)Department
of Electrical and Computer Engineering, Texas A&M University, College Station
(2) Laboratory
of Physical and Structural Biology, Program in Physical Biology, NICHD, National Institutes of Health, Bethesda, MD 20892, USA
(3) Department
of Experimental Physics, University of Szeged, H-6720, Hungary
When noise dominates an information system, like in nano-electronic systems of the foreseeable future, a natural question occurs: can we perhaps utilize the noise as information carrier? Another question is: Can a deterministic logic scheme be constructed that may be the explanation how the brain can efficiently process information, with random neural spike trains of less than 100 Hz frequency, and with similar number of neurons than the number of transistors in a 16 GB Flash dive? The answers to these questions are yes. See more at: http://www.ece.tamu.edu/%7Enoise/research_files/noise_based_logic.htm
Invited talk at the Fourth International Workshop on Natural Computing, September 22-26, 2009, Himeji, Japan
Texas A&M University, Department of Electrical and Computer Engineering
Present active collaborators in noise-based logic: (In chronological order of the first joint paper submission, or expected submission. Brown color: joint results in this talk.)
Sunil Khatri, (computer engineering faculty, TAMU): "quantum-mimics", memory, chip, complexity, etc. Swaminathan Sethuraman (mathematician, fresh PhD, TAMU): "quantum mimics", etc. Sergey Bezrukov (chief scientist, NIH): brain: information processing/routing, circuitry, efficiency, etc. Zoltan Gingl (physics faculty, Univ. of Szeged, Hungary); modeling for circuit realization, etc. Ferdinand Peper (senior computer scientist, Kobe Research Center, Japan): "quantum mimics", tokens, etc. Kamran Entesari (electrical engineering faculty,TAMU): noise generators for chip realization, etc. Khalyan Bollapalli (computer engineering PhD student, TAMU): chip realization Zoltan Bacskai (physics PhD student, Univ. of Szeged, Hungary): DSP circuit realization Gabor Schmera (mathematician, US Navy, SPAWAR): Languevin equations and numeric solutions, etc.
Texas A&M University, Department of Electrical and Computer Engineering
Our related papers in chronological order (brown: subject of this talk): • L.B. Kish, "Thermal noise driven computing", Appl. Phys. Lett. 89 (2006) 144104; http://arxiv.org/abs/physics/0607007 • L.B. Kish, "Noise-based logic: binary, multi-valued, or fuzzy, with optional superposition of logic states.", Physics Letters A 373 (2009) 911-918; http://arxiv.org/abs/0808.3162 • L.B. Kish, S. Khatri, S. Sethuraman, "Noise-based logic hyperspace with the superposition of 2^N states in a single wire", Physics Letters A 373 (2009) 1928-1934, http://arxiv.org/abs/0901.3947 • S. Bezrukov, L.B. Kish, "Deterministic multivalued logic scheme for information processing and routing in the brain", Physics Letters A 373 (2009) 2338-2342, http://arxiv.org/abs/0902.2033 • K. Bollapalli, S. Khatri, L.B. Kish, "Low-Power VLSI Design using Superposition of Sinusoidal Supplies" Austin Conference on Integrated Systems and Circuits (ACISC) 2009.
Texas A&M University, Department of Electrical and Computer Engineering
Content 1. The device size-speed-error-energy issue in classical digital and single electron logic.
2. Continuum-noise-based logic, binary and multivalued logic.
3. Utilizing the logic hyperspace: 2N bits [2^(2^N) logic values] in a single wire, like in a quantum computer.
4. Implementation of the hyperspace for neurons and their stochastic spike trains. Deterministic, multivalued brain logic and routing the information in the brain.
Texas A&M University, Department of Electrical and Computer Engineering
It is fashionable to cite old, historical objections against the potentials of science and then point it out how much science and technology has been outperforming even the most courageous expectations. For example the citation in Popular Mechanics (1949), forecasting the perspectives of science:
Computers in the future may weigh no more than 1.5 tons.
Texas A&M University, Department of Electrical and Computer Engineering
However, let's go against the fashion, while staying with computers ...
Texas A&M University, Department of Electrical and Computer Engineering
In the "Blade Runner" movie (made in 1982) in Los Angeles, at 2019...
Texas A&M University, Department of Electrical and Computer Engineering
In the "Blade Runner" movie (made in 1982) in Los Angeles, at 2019, the Nexus-6 robots are more intelligent than average humans.
Texas A&M University, Department of Electrical and Computer Engineering
2019 is only 10 years from now and nowadays we have been observing the slowdown of the evolution of computer chip performance. We are simply nowhere compared a Nexus-6.
Texas A&M University, Department of Electrical and Computer Engineering
Isaac Asimov (1950's): The Three Laws of Robotics
Texas A&M University, Department of Electrical and Computer Engineering
Isaac Asimov (1950's): The Three Laws of Robotics:
1. A robot may not injure a human being, or, through inaction, allow a human to come to harm. 2. A robot must obey orders given to him by human beings except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Texas A&M University, Department of Electrical and Computer Engineering
Isaac Asimov (1950's): The Three Laws of Robotics:
Not even the best supercomputer systems are able to address such refined perception of situations! We have great problems even with the most elementary necessities, such as recognition of natural speech of arbitrary people or speech in background noise.
Texas A&M University, Department of Electrical and Computer Engineering
A physical mechanism is causing the slowdown and evaporation of the dreams? The Speed-Error-Energy triangle of microelectronics:
(Miniaturization)
Speed
Error
Energy
Texas A&M University, Department of Electrical and Computer Engineering
Only two logic values are utilized in a single wire in today's digital circuitry
U0 (power supply voltage) 1
1
1
1
UH Usignal(t) UL 0
0
0
0
0
Time Clock generator events Texas A&M University, Department of Electrical and Computer Engineering
Model-picture of speed and dissipation versus miniaturization (LK, PLA, 2002) A switch is a potential barrier which Maximal clock frequency exists (off position) or not (on position). To control/build the potential barrier we need energy.
f 0 (RC)1
U 02 Dissipation by a single unit P1 f 0 E1 (RC) CU R 1
2 0
Total dissipation by the chip PN NU 02 /R NU 02 U 02 /s 2
CMOS drivers' channel resistance
number of units N
1 s2
R 1
2
C
CMOS gate capacitance
U0
C s2
Cs
Texas A&M University, Department of Electrical and Computer Engineering
Nice, but how about the noise and errors? Unavoidable noise: Thermal noise (Johnson noise)
R
Su ( f ) = 4kTR
C
misleading!
u(t)
(T)
1 kT 2 C uc = E C = (one thermodynamical degree of freedom) 2 2
kT u = C 2 c
Energy equipartition theorem; only the capacitance matters!
Texas A&M University, Department of Electrical and Computer Engineering
False bit flips. Gaussian noise can reach an arbitrarily great amplitude during a long-enough period of time. U0 (power supply voltage)
A m p l i t u d e
1
1
1
1
UH Usignal(t) UL 0
0
0
0
0
Time Clock generator events
time Same as the thermal activation formula, however, here we know the mean attempt frequency more accurately.
For band-limited white noise, frequency band (0, fc) :
U th2 2 (U th ) = exp 2 f c 3 2U n
where
U n = S(0) f c
Texas A&M University, Department of Electrical and Computer Engineering
(1/year)
Minimal energy need 1012
9
109 10
11*Un noise margin is not safe for future progress.
10 transistors Clock frequency: 20GHz
12 Un noise margin is very safe.
10
10 transistors
6
103
Frequency of bit flip errors
Conclusion:
Clock frequency: 2GHz
100 8
10-3
10 transistors
10-6 10
1 transistor
-9
The breakdown is extremely progressive. 20% change of the thermal noise or the threshold yields a change by factor of 109
10-12 10-15 8
9
10
11
12
U /U th
n
Texas A&M University, Department of Electrical and Computer Engineering
Thermal death of Moore's law (Kish, Physics Letters A, 2002) See more: L.B. Kish, "Moore's Law and the Energy Requirement of Computing versus Performance", IEE Proc. - Circ. Dev. Syst., 2004.
Clock frequency has not been increased since then! 1
Noise margin, V
2002
Actual noise margin, old
• Optimistic estimation: • No hot electron noise • No 1/f noise • No cross-talk noise • No variability errors
2002 2003 Required noise margin, old
0.1
Required noise margin, new
10
Actual noise margin, new
Size, nm
2003 100
Texas A&M University, Department of Electrical and Computer Engineering
November 2002
January 2003
Conclusion was (2002): if the miniaturization is continuing below 30-40 nm, then the clock frequency cannot be increased.
No increase since 2003 ! Prophecy fulfilled earlier!
Texas A&M University, Department of Electrical and Computer Engineering
Can we gain the energy back? Some of the most famous criticisms of reversible computing approaches. Wolfgang Porod, David Ferry, and coworkers. W. Porod, Appl. Phys. Lett. 52, 2191 (1988); and references therein; W. Porod, R.O. Grondin, D.K. Ferry, Phys. Rev. Lett. 52, 232-235, (1984); W. Porod, R.O. Grondin, D.K. Ferry, G. Porod, Phys. Rev. Lett. 52, 1206, (1984); and references therein. Their most important general argument:
Logical reversibility has nothing to do with physical reversibility. How about the errors??? Ralph Cavin, et al, FNL 2005: If we want to do reversible computing with the original error rate then we end up at more energy dissipation.
Texas A&M University, Department of Electrical and Computer Engineering
Perhaps, the most important conclusion of the debates:
• Claims about high speed without error rate and energy efficiency aspects are interesting but meaningless for practical developments. • Claims about high energy efficiency without error rate and speed drop aspects are interesting but meaningless for practical developments. • Claims about efficient error correction without energy requirement and speed drop aspects are interesting but meaningless for practical developments. • These speed-error-energy implications must be addressed at the system level otherwise they are meaningless for practical developments. Maybe we won at the single gate level, which is interesting but unimportant, but lost at the system level.
Texas A&M University, Department of Electrical and Computer Engineering
In today's chips, the signal and the disturbances are parallel vectors, thus disturbances have maximal weight in the total voltage.
Threshold
Signal
+
Cross-talk
+
Noise
=
Texas A&M University, Department of Electrical and Computer Engineering
Another question: multi-value logic. Adding multi-levels to current schemes to implement multivalue logic of K values would increase the energy need by a factor of K2.
U0 (power supply voltage) 1
1
1
1
UH Usignal(t) UL 0
0
0
0
0
Time Clock generator events Texas A&M University, Department of Electrical and Computer Engineering
1. Can we use signals which is orthogonal on the crosstalk+noise? 2. Can we use N>1 signals which are orthogonal to each other, to make a multivalue logic? If we use superposition of the vectors in a binary fashion (on/off) then an N-dimensional signal space would make a logic scheme with K=2N logic values in a single wire. Orthogonal sinusoidal signals would do, however the smallest possible signal is the noise in the information channel. Thus we explore the noise-based direction here.
Threshold
Signal-1
Cross-talk + Noise Threshold
Signal-2 Texas A&M University, Department of Electrical and Computer Engineering
Continuum-noise-based logic: Binary, multi-valued, or fuzzy, with optional superposition of logic states L.B. Kish, Physics Letters A 373 (2009) 911-918, ( http://arxiv.org/abs/0808.3162 )
Noises: independent realizations of a stochastic process (electronic noise) with zero mean. Examples: thermal noises of different resistors or current noises of different transistors: Vk (t)
N-dimensional logic space with orthogonal logic base vectors:
Vi (t)Vj (t) = i, j Generally, a logic state vector is the weighted superposition of logic base vectors: aL2 + a 2H = 1
N
X(t) = aiVi (t) i =1
(Binary H) aL L + a H H
H
fuzzy
For example, a binary logic base is:
L2 (t) = 1
H 2 (t) = 1
L (Binary L)
H (t)L(t) = 0
Multidimensional logic hyperspace was also introduced by multiplying the base noises, see later. Texas A&M University, Department of Electrical and Computer Engineering
Noise-based logic Continuum noise ?
Random spike trains ?
Concerns: Stochastic logic? Slow; repeated operations ? Deterministic logic? Averaging (statistics) slowdown? Speed? Number of logic values? Energy need; power dissipation/performance? Devices and logic gates? Error probability?
Texas A&M University, Department of Electrical and Computer Engineering
Basic structure of noise-based logic with continuum noises: L.B. Kish, Physics Letters A 373 (2009) 911-918
Reference (base) noises
Input signal (noise)
Input stage: Correlators
Reference (base) noises
DC
Logic units DC (fast errors)
DC
Output stage:
Output signal (noise)
Analog switches
These two units can together be realized by a system of analog switches
Note: analog circuitry but digital accuracy due to the saturation operation represented by the switches!
Texas A&M University, Department of Electrical and Computer Engineering
The basic building elements of noise based-logic (out of the noise generators which can be simply resistors or transistors) are the same as that of analog computers: linear amplifiers; analog multipliers; adders; linear filters, especially time average units which are low-pass filters; analog switches; etc. Note: analog circuitry but digital accuracy due to the saturation operation represented by the switches!
Analog Multiplier
X1(t)
Y(t) = X1(t) X2(t)
X
(Inputs) X2(t)
(Output)
Time average
Y = X(t)
X(t) (Input)
R
C
where = RC
(Output)
Analog switch, follower If X>UH then switch is closed
X (Input)
UL,UH
If XUH then switch is open
X (Input) UL,UH
If X
