Untangling complex syste.., p.71
Untangling Complex Systems, page 71
i
E
∂ ( t)
w
η
η
2
(
)
i = wi −
= wi + xi out − target [10.77]
w
∂ i
When in the network there are hidden layers, the error-correction rule transforms in back- propagation
one (Rumelhart et al. 1993). In time series prediction, the available data are often divided into two
sets: training and testing sets. The data of the training set are used during the learning stage when
the weights of the network are optimized. The data of the testing set are used to check whether the
network is suitable to make reliable predictions in the future or not.
352
Untangling Complex Systems
BOX 10.2 CHUA’S ELECTRONIC CIRCUIT
The Chua’s circuit is an electronic circuit that gives rise to a chaotic dynamic. It was invented
in 1983, by Leon Chua, an American electrical engineer and computer scientist working at
the University of California, in Berkley, since 1971. Chua wanted to design a concrete labora-
tory circuit that could be formally described by a system of differential equations close to that
formulated by Lorenz (remember equations [10.36–10.38]). The final goal was that of dem-
onstrating that chaos is a physical phenomenon and not an artifact of computer simulations
and round-off errors in computations. Chua’s circuit is shown in the following figure B10.2.
IR
R
L
V
C
R
2
C 1
VR
FIGURE B10.2 Chua circuit (on the left) and the function relating the current and voltage of the Chua
diode (on the right).
It consists of five elements. Four of them are traditional linear passive components, which are
one inductance ( L > 0), two capacitors and one positive resistance. Interconnection of passive
elements gives rise to trivial dynamics, with all element voltages and currents decaying to zero
over time. The most straightforward circuit that gives rise to oscillatory or chaotic dynamics
requires, at least, one locally active nonlinear element, such as the Chua’s diode V . Chua’s
R
diode must be characterized by a nonlinear current ( I ) versus voltage ( V ) function, such as
R
R
that depicted in the right part of the earlier figure. There exist several generalized versions of
the Chua’s Circuit. The Chua’s Circuit has been used as a physical source of pseudo-random
signals, and in numerous experiments on synchronization studies, and simulations of brain
dynamics. Arrays of Chua’s circuits have been used to generate 2-dimensional spiral waves,
3-dimensional scroll waves, and stationary patterns, such as Turing patterns (Chua 1998).
10.9 MASTERING CHAOS
The deterministic optimism, flourished at the beginning of the ninetieth century and promoted
by Pierre-Simon Laplace, was shattered by two events. First, the formulation of the “Uncertainty
Principle” by Heisenberg in 1927 and regarding the microscopic world. Second, the discovery of
chaotic dynamics in the macroscopic world. In the second half of the ninetieth century, it was Henri
Poincaré the first who realized the existence of nonlinear macroscopic dynamics extremely sensi-
tive to the initial conditions. Poincaré was studying the three-body problem interacting through the
gravitational force, and he discovered the sensitivity to the initial conditions when he tried to solve
the case wherein the third body had a tiny mass ( M ) compared with the other two (
).
3
M 3<< M 2< M 1
After many years of research, we now know that chaotic systems are common in nature. Sometimes,
we do not see the catastrophic effects of chaos only because those effects manifest after a long time.
For example, the Solar System is intrinsically chaotic, but “astronomically” stable. In fact, for col-
lisions between planets, such as Mercury and the Sun or Mercury and Venus, we should wait for at
least 109 years (Cencini et al. 2009).
The Emergence of Chaos in Time
353
In the previous paragraphs, we have learned that extreme sensitivity to tiny perturbations
characterizes chaotic systems. This feature, labeled as “butterfly effect,” has been considered trou-
blesome, for a long time. Since nobody can predict precisely how chaotic systems evolve over long
periods, everybody has dealt with them in just one way: avoiding chaotic dynamics as long as it has
been possible.
The first who realized that the “butterfly effect” may be fruitful in practical situations was John
von Neumann in 1950 (Shinbrot et al. 1993). If it is true that the climate is a chaotic system, then,
von Neumann thought, small, carefully chosen atmospheric disturbances could lead to the desired
large-scale change in weather. We still struggle to find out what might be those tiny perturbations
that would reduce the average temperature of our planet, which has been heating up in the last
decades (Salawitch et al. 2017). Nevertheless, the von Neumann’s idea is in principle sensible. In
fact, the “butterfly effect” permits the exploitations of tiny perturbations to master chaotic systems.
This possibility is tough to pursue in case of multi-dimensional chaotic systems, like the climate,
but it is much easier in the case of low-dimensional chaotic systems. For example, in 1985, NASA
scientists succeeded in sending their spacecraft ISEE-3/ICE more than 80 million km across the
Solar System, achieving the first scientific cometary encounter, by using only small amounts of
residual hydrazine fuel. This feat was made possible by the “butterfly effect” on the dynamic of the
three-body system, composed of the Earth, Moon and the spacecraft. It would not have been pos-
sible in a nonchaotic system (Farquhar et al. 1985).
Why is it so appealing mastering chaotic dynamics? Because chaotic systems have a wealth of
dynamical solutions. In fact, the skeleton of a chaotic attractor is a collection of an infinite number
of periodic orbits, each one being unstable. Every unstable periodic orbit gives a specific perfor-
mance. The overall chaotic orbit has a performance that is a weighted average of the performances
attained by the single periodic orbits. There are periodic orbits that give better performances than
the weighted average. Therefore, if the goal is to optimize the performance of the system, it is useful
to select the unstable high-performance periodic orbits.
How is it possible to succeed in this challenge? It seems like taming a crazy horse! The dynam-
ics in the chaotic attractor is ergodic, which means that during its time evolution, the system visits
the neighborhood of every point of the unstable periodic orbits embedded within the chaotic attrac-
tor. Therefore, there are two main strategies for mastering a chaotic dynamic: one based on feed-
back and the other on non-feedback methods (Boccaletti et al. 2000). The first approach includes
those means that select the perturbation based upon a knowledge of the state of the system. The
dynamic of the system is observed for suitable learning time, and when the ergodic chaotic orbit
comes close to the desired unstable periodic orbit, a small “kick” is given to place the system on
or very close to the desired periodic orbit. The small “kick” could be a tiny perturbation to either
a control parameter of the system or a system state variable accessible to the operator. Even if the
“kick” is effective, the system, quite easily, will move far apart from the desired unstable periodic
orbit due to the noise and the intrinsic instability of the periodic orbit. Therefore, other small
“kicks” will be needed to be reapplied to reposition the system orbit closer to the desired periodic
orbit. By following this procedure, continuously, the system can be kept close to the desired peri-
odic orbit indefinitely. The second strategy for mastering chaotic dynamics consists in perturbing
periodically or stochastically the system, even without knowing the actual dynamical state, pro-
ducing drastic changes and leading eventually to the stabilization of some periodic behavior. The
limit of the second strategy is that it is not goal-oriented, and the operator cannot decide the final
periodic orbit.
What is impressive is that thanks to chaos, it is possible to produce an infinite number of dynami-
cal behaviors using the same system, with the only help of adequately chosen tiny perturbations.
This action is not the case for a non-chaotic system; typically, small disturbances can only change
their dynamics slightly. We need disturbances of the same order of magnitude of the values that
the dynamical variables assume in the unperturbed evolution to steer a non-chaotic system in the
direction we desire.
354
Untangling Complex Systems
10.9.1 aPPlicaTions
The idea of mastering chaos has stimulated applications in widely diverse fields of study; for
example, in mechanics (by controlling the chaotic vibrations of a magneto-elastic ribbon [Ditto
et al. 1990]), in fluid mechanics (by controlling chaotic convection [Singer et al. 1991]), in elec-
tronics (by controlling a chaotic diode resonator circuit [Hunt 1991]), in nonlinear optics (by
controlling the chaotic output of a laser [Roy et al. 1992]), in chemistry (by controlling the
Belousov-Zhabotinsky [Petrov et al. 1993] or the combustion [Davies et al. 2000] or the peroxidase-
oxidase [Lekebusch et al. 1995] reactions when they run in chaotic regime), in physiology (by
controlling chaos in heart beating [Garfinkel et al. 1992], and maintaining the chaotic state in
the neuronal activity of hippocampal slices that become periodic in the case of epileptic seizures
[Schiff et al. 1994]).
Two applications have attracted considerable attention in the scientific community over the past
few years; namely the control of chaotic dynamics for communicating and for computing.
10.9.1.1 Communication by Chaotic Dynamics
When we studied the Kolmogorov-Sinai entropy, we learned that any chaotic system could be
viewed as an information source that produces signals. In fact, a chaotic dynamic is a point
that follows a “strange” trajectory in its phase space. If the phase space is partitioned in many
pieces, each labeled by a different symbol, the chaotic system becomes a symbol source, because
a symbolic orbit is obtained by writing down the sequence of symbols corresponding to the
successive partition elements visited by the point in its orbit. Since the chaotic dynamic is a
continuous-time waveform source, it can also be transformed into a digital signal source. Slaving
the output of a chaotic oscillator allows for the transmission of the desired message, namely a
sequence of desired “high” and “low” values encoded with binary symbols “1” and “0”, respec-
tively (Boccaletti et al. 2000).
There is another approach for exploiting chaos in communication, and it is based on chaos syn-
chronization (Boccaletti et al. 2002). Two chaotic systems starting from different initial conditions
evolve in an unsynchronized manner. The feeding of the right bias from one system to another can
push the two systems into a synchronized state. The two chaotic systems remain in step with each
other during the transmission of a signal masked by the chaotic contribution. When the receiver syn-
chronizes to the transmitter, the message is decoded by subtraction between the signal sent by the
transmitter and its copy generated at the receiver. Finally, the true message is decoded (Boccaletti
et al. 2000).
10.9.1.2 Computing by Chaotic Dynamics
In principle, any chaotic system can mimic a Turing machine. A Turing machine (Turing 1936) is a
computing device consisting of a programmable read/write head with a paper tape passing through
it. It has inspired the design of electronic computers that are currently based on the Von Neumann’s
architecture (Burks et al. 1963). Such architecture consists of four components (remember what
we have learned in Chapter 2): a processor, making the computation; a memory, storing data and
instructions; an information exchanger, allowing the flow of information in and out of the com-
puter; an information bus, connecting the other three elements. The tape of the Turing machine
works as both the memory and the information exchanger. In fact, it is divided into squares, each
square bearing a single symbol (0 or 1), and it is of unlimited length. It serves both as a vehicle for
input and output, and as working memory for storing the intermediate results of a computation.
The read/write head is programmable, and it works as both the processor and the bus bearing infor-
mation from the head to the tape and vice versa. Although the head can perform a limited number
of operations, like reading and writing on the square of the tape under the head, moving the head
and halting, the Turing machine is capable of computing everything, if appropriately programmed.
The Emergence of Chaos in Time
355
A chaotic system can mimic a Turing machine. The time series that it generates acts as the tape,
which acts as information exchanger and memory, whereas the strange attractor that the chaotic
system traces when it evolves in its phase space plays as the head-processor of the peculiar chaos-
based Turing machine. In fact, it has been demonstrated that nonlinear deterministic systems can
emulate all the fundamental binary logic functions12 through a threshold-based morphing mecha-
nism (Ditto et al. 2010). The morphing mechanism grounds in three steps. For a chaotic system,
whose state is represented by the variable x, the first step is to consider the input. If xʹ represents input 0, then, in the second step, the temporal update of the state of the system, f( xʹ) (where f( x) is the function describing the time evolution of the system) is the output. Finally, in the third step,
after fixing a threshold value, x , the output f( x
, the
th
ʹ) is transformed in binary values: if f( xʹ) ≤ xth
output is 0, whereas if f( xʹ) > x , the output is 1. For encoding input 1, a specific increment Δ x will th
be added to xʹ, and f( xʹ + Δ x) will be its output. The most promising property of chaos computing is the ability to reconfigure the chaotic dynamic to any logic gate by exploiting the techniques
developed for controlling chaos.
So far, chaos-based computing has been implemented, almost exclusively, by conventional
Complementary Metal-Oxide Semiconductor (CMOS)-based Very Large Scale Integrated (VLSI)
circuits. Since nonlinear systems that exhibit chaotic dynamics are abundant in nature, we may
expect that several other physical and chemical systems can implement chaos-based computing;
for example, the “Hydrodynamic Photochemical Oscillators” we have studied in this chapter
(Hayashi et al. 2016; Gentili et al. 2017b).
10.10 KEY QUESTIONS
• When can we observe chaotic dynamics?
• Which are the possible evolutions of a system described by linear differential equations?
• When does a double pendulum originate chaotic dynamics?
• What is the equation that describes the dynamics of a population as a function of the food
available?
• Describe the possible solutions of the logistic map and the features of its bifurcation
diagram.
• When do convective rolls emerge?
• What is the difference between the Rayleigh and Marangoni numbers?
• How does Entropy Production change in the case of convection?
• Why is convection so important?
• Describe the features of the Lorenz’s model.
• What is a “Hydrodynamic Photochemical Oscillator?”
• How do we recognize chaotic time series?
• Can we predict chaotic time series?
• Is it possible to master chaotic dynamics?
• Which are possible applications of mastering chaos?
10.11 KEY WORDS
Linear and nonlinear equations; Double pendulum; Logistic map; Bifurcation diagram; Rayleigh
and Marangoni numbers; MaxEP; Butterfly effect; Lyapunov exponent; Strange attractors; Transient
Chaos; Hamiltonian Chaos; Dissipative Chaos; Artificial Neural Networks.
12 Any Boolean circuit can be built by an adequate connection of NOR and NAND logic gates. This implies that universal computing is feasible if the NOR and NAND gates can be implemented.
356
Untangling Complex Systems
10.12 HINTS FOR FURTHER READING
• An enjoyable and introductory reading about chaos is the book by Gleick (1987). An easy
and pleasant introduction to Chaos has been written by Feldman (2012), who has also pro-
posed interesting and basic courses about dynamical systems and chaos in the Complexity
Explorer website.
• The subject of the analysis of aperiodic time series applied to physics, engineering, biol-
ogy, and medicine can be studied in deep with the book by Kantz and Schreiber (2003),
