W h a t I s a C o m p l e x System?
This book aims to establish concepts and methods for analyzing economic systems through a unified, physically motivated perspective Economies are often viewed as complex social systems, but defining complex systems precisely remains an unresolved issue Heuristically, complex systems can be described as intricate networks of interrelated components that exhibit unpredictable behavior.
Complex systems consist of numerous interacting particles or elements, which can be similar or diverse These elements engage in interactions that can vary in complexity and are often characterized by nonlinear relationships.
In order to give this formal definition a physical context, we should qualitatively discuss some typical systems that may be denoted truly complex.
Branches of science present a range of examples, from simple to complex A straightforward illustration is granular matter, which consists of numerous similar granules The stability of granular systems is influenced by the shape, position, and orientation of these components, with the overall structure defined by the complete set of particle coordinates and shape parameters Additionally, when subjected to external force fields, the granules exhibit irregular movement and engage in numerous elastic collisions with one another.
A driven granular system exemplifies a complex system, characterized by its continuous structural changes influenced by external fields and interactions among its components.
Earth's climate is a complex system that includes the atmosphere, biosphere, cryosphere, and oceans, while also accounting for the influence of extraterrestrial factors like solar radiation and tides.
Computers and information networks represent a new category of complex systems, particularly in the realm of artificial intelligence hardware In this context, traditional logic and algebra are being transformed by advanced knowledge and learning processing techniques.
In biology, higher animals are intricate systems composed of various organs that interact closely, performing a multitude of complex functions Each organ is made up of highly specialized cells that work together in a well-coordinated manner.
The human brain is an incredibly complex organ, consisting of approximately 100 billion nerve cells that work together to enable functions such as recognizing visual and auditory patterns, speech, and other cognitive activities Each cell contains intricate structures, including a nucleus, ribosomes, mitochondria, and membranes, which are made up of numerous components At the most fundamental level, various biochemical processes occur simultaneously, including DNA replication and protein synthesis.
The hierarchy of systems can also be viewed in reverse, as animals create various types of societies Among these, the global human society stands out as the most intricate, particularly in its economic framework This system encompasses a wide array of participants, including managers, employers, and consumers, as well as capital goods like machines, factories, and research centers Additionally, it involves natural resources, traffic, and financial systems, categorizing it as a complex system Economic systems are intricately linked to broader human societies, reflecting diverse human activities alongside political, ideological, ethical, cultural, and communicative practices.
Complex systems exhibit permanent structural changes across different scales, reflecting an evolution that deviates from thermodynamic equilibrium The fundamental components of these systems are their elements, which typically possess limited degrees of freedom.
To understand the evolution of complex systems, we must begin at the microscopic level and examine how this perspective shifts at larger scales This transition often leads to a significantly different description of the system at the macroscopic level Two key challenges arise in this process: first, we need to define the relevant scales of interest, and second, we must demonstrate how the seemingly chaotic motion of individual elements can lead to notable collective phenomena at larger scales.
Defining appropriate microscopic and macroscopic scales can be challenging, particularly in ecology, where a hierarchy exists from molecular levels to animals, humans, and even economic systems We can approach this from either a quantum-mechanical perspective or a classical many-body system viewpoint However, accurately describing complex systems at the microscopic level requires an overwhelming amount of information, which is often unmanageable.
A macroscopic description enables significant data compression by focusing on large-scale properties rather than microscopic motions, making the selection of the appropriate macroscopic level crucial and dependent on the specific inquiry To analyze complex systems, it is essential to identify relevant quantities or variables that effectively represent these systems Each macroscopic system comprises collective large-scale quantities that are vital for understanding its characteristics For example, in financial market analysis, temporally variable asset prices serve as relevant quantities However, the intricate dynamics of the global economy remain obscured behind these price fluctuations, with numerous microscopic and macroscopic degrees of freedom forming a vast array of irrelevant observables compared to the limited relevant quantities.
Predicting the future evolution of relevant quantities in complex systems poses a significant challenge due to the strong coupling between relevant and irrelevant degrees of freedom Consequently, to accurately forecast the future values of the relevant degrees of freedom, one must also understand the precise dynamics of the irrelevant degrees of freedom.
This strategy, while seemingly futile due to the complexity of the underlying system, allows for the estimation of an upper limit on the total degrees of freedom By isolating a significant portion of the complex system along with its environment at a specific time, we can treat the momentary state as an initial condition for future evolution If the development of the relevant degrees of freedom mirrors that of an open system, the isolated system is sufficiently large to quantitatively describe the complex system it contains Mathematically, this approach embeds the evolution of the complex system within a clearly defined initial problem.
Determinism Versus Chaos
In this section, we will explore the mathematical analysis of a complex system, building on the global system outlined in the previous thought experiment This analysis encompasses both the complex system itself and a significant portion of its surrounding environment To accurately determine all time-dependent relevant quantities, it is essential to solve the complete set of microscopic equations of motion governing the global system.
The future evolution of embedded complex systems is fully predictable due to deterministic Newtonian mechanics, which asserts that if the momentum and positions of all particles are known at a given moment, their trajectories can be calculated However, this deterministic principle has been challenged twice in the realm of modern physics.
Quantum mechanics reveals that while we cannot precisely predict a particle's trajectory, the deterministic nature of wave functions remains intact Additionally, the theory of deterministic chaos indicates that in classical mechanics, predictability relies on having complete and precise knowledge of the system's microscopic configuration Chaos is absent in linear systems, where the output is directly proportional to the input, exemplified by the superposition principle This principle asserts that the combination of two solutions yields another valid solution within linear mechanical systems, which are generally well understood apart from some technical issues The emergence of chaos in nonlinear mechanical systems necessitates the breakdown of linearity and the superposition principle.
Nonlinearity is essential for chaotic behavior, but it is not the sole factor; for example, the simple pendulum's nonlinear equation produces predictable elliptic functions, lacking randomness Additionally, solitons exemplify regular collective motion in nonlinear systems, where their stability arises from the balance between nonlinearity and dispersion effects.
Classical mechanics addresses standard problems like falling bodies, pendulums, and the dynamics of planetary systems, often involving simple systems like a single planet and the sun, which require minimal degrees of freedom These iconic examples facilitated the quantitative development of mechanics by pioneers such as Galileo and Newton, who focused primarily on one- or two-body problems, devoid of chaotic behavior.
As the number of degrees of freedom in a mathematical system increases, the complexity of the processing and the variety of potential solutions grow significantly This heightened complexity leads to a greater tendency toward chaos, making the system appear unpredictable In deterministic, mechanical many-body systems, this unpredictability stems from their sensitive dependence on initial conditions, which can only be measured approximately due to the limitations of measuring instruments.
In order to understand this statement, we analyze a microscopic, mechanical system with 2N degrees of freedom The dynamics can be rewritten in terms of deterministic Hamilton’s equations as dq i
In the context of Hamiltonian mechanics, the generalized coordinates \( q_i \) (where \( i = 1, \ldots, N \)) and their corresponding generalized momenta \( p_i \) define the degrees of freedom of a system These microscopic degrees of freedom can be represented as a supervector \( \Gamma = \{ q_1, \ldots, q_N, p_1, \ldots, p_N \} \), with each vector \( \Gamma \) corresponding to a specific microscopic state Consequently, the entire system can be visualized as a point within a 2N-dimensional phase space, characterized by 2N axes that reflect the degrees of freedom This phase space is crucial for understanding the dynamics of many-body systems.
In the 2N-dimensional phase space, nearly all system trajectories exhibit instability when faced with minor perturbations The stability of any given trajectory in response to infinitesimally small disturbances is examined through the analysis of Lyapunov exponents This approach is crucial for understanding the system's behavior under slight changes.
In a schematic phase space, an infinitesimally small ball with an initial radius ε(0) represents the starting position of neighboring trajectories Over time, as the dynamics unfold, this ball transforms into an ellipsoid, with its center moving through the phase space The distortion of the ball, due to its infinitesimal nature, is described by linearized theory, ensuring it maintains its ellipsoidal shape with 2N principal axes εα(t) The Lyapunov exponents are then defined as λα = lim t → ∞ lim ε(0) → 0, capturing the behavior of these evolving trajectories.
The limit as ε approaches 0 is crucial because, with a finite radius ε, the ball cannot be accurately depicted as an ellipsoid as time increases, due to the emergence of nonlinear effects Additionally, the long-time limit as t approaches infinity is essential for collecting sufficient data to represent the complete trajectory It is important to note that the distance between infinitesimal neighboring trajectories diverges if at least one Lyapunov exponent has a positive real part.
When the initial ball has a finite diameter, it undergoes significant distortion, transforming into an amoeba-like shape This shape eventually extends into extremely fine filaments that disperse throughout the entire accessible phase space Such a mixing flow is a defining characteristic of systems with a high degree of complexity.
The existence of Lyapunov exponents with a positive real part in microscopic mechanical systems is still under investigation Due to time-reversal symmetry, each Lyapunov exponent has a corresponding exponent with an opposite sign Therefore, regular behavior is anticipated only when the real parts of all Lyapunov exponents are zero.
Fig 1.2 The deformation of a finite ball of the phase space in the course of its time evolution.
Complicated many-body systems typically do not exhibit predictable behavior, as demonstrated by computer simulations Even relatively simple mechanical systems with limited degrees of freedom can display chaotic behavior Furthermore, economic systems like stock markets, travel networks, and human societies show significant chaotic behavior on microscopic scales due to their vast number of degrees of freedom.
The Probability Distribution
Many-body systems often surpass the limits of traditional mathematical mechanics, yet we can analyze their properties using statistical physics methods To achieve this, we need a comprehensive concept that characterizes complex systems at the microscopic level This concept must address the inherent unpredictability of chaotic motions in many-body systems while also serving as a foundation for linking various significant quantities at the macroscopic scale.
Accurate predictions regarding the microscopic movement of particles are unattainable without precise initial conditions However, we can estimate the probability of a specific microscopic state based on the system's preparation or through appropriate empirical evaluations.
Probability is easily understood through simple games like dice or coin tossing, where it is defined by the relative frequency of outcomes after countless repetitions This concept highlights our limited knowledge about uncertain events, such as the results of a dice game However, the traditional frequency-based definition of probability, established in the early 20th century, falls short when applied to complex systems like financial markets, traffic networks, and human societies.
We cannot determine frequencies through repeated scientific experiments due to a lack of information on possible outcomes A solution to this challenge is interpreting probability as a degree of belief in an event's occurrence, a concept rooted in Bayes' original perspective Bayesian statistics is highly versatile, applicable to any conceivable process regardless of the ability to repeat it under controlled conditions This approach is especially relevant for economic and social systems, where scientific experimentation is often impractical.
To define the concept of probability more accurately, we utilize set theory terminology In this context, the components are referred to as events, which reflect their probabilistic characteristics Specifically, each microscopic state, represented by a vector Γ within the phase space, corresponds to a distinct event in the underlying microscopic system.
Events are categorized into different sets, including the universal set of all events, denoted as Ω, and the empty set representing no events An arbitrary region within the phase space corresponds to a specific set of events, while the entire phase space is interpreted as the universal set Ω Additionally, all possible sets of events create a closed system when subjected to union and intersection operations.
Probability is defined as a measure, P(A), representing our belief in the occurrence of an event within set A, where P(A) is always nonnegative (P(A) ≥ 0) and normalized to one (P(Ω) = 1) For two nonoverlapping sets A and B, where A∩B = ∅, the probability of an event occurring in A∪B is the sum of the probabilities of the event occurring in either A or B, establishing a fundamental axiom of probability theory.
We generalize this relation to a countable collection of nonoverlapping sets
A i (i= 1,2, ) such thatA i ∩A j =∅ for alli=j and obtain
Following our exploration of set theory, we can proceed with our initial problem by examining the set of events A(Γ), which represents the system's presence within an infinitesimal volume element dΓ = N i=1[dq i dp i] in the phase space, centered around the microscopic state Γ.
The primary challenge is to determine the probability P(A(Γ), t), which indicates the likelihood that a microscopic state belongs to A(Γ) at time t This a priori probability is based on our experiential assumptions and reflects our degree of belief It is useful to express this as dP = ρ(q1, , qs, p1, , ps, t)dq1 dqsdp1 dps = ρ(Γ, t)dΓ, where ρ(Γ, t) represents the probability density for the state Γ at time t By definition, this density is a nonnegative quantity, and any finite region R of the phase space can be viewed as a collection of infinitesimal volume elements Consequently, the probability of locating a microscopic state within this region at time t can be calculated accordingly.
If we expand the region R over the whole phase space, we receive the normalization condition ρ(Γ, t)dΓ = 1 (1.7)
This equation corresponds to P(Ω) = 1 in the language of set theory, reflecting our knowledge that the certainty of finding the system somewhere in the phase space is always true.
Bayesian statistics offers a probabilistic framework that is essential when dealing with limited information about a system Even with sufficient data, this approach remains advantageous for modeling complex systems While the mathematical foundation of a probability distribution based on microscopic principles can be intricate, the probabilistic concept allows for addressing the challenge posed by the finite accuracy of measurable initial conditions.
The Liouville Equation
This article explores the possibility of determining the evolution of the probability distribution for a specific microscopic system We begin by considering an initial state, Γ 0, which is realized with a certain probability, represented as ρ(Γ 0, t 0)dΓ.
In deterministic microscopic motion, the initial state transitions to a different microscopic state along the trajectory Γ(t) = {q i (t), p i (t)}, adhering to the boundary condition Γ(0) = Γ 0 This process ensures that the probability density ρ remains conserved along each trajectory of the complex system, necessitating the condition dρ/dt = ∂ρ.
After replacing the velocities ˙q i and forces ˙p i by the equations (1.1), we arrive at
∂t + ˆLρ= 0, (1.9) where we have introduced the Liouvillian of the system
The Liouville equation, known as relation (1.9), is a fundamental equation in statistical physics, akin to the significance of the Schrödinger equation in quantum mechanics In this context, the Liouvillian serves a role parallel to that of the Hamiltonian in Newtonian mechanics, establishing the rules governing the evolution of microscopic states In statistical physics, the Liouvillian defines the equations of motion, represented by the distribution function ρ.
In microscopic systems, the mechanical and statistical representations of evolution are fundamentally equivalent The distinction between these two descriptions arises from the definition of the objects involved: classical mechanics focuses on points in phase space, whereas statistical physics centers around distribution functions.
The Liouville equation plays a crucial role in understanding the evolution of economic systems and other complex systems by integrating probabilistic and deterministic aspects of their dynamics According to Bayesian statistics, an economic situation at an initial time \( t_0 \) can be characterized by a probability distribution \( \rho(\Gamma, t_0) \) The Liouville equation then serves as a deterministic framework that maps this initial distribution to a new probability distribution \( \rho(\Gamma, t) \) at a later time \( t > t_0 \), effectively preserving our degree of belief throughout the evolution process.
Econophysics
Economics primarily focuses on choice and decision-making related to scarce resources It examines how individuals optimally utilize limited resources, highlighting its role as a discipline that addresses specific aspects of reality concerning scarcity.
Economics examines the behavior of key decision-making entities, including households, financial markets, and government agencies It views the outcomes of economic processes as the result of both intrinsic mechanisms within the economic system and various external factors that influence these processes.
Economists are recognizing that economic systems are deeply intertwined with their human and natural environments They understand that many seemingly external factors are actually components of a broader, intricate system of physical, biological, and social mechanisms that influence the evolution of our planet.
Physics offers valuable insights into economic issues through its analytical methodology, which involves breaking down complex systems into their individual components This approach allows for a deeper understanding of various phenomena, including migration, commuting, production choices, financial transactions, traffic, and transportation, by applying physical methods to analyze their underlying principles.
To effectively analyze economic systems, it is essential to identify key quantities that capture their fundamental characteristics This challenge remains a central focus of economic research A significant aspect of this endeavor involves formulating general equations that illustrate the dynamics of these quantities and their interrelationships, grounded in universal principles.
A second task should be the solution of these equations by applying modern techniques of statistical physics This is, roughly speaking, the field of econophysics.
Econophysics offers fresh insights into economic decision-making and risk management, enhancing our comprehension of complex systems However, it is crucial to recognize that econophysics should not be viewed as a substitute for traditional and modern economic sciences, which still demand a solid foundation in economic knowledge.
S 0 m e N o t a t i o n s of P r o b a b i l i t y T h e o r y
Probability Distribution of Relevant Quantities
The microscopic probability distribution ρ(Γ, t) encompasses all degrees of freedom equally However, even if this function could be determined, its impracticality arises from the vast number of degrees of freedom it contains, rendering it unusable for analyzing complex systems.
In the study of complex systems, we focus on a limited set of relevant degrees of freedom, which can be seen as a form of reductionism However, defining which degrees of freedom are pertinent for describing these systems remains ambiguous As discussed in the previous chapter, the relevant quantities are determined empirically based on the specific problem at hand.
To analyze the complete phase space, we divide it into two subspaces: one containing the relevant degrees of freedom and the other comprising the irrelevant degrees of freedom Each microscopic state, denoted as Γ, can be expressed as a combination of the relevant degrees of freedom set Y = {Y 1, Y 2, Y N rel} and the irrelevant degrees of freedom set Γ irr, leading to the formulation Γ = Y relevant degrees of freedom + Γ irr irrelevant degrees of freedom.
The microscopic probability density may be written asρ(Γ, t) =ρ(Y, Γ irr , t).
In order to eliminate the irrelevant degrees of freedom, we integrate the probability density overΓ irr p(Y, t) dΓ irr ρ(Y, Γ irr , t) (2.2)
The probability density p(Y, t) effectively characterizes complex systems by transitioning from a microscopic to a macroscopic representation through the elimination of irrelevant degrees of freedom This normalization ensures that the integral of p(Y, t) over all relevant variables equals one, highlighting its significance in statistical mechanics.
The integration over all irrelevant degrees of freedom means that we suppose a maximal measure of ignorance of these quantities.
In geometrical terms, the relevant degrees of freedom in a system can be visualized as a point within an N-dimensional subspace of the phase space When an observer focuses solely on the relevant data, they may perceive an unpredictable evolution of the system's macroscopic quantities.
The dynamical evolution of relevant quantities is influenced by hidden irrelevant degrees of freedom at microscopic scales Consequently, different microscopic trajectories in phase space can yield the same macroscopic outcomes, while identical macroscopic initial conditions may evolve into divergent paths.
Despite having precise knowledge of initial conditions, we cannot fully predict future developments This limitation, imposed by focusing on relevant quantities, results in a continuous loss of certainty in our beliefs.
The average of an arbitrary functionf(Γ) is obtained by adding all values off(Γ) considering the statistical weightρ(Γ, t)dΓ Hence f(t) dΓ ρ(Γ, t)f(Γ) (2.4)
The mean value can vary over time due to the time-dependent nature of the probability density When the function f is solely dependent on the relevant degrees of freedom, represented as f = f(Y), we can express this relationship mathematically as f(t) dΓ ρ(Γ, t)f(Y) dY p(Y, t)f(Y).
The distribution function p(Y, t) conceals the dynamics of irrelevant degrees of freedom, while the relevant probability density fulfills all criteria required for a comprehensive description of a complex system based on the chosen relevant degrees of freedom.
Measures of Central Tendency
In the analysis of probability distribution functions, we can simplify a multivariable probability density p(Y, t) to a single variable function by integrating over all other degrees of freedom, leaving only one relevant degree of freedom This reduction facilitates a clearer understanding of the underlying probabilities.
To determine the typical value of a complex problem when the probability distribution function p(Y, t) is known, one often refers to the mean or average, denoted as y(t) = ∫ Y p(Y, t) dY However, it is important to note that there is no definitive answer to this question.
There are two other major measures of central tendency The probability
P < (y, t) gives the time-dependent fraction of events with values less theny,
The functionP < (y, t) increases monotonically withyfrom 0 to 1 Using (2.7), the central tendency may be characterized by the mediany 1/2 (t) The median is the halfway point in a graded array of values,
Finally, the most probable valuey max(t) is another quantity describing the mean behavior This quantity maximizes the density function
In cases where an equation yields multiple solutions, the most likely value, denoted as y max(t), corresponds to the highest probability p It is essential to note that, aside from unimodal symmetric probability distribution functions, the three quantities exhibit distinct differences Understanding these differences is crucial for accurately interpreting empirical averages derived from a limited set of observations.
In a limited number of trials, sampling the most probable value first and averaging these measures will yield results close to y max(t) Conversely, as the number of observations increases, the empirical average increasingly aligns with the true average y(t).
2.1.3 Measure of Fluctuations Around the Central Tendency
In analyzing a single degree of freedom, repeated observations of this variable are anticipated to cluster around a central tendency The range of this clustering is indicative of deviations from that central tendency One effective way to quantify this range is by calculating the average of the absolute values of the spread.
The absolute value of the spread is undefined for probability distribution functions that decay at a rate of Y − 2 or slower for large values of Y An alternative measure of dispersion is the standard deviation (σ), which is calculated as the square root of the variance (σ²).
The standard deviation does not always exist, such as for probability densities p(Y, t) with tails decaying as or slower thanY − 3
In the context of multivariable probability distribution functions, we examine p(Y, t) where Y consists of relevant degrees of freedom, denoted as Y = {Y1, Y2, YN rel} Here, N rel represents the dimensionality of the vector Y The moments of order n are calculated using the average m(n) α1α2 αn(t) integrated over the n-dimensional space of Y.
The first moment, denoted as m(1)α(t), represents the mean yα(t) of component α, establishing a formal vector m1(t) that generalizes the central tendency of the underlying dynamics Additionally, the second moment, m(2)αβ(t), is defined as the average m(2)αβ(t) = ∫Y p(Y, t)YαYβ dY, capturing the relationship between components α and β over time.
The quantities referred to are components of the correlation matrix For the definition to hold true, the integral on the right-hand side must converge, indicating that a necessary condition for the existence of a moment of order n is that the probability density function decreases at a rate faster than a specific threshold.
|Y| − n − N rel for|Y| → ∞ This is trivially obeyed for probability distribution functions that vanish outside a finite region of the space of relevant degrees of freedom.
Statistical problems frequently utilize moments to simplify the complex task of understanding the complete functional behavior of probability densities In many practical scenarios, knowing all moments can be equivalent to understanding the probability distribution function However, achieving strict equivalence between moments and probability density necessitates additional constraints.
The moments of a probability distribution are intricately linked to its characteristic function, which is defined as the Fourier transform of the probability distribution \( \hat{p}(k, t) \) given by the equation \( \int dY \, e^{ikY} p(Y, t) \) This relationship can be inverted to express the probability distribution in terms of its characteristic function, represented as \( p(Y, t) = \frac{1}{(2\pi)^N} \int d^N k \, e^{-ikY} \hat{p}(k, t) \).
Thus, the normalization condition (2.3) is equivalent to ˆp(0, t) = 1, and the moments of the probability density can be obtained from derivatives of the characteristic function atk= 0: m (n) α 1 α 2 α n (t) = (−i) n n l=1
If all moments exist, the characteristic function may also be presented as the series expansion ˆ p(k, t) ∞ n=0
The inversion formula demonstrates that distinct characteristic functions correspond to unique probability distribution functions, confirming the true nature of the characteristic function ˆ p(k, t) Furthermore, the simple derivation of moments from equation (2.16) highlights the direct relevance of the characteristic function to measurable quantities.
Another important function is the cumulant generating function, which is defined as the logarithm of the characteristic function Φ(k, t) = ln ˆp(k, t) (2.18)
This leads to the introduction of the cumulantsc α 1 α 2 α n (t) as derivatives of the cumulant generating function atk= 0, c (n) α 1 α 2 α n (t) = (−i) n n l=1
Each cumulant of ordernis a combination of moments of orderl≤n, as can be seen by substitution of (2.17) and (2.18) into (2.19) We get for the first cumulants c (1) α =m (1) α c (2) αβ =m (2) αβ −m (1) α m (1) β c (3) αβγ =m (3) αβγ −m (2) αβ m (1) γ −m (2) βγ m (1) α −m (2) γα m (1) β + 2m (1) α m (1) β m (1) γ
First-order cumulants represent the averages of individual components \( Y^\alpha \) In contrast, second-order cumulants characterize the covariance matrix \( \sigma \), where the elements are defined as \( \sigma_{\alpha\beta} = c^{(2)}_{\alpha\beta} = m^{(2)}_{\alpha\beta} - m^{(1)}_{\alpha} m^{(1)}_{\beta} \) This covariance serves as a generalized measure of the relationship between the values.
Y deviate from the central tendencies In particular, for a single variable
Moments and Characteristic Functions
In the context of multivariable probability distribution functions, we examine p(Y, t) where Y represents a set of relevant degrees of freedom, denoted as {Y1, Y2, YN rel} The dimension of vector Y corresponds to the number of relevant degrees of freedom, N rel The moments of order n are defined through the average m(n) α1 α2 αn(t), integrated over the variables in Y.
The first moment, denoted as m(1)α(t), represents the mean yα(t) of component α, establishing a formal vector m(1)(t) that generalizes the central tendency of the underlying dynamics In contrast, the second moment, m(2)αβ(t), is defined as the average m(2)αβ(t) = ∫Y p(Y, t)YαYβ dY, capturing the relationship between components α and β over time.
The quantities referred to are components of the correlation matrix For the definition to hold significance, the integral on the right side must converge Therefore, a necessary condition for the existence of a moment of order n is that the probability density function decreases at a rate faster than a specified threshold.
|Y| − n − N rel for|Y| → ∞ This is trivially obeyed for probability distribution functions that vanish outside a finite region of the space of relevant degrees of freedom.
Statistical problems frequently utilize moments to simplify the complex task of defining the complete behavior of probability densities In many practical scenarios, having access to all moments can be as informative as knowing the probability distribution function itself However, achieving a strict equivalence between moments and probability density necessitates additional constraints.
The moments of a probability distribution are intricately linked to its characteristic function, defined as the Fourier transform of the distribution \( \hat{p}(k, t) = \int dY \, e^{ikY} p(Y, t) \) This relationship allows us to express the probability distribution in terms of its moments, leading to the inverse relation \( p(Y, t) = \frac{1}{2\pi} \int dk \, e^{-ikY} \hat{p}(k, t) \).
Thus, the normalization condition (2.3) is equivalent to ˆp(0, t) = 1, and the moments of the probability density can be obtained from derivatives of the characteristic function atk= 0: m (n) α 1 α 2 α n (t) = (−i) n n l=1
If all moments exist, the characteristic function may also be presented as the series expansion ˆ p(k, t) ∞ n=0
The inversion formula demonstrates that distinct characteristic functions correspond to unique probability distribution functions, highlighting the true nature of the characteristic function ˆ p(k, t) Furthermore, the simple derivation of moments through equation (2.16) establishes a direct connection between the characteristic function and measurable quantities.
Cumulants
Another important function is the cumulant generating function, which is defined as the logarithm of the characteristic function Φ(k, t) = ln ˆp(k, t) (2.18)
This leads to the introduction of the cumulantsc α 1 α 2 α n (t) as derivatives of the cumulant generating function atk= 0, c (n) α 1 α 2 α n (t) = (−i) n n l=1
Each cumulant of ordernis a combination of moments of orderl≤n, as can be seen by substitution of (2.17) and (2.18) into (2.19) We get for the first cumulants c (1) α =m (1) α c (2) αβ =m (2) αβ −m (1) α m (1) β c (3) αβγ =m (3) αβγ −m (2) αβ m (1) γ −m (2) βγ m (1) α −m (2) γα m (1) β + 2m (1) α m (1) β m (1) γ
First-order cumulants represent the averages of individual components Yα, while second-order cumulants establish the covariance matrix σ, defined by the elements σαβ = c(2)αβ = m(2)αβ - m(1)α m(1)β This covariance serves as a generalized metric for assessing the relationship between the values.
Y deviate from the central tendencies In particular, for a single variable
Y, the second-order cumulant is equivalent to the varianceσ 2 Higher-order cumulants contain information of decreasing significance Especially if all higher-order cumulants vanish, we can easily deduce using (2.18) and (2.15) that the corresponding probability densityp(Y, t) is a Gaussian probability distribution p(Y, t) = 1
The Marcienkiewicz theorem demonstrates that, in a given distribution, either all cumulants except the first two are zero, or there exists an infinite number of non-zero cumulants This implies that the cumulant generating function cannot exceed a polynomial degree of two.
Higher-order cumulants provide insight into deviations from Gaussian behavior in probability distributions For a single variable Y, the normalized third-order cumulant, known as skewness (λ 3 = c (3) /σ 3), measures the asymmetry of the distribution Additionally, the fourth-order cumulant, referred to as excess kurtosis (λ 4 = c (4) /σ 4), quantifies the initial correction to Gaussian behavior in symmetric distributions.
Generalized Rate Equations
The Formal Solution of the Liouville Equation
We can formally integrate the Liouville equation (1.9) to obtain the solution ρ(Γ, t) = exp
The expression examines the microscopic equations of motion linked to the Liouvillian operator ˆL The operator exp{−Ltˆ } serves as the time propagator for the system's dynamical variables To enhance comprehension of the time propagator's significance, we can expand the exponential function in terms of powers of t: ρ(Γ, t).
The right-hand side of the equation can be seen as a perturbative solution derived from the iterative integration of the Liouville equation To illustrate this, we reformulate the Liouville equation into an integral equation, which allows us to establish the mapping ρ(n+1)(Γ, t) = ρ(Γ, 0) - t.
Lρˆ (n) (Γ, τ)dτ (2.25) with the initial functionρ (0) (Γ, t) =ρ(Γ,0) The seriesρ (0) , ρ (1) , , ρ (n) , converges eventually against the solutionρ(Γ, t) of the Liouville equation In fact, we receive ρ (1) =ρ (0) −Lρˆ (0) t ρ (2) =ρ (0) −tLρˆ (0) +t 2
As expected, the solutionsρ (0) , ρ (1) , ρ (2) , of the hierarchical system (2.25) are identical with the first terms of the expansion (2.24).
For complex systems, the formal solution is often too intricate to be practically useful Therefore, to effectively describe the dynamic behavior of these systems, we need to explore alternative approaches.
The Nakajima-Zwanzig Equation
Understanding the relevant probability density p(Y, t) is essential for analyzing complex systems at specific degrees of freedom We can derive this function from the complete microscopic probability distribution ρ(Γ, t) using our prior knowledge Initially, solving the Liouville equation with all microscopic degrees of freedom is necessary Afterward, we can eliminate the irrelevant degrees of freedom from the microscopic distribution function through integration.
To address the unrealistic nature of the procedure, we aim to determine if an equation exists that accurately describes the evolution of p(Y, t) while incorporating only the relevant degrees of freedom.
To obtain the distribution function of the irrelevant degree of freedom, we can eliminate the relevant degrees of freedom from each microscopic probability distribution This leads us to the function ρ irr(Γ irr , t) dY ρ(Y, Γ irr , t), while ensuring that the normalization condition dΓ irr ρ irr(Γ irr , t) = 1 is satisfied.
The product of the probability distributions (2.27) at the initial timet 0 and (2.2) at the timet is again a probability density ρ(Y, Γ irr , t, t 0) =ρ irr(Γ irr , t 0)p(Y, t) (2.29)
The surrogate probability distribution differs from the microscopic probability density ρ(Γ, t), yet the average values of any functions based on the relevant degrees of freedom remain consistent when using the density ρ This is illustrated by the equation p(Y, t) dΓ irr = ρ(Y, Γ irr, t) dΓ irr = ρ(Y, Γ irr, t, t 0 ).
The surrogate probability distribution generation, known as projection formalism, involves applying a projection operator to the probability distribution function This process can be represented symbolically as ρ(Y, Γ irr, t, t 0) = ˆP ρ(Γ, t), highlighting the use of a specialized projection operator.
Apart from ˆP, we still need the complementary operator ˆQ= 1−Pˆ Using (2.32), it is simple to demonstrate that these operators have the “idempotent” properties
Pˆ 2 = ˆP , Qˆ 2 = ˆQ, and PˆQˆ = ˆQPˆ= 0, (2.33) typically for all projection operators The first equation is a direct consequence of (2.32), while the last two follow from
To describe the time-dependent evolution of the relevant probability density, we must first establish the initial distribution at time t=0 While we can accurately define this distribution for simple physical systems through repeatable experiments, estimating it for social or economic phenomena requires varying degrees of accuracy based on our experience For the relevant degrees of freedom, we assume that their initial values, Y0, are well-known at time t=0, allowing us to express the probability density as p(Y, t0) = p(Y, t0 | Y0, t0) = δ(Y − Y0).
In this context, p(Y, t|Y₀, t₀) represents the probability density of the relevant degrees of freedom at time t, given the initial state Y₀ This procedure is applicable to all conceivable initial distributions, and we will later demonstrate that these cases can be represented by equation (2.36) However, we lack significant information regarding the irrelevant degrees of freedom.
In the context of Bayesian statistics, we can assume that relevant and irrelevant degrees of freedom are uncorrelated, particularly in the initial state This statistical independence between relevant and irrelevant degrees cannot be confirmed or denied, serving as an "a priori" assumption that reflects our belief level Based on these assumptions, the initial microscopic probability distribution can be expressed as ρ(Γ, t 0) = ρ irr(Γ irr, t 0)δ(Y − Y 0) = ρ(Y, Γ irr, t 0, t 0) (2.37), highlighting its defining properties.
Now, we apply the projection operator ˆP to the Liouville equation (1.9) and obtain
By substituting ρ(Γ, t) with the formal solution of the Liouville equation, as outlined in equation (2.23), while considering the initial time t=0, we obtain the following results.
PˆLˆQρˆ (Γ, t) = ˆPLˆQeˆ − L(t ˆ − t 0 ) ρ(Γ, t 0) (2.40) For the further treatment of this expression, we need the identity e − L(t ˆ − t 0 ) = e − L ˆ 1 (t − t 0 ) − t t 0 dt e − L ˆ 1 (t − t ) Lˆ 2 e − L(t ˆ − t 0 ) , (2.41) where we have split the Liouvillian into two arbitrary parts, ˆL 1 and ˆL 2, via
Lˆ = ˆL 1 + ˆL 2 This identity may be checked by the derivative with respect to the time
Then, substituting the integral kernel using (2.41), we obtain
=−Leˆ − L∆t ˆ (2.43) with ∆t = t−t 0 Thus, the identity (2.41) is proven In particular, if we replace ˆL 1 by ˆLQˆ and ˆL 2 by ˆLP, we getˆ e − L(t ˆ − t 0 ) = e − L ˆ Q(t ˆ − t 0 ) − t t 0 dt e − L ˆ Q(t ˆ − t ) LˆPeˆ − L(t ˆ − t 0 ) (2.44)
We substitute (2.44) into (2.40) so that we obtain
The first addend on the right-hand side vanishes due to a property derived from the Taylor expansion of the exponential function Although the expansion appears to be an infinite series, all coefficients must identically vanish as indicated by the previous equation To proceed, we express the integral kernel in a more symmetric manner, and by examining the relationship \( \hat{Q} = \hat{Q}^2 \), we draw our conclusions.
Qˆ = ˆQe − Q ˆ L ˆ Qτ ˆ Q.ˆ (2.46) From (2.23), we see that
PˆLˆQρˆ (Γ, t) =− t t 0 dt PˆLˆQeˆ − Q ˆ L ˆ Q(t ˆ − t ) QˆLˆP ρˆ (Γ, t ), (2.47) and coming back to (2.39), we obtain
By integrating the relationship across all irrelevant degrees of freedom, we derive a closed linear integro-differential equation for the probability distribution function pertaining to the relevant degrees of freedom This process is illustrated in equation (2.32).
LˆQeˆ − Q ˆ L ˆ Q(t ˆ − t ) QˆLρˆ irr(Γ irr , t 0) p(Y, t ) (2.49) or, more precisely,
+ t t 0 dt Kˆ(t 0 , t−t )p(Y, t |Y 0 , t 0), (2.50) where we have introduced the frequency operator
The Nakajima–Zwanzig equation, also known as the generalized rate equation, accurately represents the evolution of the relevant probability distribution function While it primarily focuses on this aspect, the complete dynamics of irrelevant degrees of freedom and their interactions with relevant degrees of freedom are encapsulated within the memory operator.
The operators ˆM and ˆK exhibit a significant dependency on the initial time t₀, highlighting that a complex system does not need to be in a stationary state Consequently, varying developments of the probability density p(Y, t| Y₀, t₀) can be observed in the same system with identical initial conditions but at different initial times.
The Nakajima–Zwanzig equation enables the prediction of the evolution of relevant probability distribution functions, provided the mathematical structures of the frequency and memory operators are known While it is possible to derive more general evolution equations using time-dependent projectors or those based on the probability distribution function, this approach sacrifices the useful convolution property inherent in the memory term Moreover, all evolution equations derived from projection formalisms are physically equivalent and mathematically accurate, indicating that no particular evolution equation holds a preferential status.
The primary challenge lies in accurately determining the operators ˆM and ˆK, as their complete determination requires solving the Liouville equation, which is impractical for complex systems Instead, we can adopt a heuristic approach to these operators in the Nakajima–Zwanzig equation, leveraging empirical experiences and mathematical insights Physical intuition is crucial throughout this process Additionally, when dealing with economic systems, it is essential to consider various economic, technological, and social factors.
Combined Probabilities
Conditional Probability
In the future, we will focus on the N-dimensional state Y and its corresponding state space, eliminating the need for additional designations related to degrees of freedom We will only revert to the previous notation when there is a potential for confusion or error.
As discussed in the previous section, p(Y, t|Y 0 , t 0 ) is the probability density that the system in the state Y 0 at time t 0 will be in the state Y at timet > t 0 Hence,
The expression R dY p(Y, t|Y 0 , t 0) (2.53) represents the probability of a system being in a specific state within region R at time t, given that it was in state Y 0 at an earlier time t 0 This conditional probability is crucial for understanding the time evolution of complex systems.
Conditional probability, denoted as P(A|B), refers to the likelihood of an event within set A occurring, given that we are aware it is also included in set B This concept can be articulated through the principles of set theory.
In this context, set A represents all system trajectories that intersect region R at time t, while set B includes trajectories that pass through point Y0 at time t0 Each trajectory is considered an event, with both A and B being subsets of the overall set Ω of all trajectories The expression P(A | B) = P(R, t | Y0, t0) indicates the probability that a trajectory in B also belongs to A Additionally, the normalization condition P(Ω | B) = 1 leads to the conclusion that dY p(Y, t | Y0, t0) = 1.
Statistical independence meansP(A | B) =P(A) (i.e., the knowledge that one event occurs inB does not change the probability that it occurs in A).
If P(A | B) > P(A), we say that A and B are positively correlated, while
P(A|B)< P(A) corresponds to a negative correlation betweenAandB.
Joint Probability
An event that belongs to both set A and set B is also included in the intersection A∩B The probability P(A∩B) represents the joint probability of the event occurring in both sets Additionally, there is a natural connection between conditional probabilities, joint probabilities, and unconditional probabilities.
Statistically independent events are defined by the condition P(A ∩ B) = P(A)P(B), which leads to the conclusions P(A|B) = P(A) and P(B|A) = P(B) A common example involves determining the probability of a complex system remaining within an infinitesimally small volume dY at time t, given it was in volume dY₀ at an initial time t₀ The corresponding infinitesimal joint probability can be expressed as dP(Y, t; Y₀, t₀) = p(Y, t; Y₀, t₀)dY dY₀, where p(Y, t; Y₀, t₀) is defined as p(Y, t|Y₀, t₀)p(Y₀, t₀).
Suppose that we know all setsB i that could condition the appearance of an event in the setA TheB i should be mutually exclusive,B i ∩B j =∅for all i=j, and exhaustive, i B i =Ω Thus, we obtain
If we take into account (A∩B i )∩(A∩B j ) =∅, we obtain due to (1.4) and (2.55)
This general relation specifies immediately in the case of the probability density to p(Y, t) p(Y, t|Y 0 , t 0)p(Y 0 , t 0)dY 0 (2.59) Because of the symmetryP(A∩B) =P(B∩A), we get also p(Y 0 , t 0) p(Y, t|Y 0 , t 0)p(Y 0 , t 0)dY (2.60)
Due to (2.54), the last equation is a simple identity Equation (2.56) permits in particular the extension of the initial condition (2.36) on any probability distributions.
It is essential to represent each joint probability distribution function, p(Y, t; Z, τ), in the specified form for accurate analysis To determine this joint probability for t > τ > t₀, one must compute the integral p(Y, t; Z, τ) dY₀ p(Y, t|Z, τ; Y₀, t₀) p(Z, τ|Y₀, t₀) p(Y₀, t₀) This calculation involves the conditional probability p(Y, t|Z, τ; Y₀, t₀), reflecting the complexity arising from the deterministic nature of microscopic dynamics, which may retain some memory at the level of relevant degrees of freedom.
The joint probability density p(Y, t; Z, τ; Y₀, t₀) is derived by integrating over all positions Y₀, with only full trajectories contributing to the conditional probability densities p(Z, τ | Y₀, t₀) and p(Y, t | Z, τ; Y₀, t₀) The relationship between these densities indicates that the dashed curves in p(Z, τ | Y₀, t₀) are excluded from p(Y, t; Z, τ) due to the filtering effect of the conditional probability Conversely, the product p(Y, t; |Z, τ) p(Z, τ | Y₀, t₀) p(Y₀, t₀) includes contributions from all relevant events, including those represented by dotted lines, particularly influencing p(Y, t | Z, τ) The equivalence p(Y, t; |Z, τ; Y₀, t₀) = p(Y, t; |Z, τ) holds true only in the absence of memory effects, such as those represented by the memory kernel in the Nakajima–Zwanzig equation, which suggests feedback between degrees of freedom When this feedback is absent, the simpler relationship p(Y, t; Z, τ) = p(Y, t | Z, τ) ρ(Z, τ) becomes valid for any time point.
The validity of the relation (2.56) is consistently upheld, as it is based on the assumption that both relevant and irrelevant degrees of freedom are initially uncorrelated, and prior information remains unknown It is essential to interpret this assumption within the framework of Bayesian statistics.
Markov Approximation
The challenge of selecting relevant degrees of freedom in complex systems is revisited By defining these relevant degrees in a manner where their changes occur at a slower rate compared to the irrelevant degrees of freedom, the memory kernel in the Nakajima-Zwanzig equation can be approximated effectively.
The Markov approximation is a useful representation for many complex systems, assuming a separation between slow relevant timescales and fast irrelevant ones However, it's important to note that no system truly exhibits a Markov character; on very fine timescales, the immediate history becomes crucial for predicting probabilistic outcomes Despite this, systems with short memory times can often be considered Markov-like within the timescale of our observations By substituting equation (2.62) into the Nakajima–Zwanzig equation (2.50), we further explore this approximation.
The evolution of the probability density p(Y, t|Y 0 , t 0 ) is governed by the equation ∂t =−Lˆ Markov p(Y, t|Y 0 , t 0 ), where the Markovian operator is defined as ˆL Markov= ˆM(t 0)−K(tˆ 0) In scenarios where certain irrelevant degrees of freedom exhibit significantly slower dynamics compared to the relevant ones, it becomes advantageous to utilize time-dependent projectors These projectors effectively capture the influence of the slow irrelevant dynamics, allowing for a more accurate derivation of the evolution equation for the probability density.
The generalization discussed does not alter the overall procedure for separating timescales, aside from introducing an explicit time dependence in the operator ˆL Markov(t) This allows for the continued application of the Markov approximation in these scenarios However, the separation of timescales may become ineffective or unpredictable if a set of irrelevant degrees of freedom presents characteristic timescales comparable to those of the relevant degrees of freedom By considering an infinitesimal time interval dt, we derive the expression p(Y, t+dt|Y 0 , t 0 ) from equation (2.63).
In general, we may express the operator 1−LˆMarkov(t)dt by an integral representation p(Y, t+dt|Y 0 , t 0 ) dZU Markov (Y, t+dt|Z, t)p(Z, t|Y 0 , t 0 ).(2.65)
We multiply (2.65) with the initial distribution function p(Y 0 , t 0) and integrate over all configurationsY 0 Considering (2.59), we get p(Y, t+dt) dZU Markov(Y, t+dt|Z, t)p(Z, t) (2.66)
The integral kernel U Markov(Y, t+dt|Z, t) represents the conditional probability density p(Y, t+dt|Z, t), indicating the transition from state Z at time t to state Y at time t+dt This is further clarified by the requirement in equation (2.61), which establishes a more general relationship: p(Y, t+dt) = ∫ dY' dZ p(Y, t+dt|Z, t; Y', t') × p(Z, t|Y', t') × p(Y', t').
A simple comparison between (2.66) and (2.67) leads to the necessary conditionp(Y, t+dt|Z, t;Y 0 , t 0) =p(Y, t+dt|Z, t) This is simply another formulation of the Markov property It is, even by itself, extremely powerful.
This property allows us to express higher conditional and joint probabilities using simple conditional probabilities By shifting the time from \( t \) to \( t + dt \), we derive a general relationship between conditional probabilities at different times, leading to the equation \( p(Y, t + 2dt | Y_0, t_0) = \int p(Y, t + 2dt | Z, t + dt) \times p(Z, t + dt | Y_0, t_0) \, dZ \).
On the other hand, the transformationdt→2dtleads to p(Y, t+ 2dt|Y 0 , t 0) dZp(Y, t+ 2dt|Z, t)p(Z, t|Y 0 , t 0) (2.69) so that we obtain from (2.65), (2.68), and (2.69) p(Y, t+ 2dt|Z, t) dXp(Y, t+ 2dt|X, t+dt) ×p(X, t+dt|Z, t) (2.70)
The Chapman–Kolmogorov equation emerges from repeatedly applying a specific procedure, resulting in a relation for finite time differences: p(Y, t|Z, t) dXp(Y, t|X, t)p(X, t|Z, t) This equation serves as a complex nonlinear functional relationship that connects all conditional probabilities derived from a given Markovian process.
The conditional probability density derived from any Markovian process must adhere to the Chapman-Kolmogorov equation, which serves as a crucial criterion for confirming the existence of the Markov property When the empirically determined conditional probabilities align with this equation, we can effectively establish the presence of the Markov property.
Determining the Markovian for a specific process through microscopic theory is likely not feasible; hence, we rely on empirical observations and considerations It is essential to establish certain rules that can aid in the construction of the Markovian operator ˆL Markov.
The parameter-free Chapman-Kolmogorov equation is a universal relation for all evolutionary processes exhibiting Markov properties Although this equation has multiple solutions, every solution of a specific equation must also satisfy the Chapman-Kolmogorov equation, regardless of the unique mathematical structure of the Markov operator This relationship allows us to glean insights into the general mathematical framework of the Markov operator Following Gardiner's approach, we define subsequent quantities for all ε > 0 to further explore this concept.
| Y − Z | ε We will see later that these quantities were chosen in a very natural way They can be obtained directly from observations or defined by suitable model assumptions.
To achieve our objective, we must construct the Markovian, ˆL Markov solely using these quantities, ensuring that any higher-order coefficients approach zero as ε tends to zero For example, the third-order quantity defined in this context must also conform to this condition.
| Y − Z |