Non-stationarity¶

The dark enemy of stationarity. You know, that assumptions that statistics papers start with, if they aren’t starting with “independent, identically distributed”. Many results that my meagre statistics skills leave me to understand work for i.i.d. assumptions can be made to go if you know that the distribution from which you sample is stationary and your sampling ergodic. But what if you know otherwise, that you distribution is either not stationary, or will take so long to sample the space on an ergodic walk that you may as well not bother waiting?

I’m working on a number of projects right now about evolving and emerging behaviours - of people, artificial agents, economies, what-have-you. Real and simulated. Frequently I find myself wanting to know what they are doing over time, to measure some statistic or other of these systems. However, I know, or suspect, that, at least where they are doing the interesting things that I care about, that they are not described by any stationary distribution. Instead, they are suffused with path-dependence, long-range correlation, trends, or possessed of ergodic walks that are just too long to be plausibly computed [on my laptop|within my research CPU allocation|before the heat-death of the universe].

Sampling from real, time-changing problems is an issue for real data, not just toy experiments - can anyone furnish me with a large ensemble of earths to check my global economic simulations against? - so I need to know how to do it better. (See also: post-normal science)

I would like to know how well we can get by with samples from non-stationary distributions. Or some techniques for working out how bad my approximations are. How long-term is the cycle in my system? How path dependent is it? How inhomogeneous? If I can’t run my simulation until I have a valid sample, what can I calculate locally that will give me insight into the larger system? Do I care that much of statistical machinery breaks down, or not? Should I be relieved that the best I can do in a certain circumstance may be to fall back upon plain old visual inspection of some graph or other?

What do machine learning people do with this? Innovation theorists? Finance market modellers? Catchment hydrologists? And Dame Nature, with that neat “evolution” trick of hers?

Hurst effect. Long-memory processes. Ergodic theorems.

• A Yatchew, W Hardle. 2006. Nonparametric state price density estimation using constrained least squares and the bootstrap. Journal of Econometrics.
• B LeBaron, W B Arthur, R Palmer. 1999. Time series properties of an artificial stock market. Journal of Economic Dynamics and Control. _. (Online)
• C E Monteleoni, T Jaakkola. 2003. Online learning of non-stationary sequences.
• Dinh-Tuan Pham, Jean-François Cardoso. 2001. Blind separation of instantaneous mixtures of nonstationary sources. Signal Processing, IEEE Transactions on. _.
• D L Stein, C M Newman. 1995. Broken ergodicity and the geometry of rugged landscapes. Phys. Rev. E. _. (Online)
• Eric M Delmelle, Pierre Goovaerts. 2009. Second-Phase Sampling Designs for Non-Stationary Spatial Variables.. Geoderma. _.
• F J Breidt, N Crato, P De Lima. 1998. The detection and estimation of long memory in stochastic volatility. Journal of Econometrics.
• F X Diebold, A Inoue. 2001. Long memory and regime switching. Journal of Econometrics.
• Gavin E Crooks. 2007. Measuring Thermodynamic Length. Phys. Rev. Lett.. _.
• G Baumann. 2010. Place, RF; Földes-Papp, Z. Meaningful interpretation of subdiffusive measurements in living cells (crowded environment) by fluorescence fluctuation miscroscopy. Curr. Pharm. Biotechnol.
• Girish Nathan, Gemunu Gunaratne. 2005. Set of measures to analyze the dynamics of nonequilibrium structures. Phys. Rev. E. _.
• H Künsch. 1986. Discrimination between monotonic trends and long-range dependence. Journal of applied Probability.
• Hong Qian. 2001. Relative entropy: Free energy associated with equilibrium fluctuations and nonequilibrium deviations. Phys. Rev. E. _.
• H R Künsch. 1989. The jackknife and the bootstrap for general stationary observations. The Annals of Statistics.
• I Berkes, L Horváth, P Kokoszka, Q M Shao. 2006. On discriminating between long-range dependence and changes in mean. The Annals of Statistics.
• Jan Beran. 1992. Statistical Methods for Data with Long-Range Dependence. Statistical Science.
• Jan Beran. 2010. Long-range dependence. Wiley Interdisciplinary Reviews: Computational Statistics. (Online)
• J Beran. 1994. Statistics for long-memory processes, vol. 61 of Monographs on Statistics and Applied Probability.
• Jean Opsomer, Yuedong Wang, Yuhong Yang. 2001. Nonparametric Regression with Correlated Errors. Statistical Science. (Online)
• Jean-René Chazottes. An introduction to fluctuations of observables in chaotic dynamical systems.
• Jeffrey E Steif. 1997. Consistent estimation of joint distributions for sufficiently mixing random fields. The Annals of Statistics.
• John C Mauro, Prabhat K Gupta, Roger J Loucks. 2007. Continuously broken ergodicity.. J Chem Phys. _.
• John V Shebalin, Shebalin, JohnV.. Broken ergodicity in two-dimensional homogeneous magnetohydrodynamic turbulence. Physics of Plasmas. _.
• John V Shebalin. 1996. Absolute equilibrium entropy. Journal of Plasma Physics. _.
• John V Shebalin, Shebalin, JohnV.. 2007. Broken symmetries and magnetic dynamos. Physics of Plasmas. _.
• J R Chazottes, D Gabrielli. 2005. Large deviations for empirical entropies of g-measures. Nonlinearity. _. (Online)
• J V Shebalin. 2010. Broken ergodicity in two-dimensional homogeneous magnetohydrodynamic turbulence. Physics of Plasmas. _.
• J W Fisher III, Alexander T Ihler, Paul A Viola. Learning Informative Statistics: A Nonparametric Approach. Learning Informative Statistics: A Nonparametric Approach.
• Katakin Marton, Paul C Shields. 1994. Entropy and the consistent estimation of joint distributions. The Annals of Probability.
• Lee Altenberg. 2004. Open Problems in the Spectral Analysis of Evolutionary Dynamics. Frontiers of Evolutionary Computation. _. (Online)
• L Giraitis, D Surgailis. 1999. Central limit theorem for the empirical process of a linear sequence with long memory. Journal of statistical planning and inference.
• L Horváth. 2001. Change-point detection in long-memory processes. Journal of Multivariate Analysis.
• Marcel Ausloos, Janusz Miskiewicz. 2009. Introducing the q-Theil index. Braz. J. Phys. 39 .
• Mark Fleischer. Transformations for Accelerating Mcmc Simulations With Broken Ergodicity.
• Nobusumi Sagara. 2005. Nonparametric maximum-likelihood estimation of probability measures: existence and consistency. Journal of Statistical Planning and Inference. _. (Online)
• O Rose. 1996. Estimation of the hurst parameter of long-range dependent time series. Research Report.
• Oscar J Mesa, German Poveda. 1993. The Hurst effect: the scale of fluctuation approach. Wat. Resour. Res.
• Patrick Alfred Pierce Moran. 1964. On the range of cumulative sums. Annals of the Institute of Statistical Mathematics. _. (Online)
• P Doukhan, G Oppenheim, M S Taqqu. 2003. Theory and applications of long-range dependence.
• P M Robinson. 2003. Time series with long memory.
• Rainer Dahlhaus, Wolfgang Polonik. 2009. Empirical spectral processes for locally stationary time series. Bernoulli. _. (Online)
• R B Davies, D S Harte. 1987. Tests for Hurst effect. Biometrika.
• R Dahlhaus. 1996. On the Kullback-Leibler information divergence of locally stationary processes. Stochastic Processes and their Applications. _. (Online)
• R G Palmer. 1982. Broken ergodicity. Advances in Physics. _. (Online)
• R M Gray. 2009. Probability, random processes, and ergodic properties.
• R N Bhattacharya, V K Gupta, E Waymire. 1983. The Hurst effect under trends. Journal of applied probability.
• S Csörgö, J Mielniczuk. 1999. Random-design regression under long-range dependent errors. Bernoulli. _.
• Sílvia Gonçalves, Halbert White. 2004. Maximum likelihood and the bootstrap for nonlinear dynamic models. Journal of Econometrics. _. (Online)
• S N Lahiri. 2003. Resampling methods for dependent data.
• Stefan Gheorghiu, Marc-Olivier Coppens. 2004. Heterogeneity explains features of “anomalous” thermodynamics and statistics. Proc Natl Acad Sci U S A. _. (Online)
• T Gneiting. 2000. Power-law correlations, related models for long-range dependence and their simulation. Journal of Applied Probability.
• Vincent Q Vu, Bin Yu, Robert E Kass. 2009. Information in the nonstationary case. Neural Comput. _.
• Walter Nadler, Ulrich H E Hansmann. 2007. Generalized ensemble and tempering simulations: A unified view. Phys. Rev. E. _.
• William Feller. 1951. The Asymptotic Distribution of the Range of Sums of Independent Random Variables. The Annals of Mathematical Statistics. (Online)
• The traditional invocation of Cosma Shalizi