Linear Dynamical System

This page contains resources about Linear Dynamical Systems, Linear Systems Theory, Dynamic Linear Models, Linear State Space Models and State-Space Representation, including temporal (Time Series) and atemporal Sequential Data.

Subfields and Concepts

 * Linear SSM
 * Discrete-time LDS
 * Continuous-time LDS
 * Linear Time-Invariant (LTI) system
 * Linear Time-Variant System
 * Parametric models / Time Series models
 * Autoregressive (AR) model / All-Pole model
 * Moving Average (MA) model / All-Zero model
 * ARMA model / Pole-Zero model
 * Autoregressive Conditional Heteroskedasticity (ARCH) model
 * Generalized ARCH (GARCH) model
 * Vector Autoregressive (VAR) model
 * Martin Distance (for comparing ARMA processes)
 * Kalman filter / Linear Gaussian SSM
 * Stochastic LDS
 * Structured LDS
 * Bayesian SSM
 * Bayesian Time Series
 * Bayesian LDS
 * SSM with Regime Switching / Jump Markov Linear Systems / Switching LDS / Switching SSM
 * Kernels on Dynamical Systems
 * Computer Vision
 * Linear Dynamic Texture
 * Kernel Dynamic Texture

Video Lectures

 * Introduction to Linear Dynamical Systems by Stephen Boyd
 * Topics in Mathematics with Applications in Finance by Peter Kempthorne, Choongbum Lee, Vasily Strela and Jake Xia

Lecture Notes

 * Dynamic Systems and Control by Emilio Frazzoli & Munther Dahleh
 * Linear Systems Theory by John Lygeros and Federico A. Ramponi
 * Linear System Theory by Claire Tomlin
 * Time Series Econometrics by Peter C. B. Phillips
 * Time Series Econometrics by Eric Zivot
 * Econometrics II by Rauli Susmel
 * Applied Econometrics by Baum
 * Dynamical Systems and Stochastic Processes by Pierre Collet
 * Linear Dynamical Systems by Stephen Boyd
 * Applied Time Series Analysis
 * Time Series Analysis I by Suhasini Subba Rao
 * Applied Forecasting for Business and Economics by Rob J Hyndman
 * Lecture 10: Sequential Data Models by Geoffrey Hinton

Books and Book Chapters
See also Further Reading.
 * Brockett, R. W. (2015). Finite dimensional linear systems. SIAM.
 * Hyndman, R. J., & Athanasopoulos, G. (2013). Forecasting: principles and practice. OTexts.
 * Murphy, K. P. (2012). "Chapter 18: State space models". Machine Learning: A Probabilistic Perspective. MIT Press.
 * Barber, D. (2012). "Chapter 24: Continuous-State Markov Models". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Barber, D. (2012). "Chapter 25: Switching Linear Dynamical Systems". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Durbin, J., & Koopman, S. J. (2012). Time series analysis by state space methods. Oxford University Press.
 * Casti, J. L. (2012). Linear dynamical systems. Academic Press Professional.
 * Prado, R., & West, M. (2010). Time series: modeling, computation, and inference. CRC Press.
 * Tsay, R. S. (2010). Analysis of Financial Time Series. 3rd Ed. John Wiley & Sons.
 * Petris, G., Petrone, S., & Campagnoli, P. (2009). Dynamic Linear Models with R. Springer New York.
 * Hespanha, J. P. (2009). Linear systems theory. Princeton university press.
 * Zadeh, L. A., & Desoer, C. A. (2008). Linear System Theory: The State Space Approach. Dover.
 * Antsaklis, P. J., & Michel, A. N. (2007). A Linear Systems Primer. Springer Science & Business Media.
 * Antsaklis, P. J., & Michel, A. N. (2006). Linear systems. Springer Science & Business Media.
 * Bishop, C. M. (2006). "Chapter 13: Sequential Data". Pattern Recognition and Machine Learning. Springer.
 * Gajic, Z. (2003). Linear dynamic systems and signals. Prentice Hall/Pearson Education.
 * Chatfield, C. (2003). The analysis of time series: an introduction. 6th Ed. CRC press.
 * Harrison, J., & West, M. (1999). Bayesian Forecasting & Dynamic Models. Springer.
 * Chen, C. T. (1998). Linear system theory and design. Oxford University Press.
 * Rugh, W. J. (1996). Linear system theory. Prentice Hall.
 * Hamilton, J. D. (1994). Time series analysis. Princeton University Press.
 * Callier, F. M., & Desoer, C. A. (1991). Linear System Theory. Springer New York.
 * Harvey, A. C. (1990). Forecasting, structural time series models and the Kalman filter. Cambridge university press.
 * Harvey, A. C. (1993). Time series models. 2nd Ed. The MIT Press.
 * Delchamps, D. F. (1988). State space and input-output linear systems. Springer Science & Business Media.
 * Cryer, J. D. (1986). Time series analysis. Duxbury Press.
 * Kailath, T. (1980). Linear systems. Prentice-Hall.
 * Luenberger, D. G. (1979). Introduction to dynamic systems. John Wiley & Sons.

Scholarly Articles

 * Archer, E., Park, I. M., Buesing, L., Cunningham, J., & Paninski, L. (2015). Black box variational inference for state space models. arXiv preprint arXiv:1511.07367.
 * Petris, G., & Petrone, S. (2011). State space models in R. Journal of Statistical Software, 41(4), 1-25.
 * Vishwanathan, S. V. N., Smola, A. J., & Vidal, R. (2007). Binet-Cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. International Journal of Computer Vision, 73(1), 95-119.
 * Chan, A. B., & Vasconcelos, N. (2007). Classifying video with kernel dynamic textures. In Computer Vision and Pattern Recognition, IEEE Conference on (pp. 1-6). IEEE.
 * Rudary, M., Singh, S., & Wingate, D. (2005). Predictive linear-Gaussian models of stochastic dynamical systems. Conference on Uncertainty in Artificial Intelligence.
 * Doretto, G., Chiuso, A., Wu, Y. N., & Soatto, S. (2003). Dynamic textures. International Journal of Computer Vision, 51(2), 91-109.
 * Martin, R. J. (2000). A metric for ARMA processes. IEEE transactions on Signal Processing, 48(4), 1164-1170.
 * Minka, T. (1999). From hidden markov models to linear dynamical systems. Technical Report, MIT.
 * Kim, C. J. (1994). Dynamic linear models with Markov-switching. Journal of Econometrics, 60(1-2), 1-22.
 * Ghahramani, Z., & Hinton, G. E. (1996). Parameter estimation for linear dynamical systems. Technical Report CRG-TR-96-2, University of Toronto, Dept. of Computer Science.
 * Kalman, R. E. (1963). Mathematical description of linear dynamical systems. Journal of the Society for Industrial and Applied Mathematics, Series A: Control, 1(2), 152-192.

Software

 * Control Systems Toolbox - MATLAB
 * System Identification Toolbox - MATLAB
 * Econometrics Toolbox - MATLAB
 * Kalman filter and Linear Dynamical System -MATLAB
 * DLM Matlab Toolbox
 * Python Control Systems Toolbox
 * dynpy - Python
 * PyDSTool - Python
 * Statsmodels - Statistical Modeling and Econometrics in Python
 * PyFlux - Python
 * DLM - R
 * dynr - R
 * timeSeries - R
 * zoo - R

Other Resources

 * State Space Models in Python
 * Time Series Analysis in R
 * Time Series - Notebook
 * State-Space Reconstruction  - Notebook
 * Time Series Analysis (TSA) in Python - Linear Models to GARCH
 * Time Series and Sequential Data - Zoubin Ghahramani
 * A comprehensive beginner’s guide to create a Time Series Forecast (with Codes in Python) - Blog post
 * Simple Time Series Forecasting Models to Test So That You Don’t Fool Yourself - Blog post
 * How To Backtest Machine Learning Models for Time Series Forecasting - Blog post
 * Cross-validation for time series - Blog post
 * Time Series Analysis - Blog post
 * How to Make Baseline Predictions for Time Series Forecasting with Python - blog post
 * 7 Ways Time-Series Forecasting Differs from Machine Learning - blog post
 * Making Predictions with Sequences - blog post
 * 11 Classical Time Series Forecasting Methods in Python (Cheat Sheet) - blog post