This page contains resources about Linear Dynamical Systems, Linear Systems Theory, Dynamic Linear Models, Linear State Space Models and State-Space Representation, including temporal (Time Series) and atemporal Sequential Data.
Subfields and Concepts[]
- Linear SSM
- Discrete-time LDS
- Continuous-time LDS
- Linear Time-Invariant (LTI) system
- Linear Time-Variant System
- Parametric models / Time Series models
- Autoregressive (AR) model / All-Pole model
- Moving Average (MA) model / All-Zero model
- ARMA model / Pole-Zero model
- Autoregressive Conditional Heteroskedasticity (ARCH) model
- Generalized ARCH (GARCH) model
- Vector Autoregressive (VAR) model
- Martin Distance (for comparing ARMA processes)
- Kalman filter / Linear Gaussian SSM
- Stochastic LDS
- Structured LDS
- Bayesian SSM
- Bayesian Time Series
- Bayesian LDS
- SSM with Regime Switching / Jump Markov Linear Systems / Switching LDS / Switching SSM
- Kernels on Dynamical Systems
- Computer Vision
- Linear Dynamic Texture
- Kernel Dynamic Texture
Online Courses[]
Video Lectures[]
Introduction to Linear Dynamical Systems by Stephen Boyd- Topics in Mathematics with Applications in Finance by Peter Kempthorne, Choongbum Lee, Vasily Strela and Jake Xia
Lecture Notes[]
- Dynamic Systems and Control by Emilio Frazzoli & Munther Dahleh
- Linear Systems Theory by John Lygeros and Federico A. Ramponi
- Linear System Theory by Claire Tomlin
- Time Series Econometrics by Peter C. B. Phillips
- Time Series Econometrics by Eric Zivot
- Econometrics II by Rauli Susmel
- Applied Econometrics by Baum
- Dynamical Systems and Stochastic Processes by Pierre Collet
- Linear Dynamical Systems by Stephen Boyd
- Applied Time Series Analysis
- Time Series Analysis I by Suhasini Subba Rao
- Applied Forecasting for Business and Economics by Rob J Hyndman
- Lecture 10: Sequential Data Models by Geoffrey Hinton
Books and Book Chapters[]
See also Further Reading.
- Brockett, R. W. (2015). Finite dimensional linear systems. SIAM.
- Hyndman, R. J., & Athanasopoulos, G. (2013). Forecasting: principles and practice. OTexts.
- Murphy, K. P. (2012). "Chapter 18: State space models". Machine Learning: A Probabilistic Perspective. MIT Press.
- Barber, D. (2012). "Chapter 24: Continuous-State Markov Models". Bayesian Reasoning and Machine Learning. Cambridge University Press.
- Barber, D. (2012). "Chapter 25: Switching Linear Dynamical Systems". Bayesian Reasoning and Machine Learning. Cambridge University Press.
- Durbin, J., & Koopman, S. J. (2012). Time series analysis by state space methods. Oxford University Press.
- Casti, J. L. (2012). Linear dynamical systems. Academic Press Professional.
- Prado, R., & West, M. (2010). Time series: modeling, computation, and inference. CRC Press.
- Tsay, R. S. (2010). Analysis of Financial Time Series. 3rd Ed. John Wiley & Sons.
- Petris, G., Petrone, S., & Campagnoli, P. (2009). Dynamic Linear Models with R. Springer New York.
- Hespanha, J. P. (2009). Linear systems theory. Princeton university press.
- Zadeh, L. A., & Desoer, C. A. (2008). Linear System Theory: The State Space Approach. Dover.
- Antsaklis, P. J., & Michel, A. N. (2007). A Linear Systems Primer. Springer Science & Business Media.
- Antsaklis, P. J., & Michel, A. N. (2006). Linear systems. Springer Science & Business Media.
- Bishop, C. M. (2006). "Chapter 13: Sequential Data". Pattern Recognition and Machine Learning. Springer.
- Gajic, Z. (2003). Linear dynamic systems and signals. Prentice Hall/Pearson Education.
- Chatfield, C. (2003). The analysis of time series: an introduction. 6th Ed. CRC press.
- Harrison, J., & West, M. (1999). Bayesian Forecasting & Dynamic Models. Springer.
- Chen, C. T. (1998). Linear system theory and design. Oxford University Press.
- Rugh, W. J. (1996). Linear system theory. Prentice Hall.
- Hamilton, J. D. (1994). Time series analysis. Princeton University Press.
- Callier, F. M., & Desoer, C. A. (1991). Linear System Theory. Springer New York.
- Harvey, A. C. (1990). Forecasting, structural time series models and the Kalman filter. Cambridge university press.
- Harvey, A. C. (1993). Time series models. 2nd Ed. The MIT Press.
- Delchamps, D. F. (1988). State space and input-output linear systems. Springer Science & Business Media.
- Cryer, J. D. (1986). Time series analysis. Duxbury Press.
- Kailath, T. (1980). Linear systems. Prentice-Hall.
- Luenberger, D. G. (1979). Introduction to dynamic systems. John Wiley & Sons.
Scholarly Articles[]
- Archer, E., Park, I. M., Buesing, L., Cunningham, J., & Paninski, L. (2015). Black box variational inference for state space models. arXiv preprint arXiv:1511.07367.
- Petris, G., & Petrone, S. (2011). State space models in R. Journal of Statistical Software, 41(4), 1-25.
- Vishwanathan, S. V. N., Smola, A. J., & Vidal, R. (2007). Binet-Cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. International Journal of Computer Vision, 73(1), 95-119.
- Chan, A. B., & Vasconcelos, N. (2007). Classifying video with kernel dynamic textures. In Computer Vision and Pattern Recognition, IEEE Conference on (pp. 1-6). IEEE.
- Rudary, M., Singh, S., & Wingate, D. (2005). Predictive linear-Gaussian models of stochastic dynamical systems. Conference on Uncertainty in Artificial Intelligence.
- Doretto, G., Chiuso, A., Wu, Y. N., & Soatto, S. (2003). Dynamic textures. International Journal of Computer Vision, 51(2), 91-109.
- Martin, R. J. (2000). A metric for ARMA processes. IEEE transactions on Signal Processing, 48(4), 1164-1170.
- Minka, T. (1999). From hidden markov models to linear dynamical systems. Technical Report, MIT.
- Kim, C. J. (1994). Dynamic linear models with Markov-switching. Journal of Econometrics, 60(1-2), 1-22.
- Ghahramani, Z., & Hinton, G. E. (1996). Parameter estimation for linear dynamical systems. Technical Report CRG-TR-96-2, University of Toronto, Dept. of Computer Science.
- Kalman, R. E. (1963). Mathematical description of linear dynamical systems. Journal of the Society for Industrial and Applied Mathematics, Series A: Control, 1(2), 152-192.
Software[]
- Control Systems Toolbox - MATLAB
- System Identification Toolbox - MATLAB
- Econometrics Toolbox - MATLAB
- Kalman filter and Linear Dynamical System -MATLAB
- DLM Matlab Toolbox
- Python Control Systems Toolbox
- dynpy - Python
- PyDSTool - Python
- Statsmodels - Statistical Modeling and Econometrics in Python
- PyFlux - Python
- DLM - R
- dynr - R
- timeSeries - R
- zoo - R
See also[]
- Stochastic Process
- Monte Carlo Methods
- Variational Methods
- Hidden Markov Models
- Estimation Theory / Statistical Signal Processing
- Nonlinear Systems
- Computational Finance
Other Resources[]
- State Space Models in Python
- Time Series Analysis in R
- Time Series - Notebook
- State-Space Reconstruction - Notebook
- Time Series Analysis (TSA) in Python - Linear Models to GARCH
- Time Series and Sequential Data - Zoubin Ghahramani
- A comprehensive beginner’s guide to create a Time Series Forecast (with Codes in Python) - Blog post
- Simple Time Series Forecasting Models to Test So That You Don’t Fool Yourself - Blog post
- How To Backtest Machine Learning Models for Time Series Forecasting - Blog post
- Cross-validation for time series - Blog post
- Time Series Analysis - Blog post
- How to Make Baseline Predictions for Time Series Forecasting with Python - blog post
- 7 Ways Time-Series Forecasting Differs from Machine Learning - blog post
- Making Predictions with Sequences - blog post
- 11 Classical Time Series Forecasting Methods in Python (Cheat Sheet) - blog post