Tags: Visual edit apiedit |
mNo edit summary Tag: Visual edit |
||
(32 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | This page contains resources about [https://en.wikipedia.org/wiki/Bayes_estimator |
+ | This page contains resources about '''Bayesian Parameter Estimation''', '''Bayesian Parameter Learning''' and [https://en.wikipedia.org/wiki/Bayes_estimator Bayes Estimator]. |
== Subfields and Concepts == |
== Subfields and Concepts == |
||
+ | * For complete (fully observed data): |
||
+ | ** Dirichlet distribution (or other priors) |
||
+ | * For incomplete (hidden/missing data): |
||
⚫ | |||
+ | ** Viterbi Algorithm |
||
+ | ** [[Variational Method|Variational Bayes]] |
||
+ | *** Stochastic Gradient Variational Bayes (SGVB) Estimator |
||
+ | * Bayesian Hierarchical Modelling / Hierarchical Bayes Model |
||
+ | ** Hyperparameter |
||
+ | ** Hyperprior |
||
* Bayesian Decision Theory |
* Bayesian Decision Theory |
||
* Bayesian Point Estimation |
* Bayesian Point Estimation |
||
+ | * Bayesian [[Statistical Signal Processing|Signal Processing]] |
||
* Bayes Risk |
* Bayes Risk |
||
+ | * Bayesian Score |
||
+ | ** Posterior variance (when MSE is used) |
||
+ | * Bayes Risk Function / Posterior Expected Loss (i.e. Posterior Expectation Value of Loss Function) |
||
+ | ** Posterior mean / Minimum MSE (MMSE) estimator / Bayes least squared error (BLSE) estimator / Squared error loss |
||
+ | ** Posterior median / Median-unbiased estimator / Absolute error loss |
||
+ | ** Posterior mode |
||
+ | * Bayes estimator |
||
+ | ** MMSE / BLSE estimator |
||
+ | ** Median-unbiased estimator |
||
+ | ** Bayes estimator for conjugate priors (eg. exponential family) |
||
* Maximum Likelihood Estimation (MLE) |
* Maximum Likelihood Estimation (MLE) |
||
* Asymptotics of Maximum Likelihood |
* Asymptotics of Maximum Likelihood |
||
− | * Cramer-Rao bound |
+ | * Cramer-Rao bound / Cramer-Rao lower bound |
* Fisher information |
* Fisher information |
||
⚫ | |||
− | * Conjugate priors |
||
⚫ | |||
** Jeffreys prior |
** Jeffreys prior |
||
+ | * Maximum Entropy (Maxent) Models / Entropic priors |
||
* Exponential family |
* Exponential family |
||
* Beta distribution |
* Beta distribution |
||
+ | * Bayesian Online Parameter Estimation |
||
+ | * Recursive Bayesian Estimation / Bayes filter |
||
+ | ** [[Kalman filter]] (special case of Bayes filter) |
||
+ | ** Wiener filter (special case of Kalman filter) |
||
+ | * Bayesian Density Estimation |
||
+ | * Nonparametric Empirical Bayes (NPEB) |
||
+ | * Parametric Empirical Bayes Point Estimation |
||
+ | * Iterative proportional fitting (IPF) |
||
* Nonparametric Methods |
* Nonparametric Methods |
||
** Kernel Density Estimation (KDE) |
** Kernel Density Estimation (KDE) |
||
** k-Nearest Neighbours |
** k-Nearest Neighbours |
||
** [[Bayesian Nonparametrics]] |
** [[Bayesian Nonparametrics]] |
||
+ | |||
== Books and Book Chapters == |
== Books and Book Chapters == |
||
+ | * Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" ''Machine Learning: A Bayesian and Optimization Perspective''. Academic Press. |
||
* Aster, R. C., Borchers, B., & Thurber, C. (2012). "Chapter 11: Bayesian Methods". Parameter estimation and inverse problems. Academic Press |
* Aster, R. C., Borchers, B., & Thurber, C. (2012). "Chapter 11: Bayesian Methods". Parameter estimation and inverse problems. Academic Press |
||
* Barber, D. (2012). "Section 9.1: Learning as Inference". ''Bayesian Reasoning and Machine Learning''. Cambridge University Press. |
* Barber, D. (2012). "Section 9.1: Learning as Inference". ''Bayesian Reasoning and Machine Learning''. Cambridge University Press. |
||
+ | * Barber, D. (2012). "Chapter 18: Bayesian Linear Models". ''Bayesian Reasoning and Machine Learning''. Cambridge University Press. |
||
− | |||
* Duda, R. O., Hart, P. E., & Stork, D. G. (2012). "Chapter 2: Bayesian Decision Theory". ''Pattern Classification''. John Wiley & Sons. |
* Duda, R. O., Hart, P. E., & Stork, D. G. (2012). "Chapter 2: Bayesian Decision Theory". ''Pattern Classification''. John Wiley & Sons. |
||
+ | * Murphy, K. P. (2012). "Section 5.7: Bayesian Decision Theory". ''Machine Learning: A Probabilistic Perspective''. MIT Press. |
||
− | |||
+ | * Koller, D., & Friedman, N. (2009). "Chapter 17: Parameter Estimation". ''Probabilistic Graphical Models''. MIT Press. |
||
+ | * Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 2: Bayesian Decision Theory". ''Pattern Recognition''. 4th Ed. Academic Press. |
||
* Bishop, C. M. (2006). "Chapter 2: Probability Distributions". ''Pattern Recognition and Machine Learning''. Springer. |
* Bishop, C. M. (2006). "Chapter 2: Probability Distributions". ''Pattern Recognition and Machine Learning''. Springer. |
||
− | |||
* MacKay, D. J. (2003). "Chapter 24: Exact Marginalization". ''Information Theory, Inference and Learning Algorithms''. Cambridge University Press. |
* MacKay, D. J. (2003). "Chapter 24: Exact Marginalization". ''Information Theory, Inference and Learning Algorithms''. Cambridge University Press. |
||
− | |||
* MacKay, D. J. (2003). "Chapter 36: Decision Theory". ''Information Theory, Inference and Learning Algorithms''. Cambridge University Press. |
* MacKay, D. J. (2003). "Chapter 36: Decision Theory". ''Information Theory, Inference and Learning Algorithms''. Cambridge University Press. |
||
− | * Bretthorst, G. L. (1998). ''Bayesian spectrum analysis and parameter estimation'' |
+ | * Bretthorst, G. L. (1998). ''Bayesian spectrum analysis and parameter estimation''. Springer Science & Business Media. |
⚫ | |||
− | |||
⚫ | |||
== Scholarly Articles == |
== Scholarly Articles == |
||
+ | * Caticha, A. (2010). Entropic inference. ''arXiv preprint arXiv:1011.0723''. |
||
− | * |
||
+ | * Caticha, A., & Preuss, R. (2004). Maximum entropy and Bayesian data analysis: Entropic prior distributions. ''Physical Review E'', ''70''(4), 046127. |
||
+ | * Ghahramani, Z. (2003). "Graphical models: Parameter learning". In ''Handbook of Brain Theory and Neural Networks''. |
||
+ | * Malouf, R. (2002). A comparison of algorithms for maximum entropy parameter estimation. In ''proceedings of the 6th conference on Natural language learning-Volume 20'' (pp. 1-7). Association for Computational Linguistics. |
||
== See also == |
== See also == |
||
* [[Bayesian Model Selection]] |
* [[Bayesian Model Selection]] |
||
+ | * [[Statistical Signal Processing|Estimation Theory]] |
||
⚫ | |||
== Other resources == |
== Other resources == |
||
* [https://metacademy.org/graphs/concepts/bayesian_parameter_estimation Bayesian parameter estimation] - Metacademy |
* [https://metacademy.org/graphs/concepts/bayesian_parameter_estimation Bayesian parameter estimation] - Metacademy |
||
+ | * [http://homepages.inf.ed.ac.uk/lzhang10/maxent.html Maximum Entropy Modelling] |
||
+ | |||
+ | [[Category:Bayesian Machine Learning]] |
Latest revision as of 14:58, 24 October 2017
This page contains resources about Bayesian Parameter Estimation, Bayesian Parameter Learning and Bayes Estimator.
Subfields and Concepts[]
- For complete (fully observed data):
- Dirichlet distribution (or other priors)
- For incomplete (hidden/missing data):
- Markov Chain Monte Carlo (MCMC)
- Viterbi Algorithm
- Variational Bayes
- Stochastic Gradient Variational Bayes (SGVB) Estimator
- Bayesian Hierarchical Modelling / Hierarchical Bayes Model
- Hyperparameter
- Hyperprior
- Bayesian Decision Theory
- Bayesian Point Estimation
- Bayesian Signal Processing
- Bayes Risk
- Bayesian Score
- Posterior variance (when MSE is used)
- Bayes Risk Function / Posterior Expected Loss (i.e. Posterior Expectation Value of Loss Function)
- Posterior mean / Minimum MSE (MMSE) estimator / Bayes least squared error (BLSE) estimator / Squared error loss
- Posterior median / Median-unbiased estimator / Absolute error loss
- Posterior mode
- Bayes estimator
- MMSE / BLSE estimator
- Median-unbiased estimator
- Bayes estimator for conjugate priors (eg. exponential family)
- Maximum Likelihood Estimation (MLE)
- Asymptotics of Maximum Likelihood
- Cramer-Rao bound / Cramer-Rao lower bound
- Fisher information
- Uninformative priors / Noninformative priors / Maximum entropy priors
- Jeffreys prior
- Maximum Entropy (Maxent) Models / Entropic priors
- Exponential family
- Beta distribution
- Bayesian Online Parameter Estimation
- Recursive Bayesian Estimation / Bayes filter
- Kalman filter (special case of Bayes filter)
- Wiener filter (special case of Kalman filter)
- Bayesian Density Estimation
- Nonparametric Empirical Bayes (NPEB)
- Parametric Empirical Bayes Point Estimation
- Iterative proportional fitting (IPF)
- Nonparametric Methods
- Kernel Density Estimation (KDE)
- k-Nearest Neighbours
- Bayesian Nonparametrics
Books and Book Chapters[]
- Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
- Aster, R. C., Borchers, B., & Thurber, C. (2012). "Chapter 11: Bayesian Methods". Parameter estimation and inverse problems. Academic Press
- Barber, D. (2012). "Section 9.1: Learning as Inference". Bayesian Reasoning and Machine Learning. Cambridge University Press.
- Barber, D. (2012). "Chapter 18: Bayesian Linear Models". Bayesian Reasoning and Machine Learning. Cambridge University Press.
- Duda, R. O., Hart, P. E., & Stork, D. G. (2012). "Chapter 2: Bayesian Decision Theory". Pattern Classification. John Wiley & Sons.
- Murphy, K. P. (2012). "Section 5.7: Bayesian Decision Theory". Machine Learning: A Probabilistic Perspective. MIT Press.
- Koller, D., & Friedman, N. (2009). "Chapter 17: Parameter Estimation". Probabilistic Graphical Models. MIT Press.
- Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 2: Bayesian Decision Theory". Pattern Recognition. 4th Ed. Academic Press.
- Bishop, C. M. (2006). "Chapter 2: Probability Distributions". Pattern Recognition and Machine Learning. Springer.
- MacKay, D. J. (2003). "Chapter 24: Exact Marginalization". Information Theory, Inference and Learning Algorithms. Cambridge University Press.
- MacKay, D. J. (2003). "Chapter 36: Decision Theory". Information Theory, Inference and Learning Algorithms. Cambridge University Press.
- Bretthorst, G. L. (1998). Bayesian spectrum analysis and parameter estimation. Springer Science & Business Media.
- Berger, J. O. (1993). Statistical decision theory and Bayesian analysis. 2nd Ed. Springer Science & Business Media.
Scholarly Articles[]
- Caticha, A. (2010). Entropic inference. arXiv preprint arXiv:1011.0723.
- Caticha, A., & Preuss, R. (2004). Maximum entropy and Bayesian data analysis: Entropic prior distributions. Physical Review E, 70(4), 046127.
- Ghahramani, Z. (2003). "Graphical models: Parameter learning". In Handbook of Brain Theory and Neural Networks.
- Malouf, R. (2002). A comparison of algorithms for maximum entropy parameter estimation. In proceedings of the 6th conference on Natural language learning-Volume 20 (pp. 1-7). Association for Computational Linguistics.
See also[]
Other resources[]
- Bayesian parameter estimation - Metacademy
- Maximum Entropy Modelling