This serves as a repository containing my Bayesian Inference learnings. Please do not get overwhelmed by the names of several packages written in the title. During my learning curve, due to the resources I used, I moved from brms through pymc3 to pyro. Initially, when I started out, I built models in brms (An R package). It was quite to easy to build and improve the model. But, eventually when it got harder to assess the outputs, underlying theory and a need for bayesian python models, I switched to Pymc3, which has a strong open source community around it. Later, once I picked up the theory and started building models in pymc3, I switched myself to pyro and trying my best to get expertise in it. You can read more about bayesian inference and the organisation of my resources below.

Why I made this repository?

Uncertainity estimation in machine learning, deep learning and reinforcement learning methods are increasingly becoming popular due to the latest research advancements in variational inference, monte carlo dropout, bayes by backpropagation and its application in convolutional and recurrent neural networks.

This repository will contain book chapter reviews, bayesian model implementations and resources for learning bayesian modeling. I did a rigorous research on this topic to come up with a list of most influential books and programming packages on this topic to layout a plan for my study. The list of books and packages are listed under Books.

There will be several folders and subfolders in this repository. Currently, this is a compilation of several knowledge that I learned from different resoures. If you would like to follow along, I would suggest you to start with Think Bayes book and then stick to Statistical Rethinking book. Further, you can access exercises from "notebooks folder". For book chapters review, I have really enjoyed reading from different authors since it helped me to understand and reason about the same concept multiple time. Currently, I am sticking to statistical rethinking book, but I do access other books I mentioned to understand a concept better.

• Foundational Exercises [Jupyter notebooks]
• Toy problems and real-world problems [Toy scripts]
• A summary of the Book chapters can be accessed from subfolders [theory folder]
• A summary of my daily notes [theory folder]

The main advantages of going bayesian are as follows:

• Bayesian methods typically involves using probability distributions rather than point probabilities such as mean. For example for a regression problem for predicting house prices, rather than predicting the price of a house, bayesian methods produces a distribution of possible predictions.
• Bayesian methods helps to derive credible intervals (similar to confidence interval but not the same) around the mean using the predicted distribution.
• Bayesian method can utilize informed or uninformed priors. Priors are nothing but prior knowledge about the distribution of samples. This is extremely useful fot getting better predictions as well as decreasing time required for traing a ML or DL model.
• Bayesian methods work efficiently even with small sample sizes for deep learning models or machine learning models.
• Bayesian methods account for variability in the measurement of the data.
• Bayesian methods are a solution to the over-fitting problem. Bayesian approach allows us to set certain priors on our features. For ex: Bayesian regression with a normal prior is same as ridge regression(L2 regularisation).

• Think Bayes
• Statistical rethinking
• Bayesian analysis with python
• Bayesian data analysis
• Information Theory, Inference, and Learning Algorithms
• Gaussian process for machine learning
• pattern recognition and machine learning
• Probability theory: The logic of science

Name	Description	Category	Level	Link
Understanding probability and posterior probability distribution	Introduction to probability, understanding joint and conditional probability, Bayes theorem and posterior distributions	Probability	Beginner
Sampling and Variational Inference	An introduction to sampling and variational methods.	Sampling and approximation methods	Advanced
Introduction to Pyro	Introduction to Pyro, a probabilistic programming package build on top of Pytorch.	Bayesian Inference	Advanced
MNIST with Pyro	MNIST with Pyro.	Neural Networks	Advanced

• 00-coin-toss-trial2.py
• 00-coin-toss-trial.py
• 01-beta-distribution.py
• 02_coin_toss_pymc3.py
• 03_normal-distribution_pymc3.py
• 04_covariance.py
• 05_trimodal-distribution.py
• 06_mixture-model-01.py
• 06_mixture-model-02.py
• 06_mixture-model-03.py
• 07_debug-01.py
• 07_debug-02.py
• 07_debug-03.py
• 07_understand-distribution-pymc3.py
• 08_developer-guide-01.py
• 08_developer-guide-02.py
• 08_developer-guide-03.ipynb
• 08_exam_ability.py
• 09_holdoutset-prediction-01.py
• 10_categorical-01.py
• 10_categorical-02.py
• 10_categorical.py
• 11_datageneration-estimation-pymc3.py
• 12_MvNormal-experiments-01.py
• 13_Changepoint_detection.ipynb
• 13_verify-datapoints-generation.py