Bayesian regression is the application of Bayes’ rule to models where the mean (or the log-mean, or the logit-mean) of the response is a linear function of covariates. Instead of returning a single coefficient vector $\hat{\mathbf{\beta }}$, Bayesian regression returns a full posterior distribution over $\mathbf{\beta }$ that encodes parameter uncertainty, propagates naturally into predictive distributions, and admits hierarchical extensions (mixed-effects, multilevel regression). Standard frequentist estimators reappear as posterior modes under specific prior choices: ridge regression is the posterior mode under a Gaussian prior on $\mathbf{\beta }$, and the lasso is the posterior mode under a Laplace prior.

The folder covers four regression likelihoods chosen for how they illustrate different aspects of Bayesian inference. Linear regression shows the clean conjugate case with a closed-form multivariate-$t$ posterior. AutoRegression demonstrates the likelihood principle in action: frequentist analysis of the same data requires different stopping-rule caveats, whereas Bayesian inference is agnostic. Logistic and Poisson regression are the canonical non-conjugate models, where the posterior is intractable and the 3.9 Laplace Approximation provides the standard tool.

Prerequisites

• 3.1 Bayesian Framework — prior, likelihood, posterior.
• 3.5.2 Normal-Normal Model — univariate building block that generalizes to the linear model.
• 3.9 Laplace Approximation — default tool for non-conjugate regression.

Contents

• 3.8.1 Bayesian Linear Regression — conjugate normal model with multivariate-$t$ posterior for the coefficients.
• 3.8.2 Bayesian AutoRegression — AR(1) time-series model as a canonical likelihood-principle illustration.
• 3.8.3 Bayesian Logistic Regression — binary response; Laplace approximation via iteratively reweighted least squares.
• 3.8.4 Bayesian Poisson Regression — count response; Laplace-based MCMC proposal.

Shared Structure

Every note in this folder writes the response $y_i$ as depending on covariates ${\mathbf{x}}_i\in {\mathbb{R}}^d$ through a coefficient vector $\mathbf{\beta }\in {\mathbb{R}}^d$ and a link function $g$:
$\mathbb{E}[y_i\mid \mathbf{\beta },{\mathbf{x}}_i]=g^{-1}({\mathbf{x}}_i^{\top }\mathbf{\beta }).$
The three regression families in this folder differ only in the response distribution and link: Gaussian with identity link (linear regression), Bernoulli with logit link (logistic regression), Poisson with log link (Poisson regression). AR(1) is a linear-regression special case with lagged outputs as covariates, raised to its own note because the stopping-rule question it exposes is pedagogically important.

The Bayesian machinery is identical across all four: place a prior $p(\mathbf{\beta })$ (typically Gaussian for ridge-like shrinkage, Laplace for lasso-like sparsity), combine with the likelihood to form the posterior, and summarize by posterior means, credible intervals, or posterior predictive distributions. Only linear regression with Gaussian prior admits a closed-form posterior; the others require 3.9 Laplace Approximation, Metropolis-Hastings, or Gibbs sampling with data augmentation.

When to Use Which

Read these three signals before choosing a regression family:

1. Response type. Continuous and unbounded → linear. Binary or bounded → logistic. Count → Poisson (or Negative-Binomial if overdispersion). Bounded-continuous → Beta regression (not covered here). Lagged continuous → AutoRegression.
2. Prior choice. Shrinkage toward zero with continuous mass → Gaussian prior (ridge analog); sparsity with point mass at zero → spike-and-slab; sparsity with continuous relaxation → Laplace prior (lasso analog) or Horseshoe prior.
3. Inference budget. Closed-form posterior available → linear regression with Normal-Inverse-Gamma prior. Laplace approximation sufficient → logistic or Poisson with moderate $n$ and $d$. Full MCMC → hierarchical or heavy-tailed priors, or when uncertainty in high-leverage tails matters.

See Also

• 3. Bayesian Statistics — parent index.
• 3.6 Hierarchical Models — multilevel regression extends all four families with group-level coefficients.
• 3.11 MCMC — the inference engine when conjugacy fails.
• 1.2. Linear Regression — frequentist counterpart.