1.1 Estimators

Estimators

An estimator is a function that maps observed data to an estimate of an unknown quantity of the underlying data‑generating process, called a parameter. Formally, given an observation sample $X=(X_1,\dots ,X_n)$, an estimator for a parameter $\theta$ is a statistic ${\hat{\Theta }}_n=g(X)$ for some function $g$ that produces a single “best guess” of $\theta$ based on the observations.

In Bayesian inference,

In classical inference, the estimator is perceived as a statistic that “plugs in” the value of the parameter maximizing the likelihood. Given that the estimation is based on a brittle model, it may not always yield the most suitable estimate.

The misalignment between the parameter $\theta$ and the estimator ${\hat{\Theta }}_n$ is known as the estimation error, and is given as
${\tilde{\Theta }}_n={\hat{\Theta }}_n-\theta .$

The bias of the estimator is the expected value of the estimation error, and is denoted as
$b_{\theta }({\hat{\Theta }}_n)={\mathbb{E}}_{\theta }[{\hat{\Theta }}_n]-\theta .$
The estimator ${\hat{\Theta }}_n$ is unbiased if ${\mathbb{E}}_{\theta }[{\hat{\Theta }}_n]=\theta$, and asymptotically unbiased if ${\lim }_{n\to \infty }{\mathbb{E}}_{\theta }[{\hat{\Theta }}_n]=\theta$. Furthermore, ${\hat{\Theta }}_n$ is consistent if it converges to the true value of $\theta$, in probability, for every possible value of $\theta$.