1.4.1 Neyman-Pearson Testing

Point Testing

A point test is a hypothesis test in which the null hypothesis fully specifies the distribution by fixing the parameter to a single value.

For binary hypothesis testing, the Neyman-Pearson Test is the point test which maximizes power subject to a constraint on the significance level $\alpha$. Define the Likelihood Ratio to be
$L(x)=\frac{{\mathbb{P}}_{H_1}(x)}{{\mathbb{P}}_{H_0}(x)}\text{or}L(x)=\frac{f_{H_1}(x)}{f_{H_2}(x)}.$
Then, for some critical threshold $\lambda >0$ and an observation $x$:

1. Accept $H_0$ if $L(x)<\lambda$.
2. Reject $H_0$ with probability $\gamma \in [0,1]$ if $L(x)=\lambda$.
3. Reject $H_0$ if $L(x)>\lambda$.

This guarantees that the test yields the smallest possible $\beta$ among all tests that do not exceed $\alpha$.

Intuition

We first begin by expressing the Neyman-Pearson Test as an optimization problem, where
$\hat{X}$

Examples

Exponential Hypothesis Testing

Let $X\sim \mathrm{Exp}(1)$. Given $X$, let $Y\sim \mathrm{Exp}(X)$. Let $c>1$. Suppose that

• The null hypothesis is $X=1:Y\sim \mathrm{Exp}(1)$, and
• The alternative hypothesis is $X=c:Y\sim \mathrm{Exp}(c)$.

Find the decision rule $\hat{X}$ (a function of $Y$) that maximizes $\mathbb{P}(\hat{X}=1\mid X=1)$ subject to $\mathbb{P}(\hat{X}=1\mid X=c)\le 5\%$.

Solution. $\alpha \le 0.05$. We find that $L(y)=\frac{f_{Y\mid X}(y\mid c)}{f_{Y\mid X}(y\mid 1)}=\frac{c{e}^{-cy}}{e^{-y}}=c{e}^{-y(c-1)},$ which is decreasing in $y$ since $c>1$. Since $L(x)\le \lambda$ is our criterion for accepting the null (we use a $\le$ instead of a strict inequality since we are working with continuous variables) , we have $L(y)=c{e}^{-y(c-1)}\le \lambda ⟺y\le \frac{\ln (\lambda /c)}{1-c}=t.$ Given $\alpha \le 0.05$, we also have

We utilize the Neyman-Pearson test with

\alpha &= \mathbb{P}(\hat{X}=1 \mid X = c) \
&=\mathbb{P}(L(Y) \leq \zeta \mid X=c) \
&= \mathbb{P}(Y \leq t \mid X=c) \
&= 1-e^{-ct} \
&= 0.05 \iff t=-\frac{\ln 0.95}{c}
\end{align}$$

Therefore, we equate the two values of $t$ to solve for the threshold $\lambda$, so that
$-\frac{\ln 0.95}{c}=\frac{\ln (\lambda /c)}{1-c}⟺\lambda =c\cdot \exp \left(\frac{-\ln 0.95}{c}(1-c)\right).$
Our rule is then

\hat{X}=\begin{cases}
1 & \text{if } L(Y) \leq \lambda \equiv Y \leq t \
c & \text{if } L(Y) > \lambda \equiv Y > t.
\end{cases}
\end{align}$$

Bernoulli Hypothesis Testing

Suppose that

• The null hypothesis is $X=0:Y\sim \mathrm{Ber}\left(1/4\right)$, and
• The alternative hypothesis is $X=1:Y\sim \mathrm{Ber}\left(3/4\right)$.

Using the Neyman–Pearson formulation of hypothesis testing, find the optimal randomized decision rule $\hat{X}$ with respect to the criterion

\min &\enspace \mathbb{P}(\hat{X}=0 \mid X=1) \
\text{s.t.} &\enspace \mathbb{P}(\hat{X}=1 \mid X=0) \le \beta,
\end{align}$$

where $\beta \in [0,1]$ is a given upper bound on the false positive probability.

Solution.

This question requires us to find the test that minimizes the false negative probability while keeping the false positive probability below $\beta$. We start by computing the likelihood ratio,
$L(y)=\frac{p_{Y\mid X}(y\mid 1)}{p_{Y\mid X}(y\mid 0)},$
such that $L(0)=\frac{1/4}{3/4}=1/3$ and $L(1)=\frac{3/4}{1/4}=3$, or

L(y)=\begin{cases} 1/3 & \text{if } y=0 \\ 3 & \text{if } y=1. \end{cases} \end{align}$$ Since $L(1) = 3 \geq 1/3=L(0)$, $L(1)$, observing $y=1$ is stronger evidence for $X=1$. Recall that with the Neyman-Pearson test, - If $\lambda \in (1/3, 3)$, then we always accept $H_{0}:X=0$ on $y=0$, since $L(0) < \lambda$, and conversely, we always reject $H_{0}$ on $y=1$, since $L(1)>\lambda$ for all $\lambda$. Therefore, $\gamma=0$, and $$\begin{align} \beta&=\mathbb{P}(Y=1 \mid X=0)+\gamma \mathbb{P}(Y=0 \mid X=0) \\ &=1/4+0 \cdot \mathbb{P}(Y=0 \mid X=0) \\ &=1/4. \end{align}$$ - If $\lambda=1/3$, then we randomize on $y=0$, and always reject on $y=1$ (since $L(1)=3 > 1/3$). Therefore, $$\begin{align} \beta &= \mathbb{P}(Y=1 \mid X=0)+\gamma \mathbb{P}(Y = 0 \mid X=0) \\ &= 1/4+\gamma(3/4) \iff \gamma=\frac{4\beta-1}{3}. \end{align}$$ - If $\lambda =3$, then we always reject $H_{0}$ on $y=0$, since $L(0)=1/3 \cancel{<}3$, and randomize on $y=1$, since $L(1)=3=\lambda$. Therefore, $$\begin{align} \beta &= \gamma\mathbb{P}(Y=1 \mid X=0)+\mathbb{P}(Y = 0 \mid X=0) \\ &= \gamma(1/4)+0 \iff \gamma=4\beta. \end{align}$$ Then, $\gamma=4\beta$ has a domain of $\beta \in [0, 1/4]$, and $\gamma=\frac{4\beta-1}{3}$ has a domain of $\beta \in [1/4, 1]$. This creates the following rule for the two cases (i.e., where $\beta < 1/4$ and $\beta > \frac{1}{4}$): $$\begin{align} \hat{X}_{\beta < \frac{1}{4}}=\begin{cases} 0 & \text{if } Y=0 \\ \operatorname{Ber}(\gamma=4\beta) & \text{if }Y=1, \end{cases} \end{align}$$ and $$\hat{X}_{\beta > \frac{1}{4}} = \begin{cases} \operatorname{Ber}\left( \gamma=\frac{4\beta-1}{3} \right) & \text{if } Y=0 \\ 1 & \text{if } Y=1. \end{cases}$$ For $\beta=1/4$, we have a standard $\hat{X}=1_{Y=1}$.