Cox-Jaynes Derivation

Cox-Jaynes Derivation

The Cox-Jaynes derivation shows that the standard rules of probability — the product rule $\mathbb{P}(AB\mid C)=\mathbb{P}(A\mid C)\mathbb{P}(B\mid AC)$ and the sum rule $\mathbb{P}(A^c\mid C)+\mathbb{P}(A\mid C)=1$ — follow from a small set of consistency requirements on degrees of belief, without any reliance on long-run frequencies. The derivation justifies Bayesian probability as a calculus of plausibility and provides the philosophical foundation for treating priors as legitimate quantitative inputs to inference.

The argument begins with real-valued plausibilities $(A\mid B)$ of a proposition $A$ given background information $B$, with the single ordering axiom that greater plausibility corresponds to a larger real number. Four consistency requirements are imposed:

1. Functional decomposition for conjunction. There exists a continuous function $F$, strictly increasing in each argument, such that $(AB\mid C)=F((B\mid C),(A\mid BC))$.
2. Associativity. $F(F(x,y),z)=F(x,F(y,z))$ for all real $x,y,z$, so that $(ABC\mid D)$ has an unambiguous value.
3. Functional decomposition for negation. There exists a continuous strictly-decreasing function $S:[0,1]\to [0,1]$ with $S(0)=1$, $S(1)=0$, such that $w(A^c\mid C)=S(w(A\mid C))$, where $w$ is the transformation introduced below.
4. Impossibility/certainty endpoints. $w(\text{impossible})=0$ and $w(\text{certain})=1$.

Under these requirements, Cox’s theorem (with completions by Jaynes) shows:

• The associativity equation forces $F(x,y)=w^{-1}(w(x)w(y))$ for a strictly increasing continuous function $w:\mathbb{R}\to [0,1]$, unique up to the power transformation $w\mapsto w^{\alpha }$ for $\alpha >0$.
• The analogous equation for $S$ forces $S(x)=(1-x^{\alpha }{)}^{1/\alpha }$ for some $\alpha >0$.
• Defining $\mathbb{P}:=w^{\alpha }$, the transformed plausibilities satisfy the three rules of probability:
  $0\le \mathbb{P}(A\mid B)\le 1,\mathbb{P}(\text{impossible})=0,\mathbb{P}(\text{certain})=1,$
  $\mathbb{P}(AB\mid C)=\mathbb{P}(A\mid C)\mathbb{P}(B\mid AC)\text{(product rule),}$
  $\mathbb{P}(A^c\mid C)+\mathbb{P}(A\mid C)=1\text{(sum rule).}$

The usual inclusion-exclusion form $\mathbb{P}(A\cup B\mid C)=\mathbb{P}(A\mid C)+\mathbb{P}(B\mid C)-\mathbb{P}(AB\mid C)$ is a downstream consequence, not an independent axiom. The theorem does not pin down a unique probability; it says only that any consistent plausibility calculus is isomorphic to ordinary probability.

Prerequisites

• 3.1 Bayesian Framework — why a non-frequentist interpretation of probability is needed to treat priors as real objects.
• 1.3 Conditional Probability — classical definitions of $\mathbb{P}(A\mid B)$ and the product rule in its standard form.

Intuition

The frequentist interpretation defines probability as a long-run relative frequency of an event under repeated trials. This works for coin flips and dice rolls, but it is awkward for statements like “there is life on Mars” or “this patient has disease $X$” — the event either happens or it doesn’t, and no repeatable trial exists to generate a frequency. A Bayesian wants to treat degree of belief as the primitive quantity and derive the rules of probability from consistency requirements on belief updates. The Cox-Jaynes derivation is the technical backbone of that program.

The central move is twofold. First, replace the abstract concept of “probability” with a concrete, intuitive one: the plausibility $(A\mid B)$ of proposition $A$ given background information $B$. Plausibilities are just real numbers with the property that a higher number means greater belief. Second, insist that plausibilities obey a handful of consistency requirements — if two different valid calculations give different answers, something is wrong — and extract the algebraic form of those requirements. The punchline is that any consistent plausibility calculus, after an appropriate monotone reparametrization, is isomorphic to classical probability. Consistency alone determines the algebra, so the product and sum rules are not conventions imported from frequency theory but theorems about how any coherent reasoner must update beliefs.

Properties

(Product Rule from Associativity) The associativity of conjunction forces a multiplicative structure on transformed plausibilities.

Compute $(ABC\mid D)$ in two ways via the functional decomposition $F$: first as $F((C\mid D),F((B\mid CD),(A\mid BCD)))$, and second as $F(F((C\mid D),(B\mid CD)),(A\mid BCD))$. Consistency demands these agree, yielding the functional equation
$F(F(x,y),z)=F(x,F(y,z))\text{for all real }x,y,z.$
Cox’s theorem (with subsequent completions by Aczél and Jaynes) shows the only continuous, strictly-monotone solutions are of the form $F(x,y)=w^{-1}(w(x)w(y))$ for a positive continuous increasing function $w$. Applying $w$ to both sides of $(AB\mid C)=F((B\mid C),(A\mid BC))$ collapses it to the multiplicative product rule
$w(AB\mid C)=w(A\mid C)w(B\mid AC).$

(Sum Rule from Negation Consistency) Two ways to compute $w(AB)$ in the nested setup $B^c\subseteq A$ force the negation function $S$ to have a specific power-law form.

Consider events with $B^c\subseteq A$ (equivalently $A^c\subseteq B$). Compute $w(AB)$ as $w(A)w(B\mid A)=xS(S(y)/x)$ using the product rule and $w(A^c\mid B)=S(w(A\mid B))$; compute it again by swapping $A$ and $B$ to get $yS(S(x)/y)$. Setting the two equal forces $S$ to satisfy
$xS(S(y)/x)=yS(S(x)/y)\text{for all valid }x,y.$
The unique continuous strictly-decreasing solutions are $S(x)=(1-x^{\alpha }{)}^{1/\alpha }$ for some $\alpha >0$, giving
$w^{\alpha }(A^c\mid C)=1-w^{\alpha }(A\mid C).$
Denoting $\mathbb{P}:=w^{\alpha }$, this is the standard sum rule $\mathbb{P}(A^c\mid C)+\mathbb{P}(A\mid C)=1$.

(Uniqueness Up to Power Transformation) The rules are forced on plausibility calculus but the specific value of the exponent $\alpha$ is not.

If $w$ satisfies the product rule, so does $\tilde{w}=w^{\beta }$ for any $\beta >0$. The Cox-Jaynes derivation therefore does not isolate a single probability assignment — it only says that any consistent plausibility calculus is isomorphic to ordinary probability. The choice of scale (which exponent $\alpha$ to fold into the definition of $\mathbb{P}$) is a convention, and taking $\alpha =1$ in the canonical normalization $\mathbb{P}(\text{certain})=1$, $\mathbb{P}(\text{impossible})=0$ gives the familiar $[0,1]$-valued probability.

(Inclusion-Exclusion as Consequence) $\mathbb{P}(A\cup B\mid C)=\mathbb{P}(A\mid C)+\mathbb{P}(B\mid C)-\mathbb{P}(AB\mid C)$ follows from the sum and product rules.

Apply the sum rule to the negation of $A\cup B=A^c\cap B^c$:
$\mathbb{P}(A\cup B\mid C)=1-\mathbb{P}(A^c{B}^c\mid C)=1-\mathbb{P}(B^c\mid A^{c}C)\mathbb{P}(A^c\mid C).$
Expand $\mathbb{P}(B^c\mid A^{c}C)=1-\mathbb{P}(B\mid A^{c}C)$ and $\mathbb{P}(A^c\mid C)=1-\mathbb{P}(A\mid C)$, then use the product rule on $\mathbb{P}(A^{c}B\mid C)=\mathbb{P}(B\mid C)\mathbb{P}(A^c\mid BC)=\mathbb{P}(B\mid C)(1-\mathbb{P}(A\mid BC))$ to collect terms. After cancellation,
$\mathbb{P}(A\cup B\mid C)=\mathbb{P}(A\mid C)+\mathbb{P}(B\mid C)-\mathbb{P}(B\mid C)\mathbb{P}(A\mid BC)=\mathbb{P}(A\mid C)+\mathbb{P}(B\mid C)-\mathbb{P}(AB\mid C).$
The full inclusion-exclusion principle for $n$ events follows by induction on $n$.

(Bayes’ Rule as Corollary) Applying the product rule in two different orders yields Bayes’ theorem.

The product rule gives both $\mathbb{P}(AB\mid C)=\mathbb{P}(A\mid C)\mathbb{P}(B\mid AC)$ and $\mathbb{P}(AB\mid C)=\mathbb{P}(B\mid C)\mathbb{P}(A\mid BC)$. Equating the two and solving for $\mathbb{P}(A\mid BC)$,
$\mathbb{P}(A\mid BC)=\frac{\mathbb{P}(A\mid C)\mathbb{P}(B\mid AC)}{\mathbb{P}(B\mid C)}.$
Reading $A$ as the hypothesis, $B$ as new data, and $C$ as background, this is exactly Bayes’ theorem with posterior on the left and prior times likelihood over evidence on the right. Bayes’ rule is not an extra axiom — it is an immediate consequence of the product rule being symmetric in its arguments.

(Scope and Limitations) The derivation establishes isomorphism to classical probability, not uniqueness of values, and leaves the assignment of priors undetermined.

Cox-Jaynes shows how to update beliefs consistently but not what initial beliefs to hold. Assigning priors remains an empirical, philosophical, or subjective matter, and different schools (objective Bayes with reference priors, subjective Bayes with elicited priors, empirical Bayes with data-driven hyperparameters) disagree. Several follow-up papers (Paris 1994, Halpern 1999, Van Horn 2003) have tightened the technical assumptions needed for the derivation — early versions glossed over continuity and differentiability requirements — so the theorem is now stated with more care than Cox originally gave, but the philosophical payoff is unchanged: probability calculus is the unique consistent language for reasoning under uncertainty.

Examples

Deriving Bayes' rule from the product rule

Start with the product rule in both orderings:
$\mathbb{P}(H\cdot D\mid I)=\mathbb{P}(H\mid I)\mathbb{P}(D\mid HI)=\mathbb{P}(D\mid I)\mathbb{P}(H\mid DI),$
where $H$ is a hypothesis, $D$ is data, and $I$ is background information. Solving the right-hand equality for $\mathbb{P}(H\mid DI)$,
$\mathbb{P}(H\mid DI)=\frac{\mathbb{P}(H\mid I)\mathbb{P}(D\mid HI)}{\mathbb{P}(D\mid I)}.$
This is Bayes’ rule with posterior $\mathbb{P}(H\mid DI)$, prior $\mathbb{P}(H\mid I)$, likelihood $\mathbb{P}(D\mid HI)$, and evidence $\mathbb{P}(D\mid I)={\sum }_{H^{\prime }}\mathbb{P}(H^{\prime }\mid I)\mathbb{P}(D\mid H^{\prime }I)$. The sum-rule expansion of the evidence uses only the product and sum rules derived by Cox-Jaynes; no frequentist assumption enters anywhere.

Checking the sum rule on a medical-test example

Let $D$ = “patient has disease,” $T$ = “test is positive,” and $I$ = background about the test and population. The sum rule says $\mathbb{P}(D\mid TI)+\mathbb{P}(D^c\mid TI)=1$: the posterior probabilities of having and not having the disease, conditional on a positive test, must sum to $1$. This is immediate from the Cox-Jaynes derivation because the sum rule is an identity in $\mathbb{P}$; any consistent reasoner who assigned, say, $\mathbb{P}(D\mid TI)=0.9$ and $\mathbb{P}(D^c\mid TI)=0.3$ would be violating the calculus. The fact that this identity is forced by consistency — rather than imposed by definition — is the payoff of the derivation.

Derivation

Goal. Sketch the two key functional-equation arguments that anchor the theorem: the product rule from associativity, and the sum rule from the negation consistency equation.

Step 1 — Product rule from associativity. Assume $(AB\mid C)=F((B\mid C),(A\mid BC))$ for some continuous, strictly-monotone $F$. Applying this in two different orderings to $(ABC\mid D)$ gives
$F(F((C\mid D),(B\mid CD)),(A\mid BCD))=F((C\mid D),F((B\mid CD),(A\mid BCD))),$
which is exactly the associativity condition $F(F(x,y),z)=F(x,F(y,z))$. Aczél (1966) proved that the only continuous, strictly-monotone solutions are
$F(x,y)=w^{-1}(w(x)w(y))$
for some continuous, strictly-increasing $w$. Applying $w$ to $(AB\mid C)=F((B\mid C),(A\mid BC))$ yields $w(AB\mid C)=w(A\mid C)w(B\mid AC)$, the multiplicative form of the product rule. The function $w$ is unique only up to the transformation $w\mapsto w^{\alpha }$ for $\alpha >0$.

One checks endpoints: the unit of multiplication forces $w(\text{certain})=1$, and for any $A$, iterating $w(A\mid C)=w(A\mid C)w(\text{certain}\mid AC)$ gives $w(\text{certain}\mid AC)=1$; a symmetric argument shows $w(\text{impossible})=0$, and $w$ maps into $[0,1]$.

Step 2 — Sum rule from the negation equation. Assume the existence of a continuous, strictly-decreasing $S:[0,1]\to [0,1]$ with $S(0)=1,S(1)=0$ and $w(A^c\mid C)=S(w(A\mid C))$. Consider propositions $A$ and $B$ with $B^c\subseteq A$ (so $A^c\subseteq B$). Compute $w(AB)$ in two ways.

First way. Apply the product rule and then the negation function:
$w(AB)=w(A)w(B\mid A)=w(A)S(w(B^c\mid A))=w(A)S(w(B^c)/w(A))=w(A)S(S(w(B))/w(A)),$
using $w(B^c\mid A)=w(B^c)/w(A)$ (a consequence of $B^c\subseteq A$ so $A{B}^c=B^c$) and the negation equation. Writing $x=w(A),y=w(B)$, this is $xS(S(y)/x)$.

Second way. Swap $A$ and $B$; by the same steps, $w(AB)=yS(S(x)/y)$.

Equating the two,
$xS(S(y)/x)=yS(S(x)/y)\text{for all }0\le x,y\le 1\text{ with }S(y)\le x.$
Jaynes (2003, §2.2) shows the only continuous, strictly-decreasing solutions with the required endpoint conditions are $S(x)=(1-x^{\alpha }{)}^{1/\alpha }$ for some $\alpha >0$. Substituting, $w(A^c)=(1-w(A{)}^{\alpha }{)}^{1/\alpha }$, or equivalently $w(A^c{)}^{\alpha }+w(A{)}^{\alpha }=1$.

Step 3 — Consolidate. Define $\mathbb{P}:=w^{\alpha }$. Raising both sides of the product rule to the $\alpha$-th power gives $\mathbb{P}(AB\mid C)=\mathbb{P}(A\mid C)\mathbb{P}(B\mid AC)$. The negation equation becomes $\mathbb{P}(A^c\mid C)+\mathbb{P}(A\mid C)=1$. The endpoints $\mathbb{P}(\text{certain})=1$, $\mathbb{P}(\text{impossible})=0$ follow from $w(\text{certain})=1,w(\text{impossible})=0$ and $\alpha >0$. Thus $\mathbb{P}$ satisfies the three axioms of classical probability, and inclusion-exclusion follows as a corollary.

See Also

• 3.1 Bayesian Framework — Bayes’ rule, priors, posteriors; uses the product rule derived here.
• 1.3 Conditional Probability — classical definition of $\mathbb{P}(A\mid B)$ that Cox-Jaynes justifies axiomatically.
• 3.3 Bayes Factors — hypothesis testing built on the product rule in the Bayesian sense.
• 3.7 Marginal Likelihood — the evidence $\mathbb{P}(D\mid I)$ whose computation uses the sum rule.