ksvanhorn.com

Home Bayes Home Jaynes Errata Articles Books Software Contact

Commentary: Note on exchangeability and de Finetti's Theorem In several places Jaynes refers to exchangeability and de Finetti's Theorem without defining these; finally, in Chapter 18 section 16 (p. 586 onward) he says a little bit about just what de Finetti's Theorem is. For those readers who are unfamiliar with this topic, Bernardo and Smith's book Bayesian Theory has a nice discussion in sections 4.2 and 4.3. Here is a brief, simplified summary of the definitions and theorems given therein, translated into the vocabulary and notation of Jaynes. Notation. We write $x_1^n$ for $x_1,...,x_n$, and $x_1^{\infty}$ for the infinite sequence $x_1,x_2,\ldots$. For simplicity, we assume that the set of possible values for each variable $x_i$ is the same finite set $S$. Definition of finite exchangeability. The variables $x_1^n$ are said to be (finitely) exchangeable for a state of information $I$ if, for any constant values $X_1^n$, we have $$P(x_1^n = X_1^n \mid I) = P(y_1^n = X_1^n \mid I)$$ whenever $y_1^n$ is a permutation of the variables $x_1^n$. Definition of infinite exchangeability. The infinite sequence of variables $x_1^{\infty}$ is said to be infinitely exchangeable for a state of information $I$ if every finite subsequence of $x_1^{\infty}$ is exchangeable for $I$. Theorem. Let $W$ be the set of probability mass functions over $S$. If $x_1^{\infty}$ is an infinitely exchangeable sequence of variables for $I$, then there exists a probability density $f$ over $W$ such that $$P(x_1^n = X_1^n \mid I) = \int_{w\in W} f(w) \prod_i w(X_i)$$ for any constant $X_1^n$. (Note that $f$ may involve delta functions, to assign positive probability to a single specific value in $W$.) In other words, we may reason as if there exists some additional variable $u$ such that the variables $x_1^n$ are independently and identically distributed when the value of $u$ is known, and $P(x_i = s \mid u = w, I)$ is $w(s)$.