Chapter 15: Paradoxes of probability theory

p. 460, equation (15.18): Should be `` $p^{(0)} = (1, 0, ..., 0)^T$ '' or, equivalently, `` $p^{(0)}_{i} = \delta_{i,0}$ ''.
p. 467, equation (15.42): insert a minus sign in front of the argument to the exponential function.
p. 473, second full paragraph: ``the right-hand sides of (15.58) and (15.61)'' should be ``the left-hand sides...''.
p. 475, equation (15.67): insert a minus sign before the argument to the exponential function.
p. 481, equation (15.89): $\frac{1}{2}\omega^2$ in the exponential function should be $-\frac{1}{2}\omega^2$ .
p. 481, equations (15.87) and (15.88): The factor $\omega^{n+\gamma-1}$ should be $\omega^{n+\gamma-2}$ .

Commentary: The Marginalization Paradox

After spending many hours going over Jaynes's treatment of the Marginalization Paradox, I've come to the conclusion that he got this one wrong: (15.72) is wrong, and (15.70) is the correct formula also for . Sections 15.8 and 15.9 are a puzzling anomaly, as Jaynes unaccountably breaks a number of the rules he emphasizes so often elsewhere in the book, and this leads him into error. I've written up my conclusions in a separate note (postscript, PDF). In summary, here is what I've shown:

The paradox arises from an unnoticed divergent integral that shows up when one tries to go from $p(\zeta \mid y,z) = f(\zeta,z)$ to $p(\zeta \mid z) = f(\zeta,z)$ ; this step is invalid because it requires multiplying by $p(y\mid z)$ and then integrating out , but it seems to have escaped notice that the improper prior over $\eta$ results in $p(y\mid z)$ also being improper.
In the specific case of the change-point problem, if one derives $p(\zeta \mid z)$ for the proper prior $\pi_b(\eta) \propto \eta^a \exp(-b\eta)$ , then takes the limit as $b\rightarrow 0$ (going to the limiting improper prior $\pi(\eta) \propto \eta^a$ ), one obtains 's answer (15.70), and not 's answer (15.72).
The issue of non-uniform convergence plays an important role in this problem, and as $p(\zeta \mid y,z)$ converges to (15.72), the distribution $p(y\mid z)$ retains significant probability mass in the (ever-smaller) region where $p(\zeta \mid y,z)$ is far from convergence.

It's worth noting, however, that my resolution of the paradox was obtained simply by following the practices Jaynes advocates in PTLOS.

Is it a disaster for Bayesian analysis if we have to abandon the use of improper priors? I don't think so. As Jaynes points out, the really important use of improper priors is as a zero-point for constructing maximum-entropy priors. Furthermore, he shows in one problem after another that even in situations where one might be tempted to say that we are totally ignorant about some parameter, simple common-sense reasoning and application of physical constraints allow us to create a defensible proper prior. There are, in fact, some pretty good reasons (beyond the MP) to stick to proper priors:

A lot of interesting problems can't be solved analytically, requiring instead the use numerical methods that generally won't work with improper priors. In particular, the use of Markov Chain Monte Carlo (e.g., BUGS) has become increasingly popular over the last decade, and this requires proper priors.
Model comparison (see Chapter 20) -- one of the more interesting applications of Bayesian methods -- requires proper priors.

Philip Dawid informs me that in 1996 he, Stone, and Zidek also wrote a response to Chapter 15, based on the version of PTLOS available on the Internet at that time; you can find it here as report 172 for 1996.

Some final technical comments:

One obtains (15.87) via the change of parameters $\omega = R/\sigma$ .
On p. 482 Jaynes talks about applying (15.89) to obtain a posterior over $\zeta$ conditional on . That is, (15.89) is to be used as a likelihood. Unfortunately, the proportionality in (15.89) retains only factors dependent on , when instead it needs to retain those factors dependent on $\zeta$ or $\sigma$ (in particular, a factor of $\exp(-n\zeta^2/2)$ is missing. (This comment comes from DSZ's response, mentioned above.)