4 Sampling statistics, Bayesian inference and the Haar distribution
Every number a quantum processor delivers is an estimate made from a finite number of shots. This chapter gathers the statistical tools of the qg unit: how shot noise propagates, which prior is the natural one for an unknown qubit, how to build intervals that do not collapse near the poles, how to correct the bias of quadratic quantities and how many shots to plan for. All of them rest on the base of Chapter 1: what is estimated is always the probability \(p_0\), and since the milli-qang is linear in it, every interval and every shot plan is stated directly in m-qg. It closes with the mathematical foundation that explains why \(qg_Z\) is the canonical read-out.
4.1 Propagating shot noise
With \(N\) shots and \(k_0\) outcomes 0, the natural estimator is \(\widehat{qg}_Z=2k_0/N-1\). It is unbiased and its variance is exactly
\[ \operatorname{Var}(\widehat{qg}_Z)=\frac{1-qg_Z^2}{N}=\frac{\sin^2\theta}{N}, \]
which attains the Cramér–Rao bound for the parameter \(qg_Z\): no unbiased estimator does better with the same information. In milli-qang it is the resolution \(\sigma_m=1000\sqrt{1-qg_Z^2}/\sqrt N\) of Chapter 1 (Figure 1.3). Propagated to the angle by the delta method, the variance becomes constant, \(\operatorname{Var}(\hat\theta) \approx1/N\), as seen in Chapter 1 (§5). The two statements are compatible: \(qg_Z\) is the curved coordinate, with Fisher information \(1/(1-qg_Z^2)\) per shot, and \(\theta\) the flat one, with information 1 (Chapter 23).
The delta method is a first-order approximation and fails where the derivative vanishes. Near a pole, \(\widehat{qg}_Z\) is exactly \(\pm1\) in a sizeable fraction of experiments, and the estimated standard error \(\sqrt{(1-\widehat{qg}_Z^2)/N}\) becomes zero: the interval collapses exactly where the state is most extreme.
4.1.1 The azimuthal phase
The phase is estimated from the two transverse bases, \(\hat\varphi=\operatorname{atan2}(\widehat{qg}_Y,\widehat{qg}_X)\) (Section 1.4). With \(N\) shots in each basis, the delta method gives
\[ \operatorname{Var}(\hat\varphi)\approx\frac{1-\tfrac12\sin^2\theta\,\sin^2 2\varphi}{N\sin^2\theta} \;\le\;\frac{1}{N\sin^2\theta}. \]
The bound on the right, which is the one the author’s working version uses, is exact on the axes (\(\varphi\) a multiple of \(90^\circ\)) and near the poles, but on the equator at \(\varphi=45^\circ\) it doubles the true variance. The essential point stands: the phase becomes unobservable towards a pole, where \(\sin\theta\to0\). On the angular scale of Chapter 1, on the equator and on an axis, the deviation is \(159.15/\sqrt N\) qga, the same as for the polar angle. The transverse reading \(\widehat{qg}_X=\sin\theta\cos\varphi\), by contrast, does not diverge: its variance is \((1-qg_X^2)/N\le1/N\), for the same reason as that of \(\widehat{qg}_Z\).
4.2 The Haar distribution in qg space
What is known about a qubit before it is measured? If its state is “any state”, the natural distribution is the Haar measure, the only one invariant under rotations. In qg units it takes a remarkable form: \(qg_Z\) is uniform on \([-1,1]\). This is Archimedes’ hat-box theorem: the area of a band of the sphere between two latitudes is proportional to the difference of heights, not of angles. It was checked on 200 000 Haar states (§15).
The statistical consequence is direct. A uniform prior on \(qg_Z\) is equivalent to \(p_0\sim\mathrm{Beta}(1,1)\), and with \(k_0\) zeros in \(N\) shots the posterior is
\[ p_0\mid k_0\ \sim\ \mathrm{Beta}(k_0+1,\ N-k_0+1), \]
from which a credible interval for \(qg_Z=2p_0-1\) follows with a beta function alone (bayes_qg_estimate, implemented without external dependencies and checked against scipy.stats.beta).
The probability equation of Chapter 1 explains why this prior is so simple. Since \(m=2000\,p_0-1000\) is linear, a distribution uniform in \(qg_Z\) is uniform in milli-qang and uniform in \(p_0\): the three statements are one. In the angle, by contrast, the same measure has density \(\tfrac12\sin\theta\) and piles up at the equator (Figure 4.1). The “uninformative” prior depends on the scale it is written in, and Haar’s is flat exactly on the scale that is measured.
4.3 Exact intervals near the poles
The comparison with 50 shots shows where the choice of interval matters (§15):
| case | delta method | Wilson | Bayes–Haar |
|---|---|---|---|
| near a pole (\(\theta=0.08\)): coverage of the 95% interval | 7.6% | 92.4% | 92.4% |
| Haar-random states: coverage | 89.8% | 95.1% | 94.9% |
Near the pole the delta method covers the true value in fewer than one experiment in ten; Wilson and the Haar posterior recover the nominal coverage. On Haar states the posterior mean has a 4% lower mean squared error than the raw frequency.
Figure 4.2 shows the cause in milli-qang. With 50 shots, the width of the delta-method interval drops to zero at the ends of the scale: with 50 zeros in 50 shots it gives \([+1000,+1000]\) m-qg, an interval with no width. With 48 zeros it gives \([+811,+1029]\) m-qg, which runs off the top of the scale. Wilson gives \([+731,+978]\) and Bayes–Haar \(+885\) \([+731,+975]\) m-qg: widths that do not collapse and stay inside \([-1000,+1000]\).
The honest reading of the table has two parts. The Haar prior repairs the collapse of the delta method and has a clean geometric justification; but the Wilson interval, standard in statistics, achieves the same coverage, and the gain over good frequentist practice is in the point estimate and is modest. The qg unit does not reduce the number of shots a given precision requires; what it brings is making the cost explicit and choosing the interval well.
4.4 Second-order unbiased estimators
Quadratic quantities have a systematic bias. The plug-in estimator \(\widehat{qg}^{\,2}\) overestimates \(qg^2\) by \((1-qg^2)/N\), because shot noise always adds variance. The correction is exact:
\[ \widehat{qg^2}=\frac{N\,\widehat{qg}^{\,2}-1}{N-1},\qquad \mathbb E\big[\widehat{qg^2}\big]=qg^2 , \]
valid for \(N\ge2\) (qg2_unbiased). It can go negative when \(qg\approx0\), the price of being unbiased. With \(N=20\) and \(qg=0.6\), the plug-in estimator averages 0.390 and the corrected one 0.358, against the true value 0.36.
It matters because the Bloch radius of Chapter 5, \(r^2=qg_X^2+qg_Y^2+qg_Z^2\), is a sum of squares: without the correction each axis adds its bias, and an entangled state looks purer than it is (§97). The same correction enters the reading of the gravitational redshift with two clocks (§126, Chapter 8).
4.5 Planning the shots
The inverse relation, how many shots a given error needs, is the one used when preparing an experiment. plan_shots turns a target standard error of the filtered \(qg_Z\) into shots and estimated processor seconds, taking into account the fraction of shots the filter will keep (§143). For a target of 0.02 (20 m-qg) on two circuits per device it planned 3000–3400 shots, and over 30 repetitions the error of the worst qubit was 0.021–0.023 (21–23 m-qg): the plan held. For reference, Chapter 1’s formula without the filter gives \(N=(1000\sin\theta/\delta)^2=2500\) shots for 20 m-qg on the equator; the plan asks for more because it discounts the shots the filter will discard. The simple rule is \(N_\text{run}=N/(1-P_\text{discard})\): with a 20 % discard, the 2500 useful shots ask for 3125.
The per-qubit prediction, however, was 1.4–1.6 times optimistic on average, and its analysis is instructive: the qubits that missed were those near a pole (\(|qg|>0.8\), with ratios of 2.4–3.1), whose \(qg_Z\) the noise pulls away from \(\pm1\) and whose real variance is larger than the ideal one. The plan uses the worst qubit and therefore holds; the safe option, assuming no value of qg, is to plan for the worst case, \(qg=0\). The author’s working version attributes that excess to the curvature \(\kappa_{qg}=1000\,\lvert qg_Z\rvert\) of Chapter 1, largest at the poles: a random pulse error shifts the reading towards the equator in proportion to \(\kappa_{qg}\), and there the binomial variance is larger. It is an interpretation consistent with Chapter 1, but it was not tested on the data of §143.
4.6 Few-shot tomography and classical shadows
The full Bloch vector is \((qg_X,qg_Y,qg_Z)\), one per measurement axis. With 10 shots per axis, six estimators were compared on 2000 states (§35). The qg version applies the Haar prior axis by axis, \(\hat r_a=(2k_a-N)/(N+2)\), and projects onto the ball:
| ensemble | linear inversion | LI + projection | max. likelihood | qg-Haar | Jeffreys |
|---|---|---|---|---|---|
| Haar pure | 0.203 | 0.157 | 0.150 | 0.165 | 0.159 |
| uniform mixed | 0.247 | 0.211 | 0.210 | 0.185 | 0.196 |
| pure near a pole | 0.199 | 0.159 | 0.109 | 0.163 | 0.159 |
| strongly mixed (\(|r|<0.3\)) | 0.298 | 0.293 | 0.293 | 0.208 | 0.245 |
Linear inversion gives unphysical states in 52–88% of the runs on pure states. On mixed states the qg-Haar estimator is the best closed form (12–29% below maximum likelihood); on pure states its shrinkage towards the centre is a mistake, and it loses to maximum likelihood, by up to 50% near a pole. The right prior depends on which states are expected.
Would classical shadows (random Pauli measurements) do better than measuring the bases directly? For the quantities qang uses, no (§62). With exact per-shot variances on 6-qubit states, shadows need 3–41 times more shots for the same error: 3.0–4.3× for all the \(qg_Z\), 5.8–8.1× for the register-mean weight witness, 9–12.6× for all the \(ZZ\) correlators and 41× for the filtered imbalance. They only match direct measurement when the target is all one- and two-qubit quantities at once. And the shot-level filter does not exist with shadows: only \(3^{-6}=0.14\%\) of the random shots have every qubit measured in \(Z\).
4.7 The foundation: Riesz–Fréchet
A conceptual question remains: why \(qg_Z\) and not some other function of the populations? The operators of an \(n\)-qubit register form a finite-dimensional Hilbert space under the Hilbert–Schmidt product \(\langle A,B\rangle=\operatorname{Tr}(A^\dagger B)\), and density matrices live in it. The Riesz–Fréchet representation theorem says that every continuous linear functional \(f\) on a Hilbert space has the form \(f(x)=\langle y_f,x\rangle\) for a unique \(y_f\); in finite dimension continuity is automatic and the theorem reduces to linear algebra (Appendix F).
The application is immediate. \(\rho\mapsto qg_Z(\rho)=\operatorname{Tr}(\rho\,\sigma_z)\) is linear in \(\rho\), so it has a unique representative, \(\sigma_z\). Every linear read-out of a state (an expectation value, a population, a per-shot average) has that form, and \(qg_Z\) is the one whose representative is the \(Z\) observable. By contrast, \(qg_S(\rho)=H(\langle0|\rho|0\rangle)\) is strictly concave in \(\rho\): no operator represents it. Two properties already seen follow from this structural fact: linear functionals have unbiased sample-mean estimators and the entropy does not (Jensen’s inequality), and the identity \(qg_S=H((1+qg_Z)/2)\) shows that \(qg_S\) carries no information about the state beyond what the representative of \(qg_Z\) already extracts.
The milli-qang inherits this property: \(m(\rho)=1000\,\operatorname{Tr}(\rho\,\sigma_z)\) is linear too, with representative \(1000\,\sigma_z\), which is why its estimator is the shot average, without bias. It is the underlying reason Chapter 1 defined the unit on probability and not on angle: \(\theta=\arccos(qg_Z)\) is not linear in \(\rho\), has no representative and its direct estimator is biased. The same holds for the angle on the qga scale of Chapter 1: it is the same \(\theta\) multiplied by \(1000/2\pi\), so the angular scale changes the unit of the angle but does not make it linear in \(\rho\).
Open proposals. The author’s working version adds three proposals to this chapter, which the book records as unverified: watching the Fisher information \(qg_F=4000\,\operatorname{Var}(\sigma_z)\) so that an ansatz does not approach a global Haar 2-design (that the Haar measure over all of \(SU(2^n)\) produces barren plateaus is a known result (McClean et al. 2018); that \(qg_F\) serves to watch it is not); shot-noise propagation formulas for the regularized energy and the radiation function of Chapter 1, which depend on penalties not yet defined; and that the natural gradient in qg canonically diagonalizes the Fubini–Study tensor, which is true only for the coordinates \((\theta,\varphi)\) of one qubit (Section 1.4).
4.8 The derived quantities in sampling
The treatise’s formulation brings the derived quantities of Chapter 1 (Section 1.11) to sampling. All of them are estimated from the same counts, and each one inherits a statistical property of this chapter.
Two Fisher informations. A shot carries about the polar bias the information \(F_{qg}=1/(1-qg_Z^2)\), which diverges at the poles, and about the qang angle \(F_\theta=1\), constant. They are the same information in two coordinates, and the information curvature is the first one in bits: \(qg_\kappa=F_{qg}/\ln2\).
Signal-to-noise and curvature when all shots agree. The plug-in estimators of \(qg_{SNR}\) and \(qg_\kappa\) divide by \(1-\widehat{qg}_Z^{\,2}\) and become infinite when the \(N\) shots coincide, which is exactly the case of Section 4.3. The Bayes–Haar posterior mean keeps them finite: with 50 out of 50, \(qg_Z=50/52=0.962\), \(qg_{SNR}=3.50\) and \(qg_\kappa=19.1\).
The geodesic distance between two samples. With \(N_1\) and \(N_2\) shots, each qang angle has variance \(1/N\), so
\[ \widehat{qg}_L=\frac{\lvert\hat\theta_1-\hat\theta_2\rvert}{2}, \qquad \operatorname{Var}\big(\widehat{qg}_L\big)\approx\frac14\left(\frac1{N_1}+\frac1{N_2}\right), \]
wherever the two states are. It is half of Cohen’s \(h\), the classical effect size for comparing two proportions, so the test is not qang’s: qang gives it the name of a distance. With 750 and 700 zeros in 1000 shots each, \(\widehat{qg}_L=0.056\) rad (8.9 qga) with standard error \(0.0224\): a difference of 2.5σ. What Chapter 1 said holds: on a common meridian it is the Fubini–Study distance; with different azimuths, a lower bound.
The dissipative deficit from counts. \(qg_\tau=\sqrt{1-r^2}\) is a quadratic quantity, and it inherits the bias of Section 4.4 with the opposite sign. With \(N\) shots per axis, the plug-in radius satisfies
\[ \mathbb E\big[\hat r^{\,2}\big]=r^2+\frac{3-r^2}{N}, \]
so shot noise makes the state look purer than it is. A pure state measured with 100 shots per axis averages \(\hat r^{\,2}=1.02\), outside the sphere. A state with \(r=0.9\) (\(qg_\tau=0.436\)) averages \(\hat r^{\,2}=0.832\) with the plug-in estimator and \(0.810\) with the corrected one, which is the true value. The square root, however, is not unbiased even with the correction: \(\widehat{qg}_\tau\) averages \(0.376\) with the plug-in radius and \(0.404\) with the corrected one, both below \(0.436\), because the root is concave and is clipped at zero. The formulation proposes \(\sqrt{\max(0,1-\hat r^{\,2})}\) as an unbiased estimator; the corrected one halves the bias, it does not remove it.
4.9 Without qang and with qang
| action | without qang | with qang | difference |
|---|---|---|---|
| confidence interval near a pole | delta method: coverage below 10% | Bayes–Haar, \(\mathrm{Beta}(1,1)\): uniform in m-qg, nominal coverage | in favour of the method, not only of qang: the Wilson interval, without qang, also recovers it; qang justifies the prior (Haar is uniform in m-qg) |
| quadratic quantities (radii, \(qg^2\)) | square the estimate: biased upward | \((N\widehat{qg}^{\,2}-1)/(N-1)\) | the standard unbiased estimator; qang packages it and applies it in every chapter |
| how many shots to take | binomial variance of the state | \(N=(1000\sin\theta/\delta)^2\) to resolve \(\delta\) m-qg | equivalent; the m-qg formula is the one used for planning |
| compare two readings | Cohen’s \(h\) | \(qg_L=h/2\), with variance \(\tfrac14(1/N_1+1/N_2)\) over the whole circle | none: it is the same test, with the name of a distance |
| estimate the lack of purity | \(1-\operatorname{Tr}\rho^2\) with tomography | \(qg_\tau\) with unbiased squares | the bias of \(\hat r^{\,2}\) disappears; that of \(\widehat{qg}_\tau\) is halved, not removed |
| measure qang quantities | classical shadows | direct \(Z\) measurement | 3–41 times fewer shots, but it is a measurement choice, not an effect of the unit |
| justify the reading | — | Riesz–Fréchet: \(qg_Z\) is the canonical linear reading | in favour: explains why \(qg_Z\) is averaged and not the angle |
Balance. The chapter’s statistics are the classical statistics of proportions. qang contributes the choice of coordinate (uniform under Haar, linear by Riesz–Fréchet) that says which prior to use and what to average; the coverage improvement exists without qang too.
4.10 In code
import math
from qang import probability, proportions, statistics
statistics.bayes_qg_estimate(48, 50) # Haar posterior: qg_Z 0.885, interval [0.731, 0.975]
statistics.wilson_qg_estimate(48, 50) # the frequentist alternative
statistics.qg2_unbiased(16, 20) # qg^2 without the (1 - qg^2)/N bias
statistics.radius2_estimate([(77, 100), (50, 100), (86, 100)]) # unbiased r^2 and its error, three axes
proportions.cohens_h(0.75, 0.70) / 2 # qg_L between two readings: 0.056 rad
probability.analyze(48, 50).milliqang # (920.0, 730.8, 977.9) m-qg with the Wilson interval
probability.shots_for_resolution(20, math.pi / 2) # 2500 shots for 20 m-qg on the equator4.11 Summary
- \(\widehat{qg}_Z\) is unbiased and attains the Cramér–Rao bound; the delta method collapses near the poles, where its m-qg interval loses its width or runs off the scale.
- Under the Haar measure \(qg_Z\) is uniform on \([-1,1]\), that is, uniform in milli-qang and in \(p_0\); the matching prior is \(\mathrm{Beta}(1,1)\) and gives exact intervals with a beta function.
- Wilson and Bayes–Haar recover the nominal coverage where the delta method covers less than 10%; neither reduces the shots required.
- Quadratic quantities need the correction \((N\widehat{qg}^{\,2}-1)/(N-1)\); without it, Bloch radii are overestimated.
- Of the derived quantities, \(qg_L\) between two samples is half of Cohen’s \(h\), with variance \(\tfrac14(1/N_1+1/N_2)\); \(qg_\tau\) estimated from counts stays biased low even with the quadratic correction.
- Planning on the worst qubit meets the target; the qubits near a pole are the ones the noise pulls away most.
- For qang’s quantities, measuring directly in \(Z\) costs 3–41 times fewer shots than classical shadows.
- \(qg_Z\), and with it the milli-qang, is the canonical linear read-out by the Riesz–Fréchet theorem; \(qg_S\) and the angle are not linear and have no representative.
Bibliographic notes
The Cramér–Rao bound is in Cramér (1946) and Rao (1945), and the invariant prior, in Jeffreys (1946). The score interval is Wilson (1927). For sampling the Haar measure, see Mezzadri (2006); classical shadows are Huang et al. (2020). The Riesz–Fréchet representation theorem is Riesz (1907) and Fréchet (1907). The numbered research notes (§) cited in this chapter and the scripts that check it are in the repository (Monteverde 2026a); the definition of the qang unit is in Monteverde (2026b).