Afterword
The workflow, and what comes next

Claim. Using about 3,000 respondents to an American adolescent-health survey, Kanazawa reported that the children of parents rated "very attractive" were girls 56% of the time, against 48% for other parents: an 8-point difference, 2.44 standard errors from zero.

Key evidence. One comparison, chosen from several natural ones (five attractiveness categories, several ways of grouping them); the regression of sex ratio on attractiveness was not significant (estimate 1.5, standard error 1.4, in Gelman's account).

The check that matters. Prior knowledge. The proportion of girls at birth is remarkably stable: known factors such as race, parental age and birth order move it by about a percentage point or less, and even extreme poverty and famine by at most about three points. Gelman and Weakliem argue that a plausible effect of beauty is a few tenths of a point, and almost certainly under one point, in either direction, and the retrodesign above shows what follows: with a standard error of 3.3 points, the study has about 5% power, and a significant result is almost bound to be wildly exaggerated and has a large chance of the wrong sign.

What it does not show. It does not show that beauty has no effect on the sex ratio; it shows that this study could not have detected an effect of plausible size. The general lesson, that small studies of small effects produce dramatic, unreplicable estimates, became one of the foundations of the replication debate (history box below).

Claim. A poll run on the Xbox gaming platform during the 2012 US presidential campaign, whose respondents were 93% male (against 47% of the electorate) and 65% aged 18 to 29 (against 19%), gives accurate election forecasts once it is analysed with multilevel regression and poststratification (MRP).

Key evidence. About 750,000 interviews with 346,000 people. Multilevel logistic regressions estimate vote intention in each of 176,256 cells defined by sex, race, age, education, state, party identification, ideology and 2008 vote; the cell estimates are then weighted by the cells' shares of the electorate, taken from the 2008 exit poll. The day-before-election estimate of Obama's two-party vote share was within 0.6 points of the result, comparable with the poll aggregates, and the median error across states was 1.8 points.

The check that matters. The forecasts were made with information available at the time (2008 exit-poll cell weights, not 2012 ones), and compared with the actual results and with the 2012 exit poll's subgroup estimates. Partial pooling (Chapter 6) is what makes 176,256 cells estimable from a sample that is sparse in most of them.

What it does not show. MRP corrects for selection on the variables used to define the cells; it cannot correct for selection on anything else, such as Xbox users differing politically from demographically similar non-users (a missing-at-random assumption, Chapter 7). The paper used approximate marginal maximum likelihood rather than full Bayesian inference, and its electoral-college intervals were too wide in the tails, which the authors attribute to state-level correlations the model did not capture.

Claim. The goals scored in a football match can be modelled as Poisson counts whose log rates are the sum of a home advantage, the attacking strength of the scoring team and the defensive weakness of the conceding team, with team strengths drawn from common distributions; such a model describes a season of Italy's Serie A well and can predict results.

Key evidence. Posterior predictive checks of the final league table for the 1991–92 Serie A season, team by team, and an application to 2007–08.

The check that matters. The authors noticed that the hierarchical model overshrinks: the best and worst teams are pulled too far toward the middle, so the model underpredicts their points. Their remedy was a mixture prior in which each team's attack and defence each belong to a top, middle or bottom group (Chapter 11), each group with its own distribution of strengths. Noticing the problem required comparing predicted and observed tables at the extremes, not only overall fit.

What it does not show. The paper does not test genuinely out-of-sample predictions against bookmakers' odds, the benchmark that matters for anyone who would use it, nor does it allow team strengths to change during a season. The model descends from Maher (1982) and Dixon and Coles (1997), whose correction for low-scoring draws it does not include. Section A.6 reproduces the basic model on a recent season and makes the comparison with the market.

Claim. Student evaluations of teaching (SET) are not related to how much students learn. Earlier meta-analyses that found moderate correlations between an instructor's ratings and the learning of their students were artefacts of small studies.

Key evidence. A reanalysis of the multisection studies (courses taught in many sections by different instructors, with a common final exam) behind three earlier meta-analyses, and a new meta-analysis of 97 such studies from 51 articles. Plots of effect size against sample size showed large correlations only in small studies; adjusting for this small-study effect left a correlation near 0.1, and near zero for the studies that also adjusted for students' prior ability.

The check that matters. The funnel-plot logic is a Type M argument across a literature: if small studies report large effects and large studies small ones, the large effects are most likely exaggerated by noise and selective publication. A Bayesian random-effects meta-analysis with study size as a moderator (Chapter 6) makes the same point with an explicit model.

What it does not show. That SETs measure nothing: they may measure student satisfaction, and they are affected by things other than teaching, such as an instructor's gender. That last question is causal, and the strongest evidence for it comes from designs, not meta-analysis: MacNell, Driscoll and Hunt (2015) randomised the perceived gender of online instructors, and Boring, Ottoboni and Stark (2016) analysed a French university where students were assigned to sections, and found that ratings were affected by gender and only weakly related to exam performance. Project 6 in Section A.9 builds on both.

Question

How much do the error rates of individual qubits on today's superconducting processors drift from day to day, how much of the variation is shared across a device, and how much better would circuits run if qubits were chosen using predicted error rates, with uncertainty, rather than the latest calibration?

Why it is new

Device calibrations are reported as point values and used as if exact. A Bayesian time-series model gives forecasts with intervals, separates measurement noise from real drift and pools information across qubits; the qubit-selection question turns it into a decision problem with a measurable payoff.

Data

IBM publishes calibration data (T1 and T2 coherence times, gate and readout error rates) for its cloud devices, refreshed by regular characterisation runs; a free account and Qiskit retrieve them, including past calibrations, and the calibration data behind a published study of how the errors of four IBM devices changed over time (collected over 90 days) are archived on Zenodo (doi:10.5281/zenodo.4898393). For the decision experiment, run the same benchmark circuits on qubits chosen by each rule on a free-tier device.

Model skeleton

Log error rates $y_{qt}$ for qubit $q$ on day $t$: $y_{qt}={\mu }_q+x_{qt}+{\epsilon }_{qt}$, with a device-level random walk or AR(1) shared component, qubit-level AR(1) deviations $x_{qt}$, qubit intercepts partially pooled by device, and a measurement-error term whose scale is informed by repeated calibrations within a day. Occasional recalibration jumps can be modelled as a mixture of normal and heavy-tailed innovations.

Checks

One-day-ahead forecast calibration (do 90% intervals contain 90% of next-day values?) against persistence (tomorrow = today) and moving averages; posterior predictive checks of autocorrelation; a randomised comparison of qubit-selection rules on the fidelity of benchmark circuits (Chapter 13).

Chapters

4 (models for rates), 5 (diagnostics), 6 (pooling across qubits and devices), 7 (measurement error, gaps in the record), 8 (state-space models and forecasting), 11 (copulas for the residuals: do qubits on one device have bad days together?), 13 (the randomised decision experiment).

Publishable result

Forecast intervals for device noise with demonstrated calibration, and an estimate of how much uncertainty-aware qubit selection improves circuit fidelity: suitable for IEEE Transactions on Quantum Engineering or the IEEE Quantum Week conference.

Risks and fallback

Devices are retired and access policies change, so archive everything you download. If the decision experiment shows no gain, the forecasting comparison is still a result. A related Bayesian thread to read first is sequential Monte Carlo for learning device parameters (Granade et al., 2012).

Question

Expected-goals (xG) models give each shot a probability of scoring from its location and context. Do some players consistently score more than their shots' xG predicts, and how much of a striker's apparent over-performance in one season survives into the next?

Why it is new

"Goals minus xG" is widely quoted for single seasons and usually treated as skill. A hierarchical model estimates the between-player spread of finishing skill with honest uncertainty, and the season-to-season check measures how much of the raw number is noise: a Type M problem in disguise.

Data

StatsBomb's open data on GitHub: event data for every shot, with StatsBomb's own xG and the positions of nearby players, for several complete competitions (men's and women's). Use their xG as an offset, or fit your own shot model as a check.

Model skeleton

For shot $i$ by player $p$: ${\text{goal}}_i\sim \text{Bernoulli}({\pi }_i)$, $logit{\pi }_i=logit({\text{xG}}_i)+{\theta }_p+\beta {\text{foot or header}}_i$, with ${\theta }_p\sim \text{Normal}(0,{\sigma }_{\theta })$ or a horseshoe prior if most players are average; extend with player-season effects to measure persistence.

DAG

Skill affects both shot selection and conversion, so "goals above xG" conditions on a post-treatment variable (shot quality); state this, and estimate the skill that operates given the shot.

Checks

Prior predictive simulation of season totals; posterior predictive checks of the distribution of goals minus xG across players; out-of-sample prediction of next-season conversion against the raw over-performance and against xG alone; calibration of the xG offset itself.

Chapters

2 (priors), 4 (logistic regression with an offset), 6 (partial pooling), 7 (measurement error in xG), 10 (sparse shrinkage), 12 (calibration).

Publishable result

An estimate of the spread of finishing skill, with the number of shots needed before a player's record is more signal than noise: Journal of Quantitative Analysis in Sports or the Sloan conference.

Risks and fallback

Open data cover selected competitions and seasons, so samples per player are small; that is the point of the hierarchy, and a finding that skill is small and hard to detect is itself publishable.

Question

What is the effect of an in-season managerial dismissal on a team's subsequent results, compared with what similar teams that kept their manager went on to do?

Why it is new

Teams improve after sackings, but sackings follow bad runs, and bad runs regress to the mean. Van Ours and van Tuijl (2016) used 14 Dutch seasons and found that teams that kept their coach after similar slumps improved as much. A Bayesian version across several leagues, with team strength modelled as a latent time series and the effect allowed to vary by league and timing, gives an effect estimate with honest uncertainty and a sensitivity analysis.

Data

Match results for many leagues and seasons from the openfootball project (GitHub, as in Section A.6); dismissal dates compiled from public club histories, double-checked against a second source.

Model skeleton

Stage 1: a dynamic version of Section A.6's model, in which each team's attack and defence follow random walks across rounds (Chapter 8). Stage 2: for each dismissal, matched control teams with a comparable slump and comparable latent strength at the same point of the season (Chapter 13); the effect is the difference in posterior expected points over the next ten matches, pooled hierarchically across leagues.

Checks

Placebo "dismissals" at random slump points in teams that did not sack; overlap of the latent strengths of treated and control teams; sensitivity to the length of the outcome window and to unmeasured confounders such as injuries.

Chapters

4 (Poisson models), 6 (pooling across leagues), 8 (latent strength over time), 13 (matching, placebo tests and sensitivity analysis).

Publishable result

A multi-league estimate of the effect of sacking, with its uncertainty: Journal of Sports Analytics or Journal of Quantitative Analysis in Sports; the method transfers to other "intervene after a slump" decisions.

Risks and fallback

Compiling dismissal dates is laborious; start with two leagues. A replication of van Ours and van Tuijl's null finding in other leagues is still a contribution.

Question

The "minimalist" conjecture says that, ordered by size, half of all elliptic curves over the rationals have rank 0 and half rank 1, so the average rank tends to 1/2. In large databases the average is much higher and was long seen to rise (Bektemirov, Mazur, Stein and Watkins, 2007); in a later database of nearly 239 million curves ordered by height, the average rank rises up to a certain height and then eventually decreases, while still being far above 1/2 (Balakrishnan et al., 2016). What do the data say about how fast, if at all, the average approaches 1/2, and how far can they support any extrapolation?

Why it is new

Number theorists reason about these data with plots and heuristics. A Bayesian model of the rank distribution as a function of height, with competing asymptotic forms (convergence to 1/2 at various rates, or to another limit) compared by out-of-sample prediction on the largest heights, puts intervals on the extrapolation and says how much larger a database would need to be to discriminate between them.

Data

Cremona's tables of elliptic curves ordered by conductor (on GitHub, with ranks), the LMFDB, and the height-ordered database of Balakrishnan et al. (2016), whose summary statistics by height range are published with the paper.

Model skeleton

For curves in height bin $b$: counts of ranks 0, 1, 2 and higher are $\text{Multinomial}(n_b,p_b)$, with $p_b$ a smooth function of $\log H$. Root-number parity splits curves into even and odd rank, so model $Pr(\text{rank}\ge 2\mid \text{even})$ and $Pr(\text{rank}\ge 3\mid \text{odd})$ as decaying functions of $\log H$ with priors informed by random-matrix heuristics. Compare forms by leave-future-out validation on the largest heights.

Checks

Posterior predictive checks of the rank proportions by bin; agreement with the 2-Selmer statistics, whose predicted limits (Poonen–Rains, Bhargava–Shankar) give a second, checkable target; prior sensitivity, since the extrapolation is where the prior matters most.

Chapters

2 (priors from heuristics), 4 (multinomial and binomial models), 8 (smooth trends in $\log H$; leave-future-out validation), 9 (fast approximate fitting across many bins), 10 (model comparison).

Publishable result

A quantified statement of the tension between data and conjecture, with intervals, and a calculation of the database size needed to resolve it: Experimental Mathematics or Research in Number Theory.

Risks and fallback

The extrapolation may be uninformative; that is a result if stated quantitatively. The same machinery applies to murmurations (He, Lee, Oliver and Pozdnyakov, 2024), the oscillating averages of Frobenius traces found by machine learning, where a Bayesian functional model could estimate the oscillation's shape with uncertainty.

Question

Coates, Kasprzyk and Veneziale (2023) trained classifiers to recover the dimension of a toric Fano variety from the growth of the coefficients of its quantum period, with about 98% accuracy. For mathematical use, the important question is the other 2%: can a model say which predictions to trust, and does it know when a new variety is unlike its training data?

Why it is new

Machine learning is increasingly used to suggest conjectures, and point predictions without calibrated uncertainty are a weak basis for them. An explicitly Bayesian model of the coefficient asymptotics gives a posterior over the dimension and a principled reason to abstain.

Data

Generate your own: for weighted projective spaces the period sequence has a closed form (a ratio of factorials in the weights), so tens of thousands of examples can be generated in R, with exact dimensions; toric varieties of Picard rank two follow from the formulae in the paper.

Model skeleton

For each variety, the log coefficients $\log c_d$ grow approximately like $Ad+B\log d+C$, with $B$ determined by the dimension. Fit the asymptotic regression with a Bayesian model that includes the curvature at small $d$ (Chapter 3), giving a posterior for $B$; the posterior probability of each dimension follows. Compare with a calibrated classifier (Chapters 10, 12) and with a prior-fitted network trained on simulated sequences (Chapter 14).

Checks

Calibration of the posterior probabilities on held-out varieties; deliberate shift tests (train on small weights, test on large); coverage of intervals for $B$ against the exact value; the accuracy achievable when the model abstains on its least certain 5%.

Chapters

3 (asymptotic regression), 5 (diagnostics), 10 (classification and shrinkage), 12 (calibration), 14 (prior-fitted networks, shortcuts and distribution shift).

Publishable result

Calibrated uncertainty and abstention for a published machine-learned invariant, a general template for "AI-assisted mathematics" claims: Experimental Mathematics or Machine Learning: Science and Technology.

Risks and fallback

Reproducing the data generation takes care; start with weighted projective spaces, where the formula is explicit and the published accuracy is near perfect, so any failure of calibration is easy to see.

Question

How much of the variation in student evaluation of teaching (SET) ratings is attributable to the instructor rather than to the course, the term, the class size and the students' expected grades, and do ratings differ by instructor gender or other characteristics once these are accounted for?

Why it is new

SETs drive hiring and promotion, and are usually summarised as raw means. A crossed multilevel ordinal model separates instructor, course and term variance; a measurement-invariance analysis asks whether the items mean the same thing for different instructors; and linking to a common final exam in multisection courses tests whether ratings track learning (dissection 4).

Data

Your institution's SET responses and grade records, de-identified, with ethical approval and the agreement of the faculty union where required; the published data of Boring, Ottoboni and Stark (2016) for a reanalysis and replication.

Model skeleton

Item responses on a 1–5 scale: a cumulative-logit (ordinal) model with crossed random effects for instructor, course and term, and student-level covariates such as expected grade; a latent factor for "teaching quality" with item loadings allowed to differ by instructor group (Chapter 11); for multisection courses, a second equation linking the instructor effect on ratings to the instructor effect on exam scores (Chapter 13).

Checks

Posterior predictive checks of rating distributions by item; leave-one-term-out prediction; sensitivity to nonresponse (response rates are often low and selective, Chapter 7); identical analyses on a randomised source where possible (MacNell, Driscoll and Hunt, 2015).

Chapters

4 (ordinal models), 6 (crossed random effects), 7 (nonresponse), 11 (factor models and invariance; the Gaussian copula for correlating Likert items with grades and class sizes), 13 (bias as a causal question).

Publishable result

A variance decomposition and bias audit with honest intervals, and policy-relevant guidance on how many evaluations are needed before an instructor's mean is informative: Assessment & Evaluation in Higher Education or Journal of Educational and Behavioral Statistics.

Risks and fallback

Access and ethics take time; start the approval process first. The reanalysis of published data is a complete paper on its own.

Question

Does ten minutes of expressive writing about exam worries just before a mathematics exam improve performance, especially for students with high test anxiety?

Why it is new

Ramirez and Beilock (2011) reported large benefits in a Science paper; a large-scale replication in the Social Sciences Replication Project (Camerer et al., 2018) did not find a significant effect in the same direction. A preregistered, randomised Bayesian replication in university mathematics courses, with a prior built from both, is publishable whatever it finds, and directly useful to instructors.

Data

Collected by you, with ethical approval: students in several sections randomised within section to expressive writing or a neutral writing control before a midterm; a validated test-anxiety questionnaire beforehand, such as the Cognitive Test Anxiety Scale (Cassady and Johnson, 2002); prior grades as a pre-treatment covariate.

Model skeleton

Exam score (standardised) $={\alpha }_{\text{section}}+\beta \text{prior grade}+\tau \text{writing}+\gamma \text{writing}\times \text{anxiety}+\epsilon$, with section effects partially pooled, anxiety measured as a latent factor from the questionnaire items (Chapter 11), and a sceptical prior on $\tau$ informed by the replication literature; heterogeneous effects with BART as a secondary analysis (Chapter 13).

Checks

A design analysis before data collection (Section A.4: the retrodesign function) to choose the number of students; prior sensitivity, reporting the posterior under the original study's effect size, a sceptical prior and a flat prior; missing exam scores handled with the mechanism made explicit (Chapter 7).

Chapters

1 (design and workflow), 2 (priors from earlier studies), 6 (sections), 7 (missing outcomes), 11 (anxiety as a latent factor; a Gaussian copula factor model for the ordinal questionnaire items), 13 (randomisation, heterogeneity, mediation through reported worry).

Publishable result

A registered report in Advances in Methods and Practices in Psychological Science or a replication in an education journal; the anxiety-measurement component connects to the AI-in-mathematics project.

Risks and fallback

Sample size per term may be modest; the Bayesian analysis can accumulate across terms with the protocol fixed in advance.

Question

Reagan et al. (2016) scored the sentiment of 1,327 Project Gutenberg novels along their length and reported that the resulting "emotional arcs" are dominated by six basic shapes, such as rags to riches, tragedy and man in a hole. How many shapes do the data support, and how many books are mixtures rather than members of a type?

Why it is new

The original analysis used singular value decomposition, clustering and self-organising maps, without a likelihood or uncertainty. A Bayesian functional mixture model, with each arc a smooth curve drawn from one of $K$ shape-specific Gaussian processes, chooses $K$ by predictive performance and gives each book a posterior over shapes.

Data

Project Gutenberg texts (public domain); a sentiment lexicon applied to sliding windows, as in the original, plus a modern sentiment model as a check on measurement.

Model skeleton

For book $b$, window sentiment $y_{bt}=f_b(t)+{\epsilon }_{bt}$ on a normalised time axis $t\in [0,1]$; $f_b\sim \mathcal{G}\mathcal{P}(m_{z_b},k)$ given its cluster $z_b$; cluster means $m_k$ smooth curves; $z_b\sim \text{Categorical}(\pi )$. Fit with a basis-function approximation to the GP (Chapter 8) and compare $K=1,\dots ,10$ by cross-validated predictive density (Chapter 11).

Checks

A simulation study with known shapes; posterior predictive checks of arc variability; stability across sentiment lexicons, window lengths and random halves of the corpus; whether the shapes predict anything external, such as downloads or genre.

Chapters

8 (Gaussian processes), 9 (variational inference for speed), 11 (mixtures and choosing $K$), 12 (text as data).

Feasibility check

A first pass takes an hour. Six public-domain novels from Project Gutenberg are split into 50 windows each, every word is scored with the Bing sentiment lexicon of Hu and Liu (2004) distributed with the tidytext package (Silge and Robinson, 2016), and each window's sentiment is the share of positive minus the share of negative scored words. A Bayesian smoothing spline for each book, with its own noise level, gives the arc with its uncertainty.

Rcopy code

library(tidytext)
books <- c(`Pride and Prejudice` = 1342, Frankenstein = 84, Dracula = 345, `Alice in Wonderland` = 11,
           `A Christmas Carol` = 46, `Moby-Dick` = 2701)
read_gut <- function(id) {                                  # the text between Project Gutenberg's start and end markers
  x <- readLines(sprintf("https://www.gutenberg.org/cache/epub/%d/pg%d.txt", id, id), encoding = "UTF-8", warn = FALSE)
  x[(grep("\\*\\*\\* ?START OF", x)[1] + 1):(grep("\\*\\*\\* ?END OF", x)[1] - 1)] }
arcs <- imap_dfr(books, function(id, title) tibble(book = title, text = read_gut(id))) |>
  unnest_tokens(word, text) |> group_by(book) |> mutate(t = (row_number() - 0.5) / n()) |> ungroup() |>
  inner_join(get_sentiments("bing"), by = "word", relationship = "many-to-many") |>
  mutate(window = pmin(floor(t * 50), 49)) |> group_by(book, window) |>
  summarise(t = (first(window) + 0.5) / 50, sentiment = mean(sentiment == "positive") - mean(sentiment == "negative"),
            scored_words = n(), .groups = "drop")
m_arcs <- brm(bf(sentiment ~ book + s(t, by = book, k = 8), sigma ~ book), data = arcs,
              seed = 8, refresh = 0, control = list(adapt_delta = 0.95), file = "fits/aw_arcs")
ends <- expand_grid(book = names(books), t = c(0.05, 0.95))
e <- posterior_epred(m_arcs, newdata = ends)                  # draws x (book, t)
tibble(book = names(books), start = colMeans(e[, c(TRUE, FALSE)]), end = colMeans(e[, c(FALSE, TRUE)]),
       p_ending_happier = colMeans(e[, c(FALSE, TRUE)] > e[, c(TRUE, FALSE)])) |>
  mutate(across(where(is.double), ~ round(.x, 3))) |> as.data.frame()

output

                 book  start    end p_ending_happier
1 Pride and Prejudice  0.191  0.145            0.277
2        Frankenstein  0.148 -0.301            0.000
3             Dracula -0.039 -0.098            0.186
4 Alice in Wonderland  0.006 -0.148            0.089
5   A Christmas Carol -0.044  0.332            0.998
6           Moby-Dick -0.042 -0.136            0.074

Rcopy code

fit_arcs <- expand_grid(book = names(books), t = seq(0.01, 0.99, length.out = 60))
f <- posterior_epred(m_arcs, newdata = fit_arcs)
fit_arcs |> mutate(est = colMeans(f), lo = apply(f, 2, quantile, 0.05), hi = apply(f, 2, quantile, 0.95)) |>
  ggplot(aes(t, est)) + geom_hline(yintercept = 0, colour = harbour[["grid"]]) +
  geom_ribbon(aes(ymin = lo, ymax = hi), fill = harbour[["sky"]], alpha = 0.25) +
  geom_point(data = arcs, aes(y = sentiment), size = 0.7, colour = harbour[["muted"]]) +
  geom_line(colour = harbour[["sky"]], linewidth = 0.9) + facet_wrap(~ book, ncol = 3) +
  scale_x_continuous(breaks = c(0.05, 0.5, 0.95), labels = c("start", "middle", "end")) +
  theme(panel.spacing.x = unit(1.2, "lines")) +
  labs(x = NULL, y = "net sentiment")

The arcs are recognisable but noisy. A Christmas Carol is the clearest rise: from about −0.04 near the start to +0.33 at the end, and the posterior probability that its ending is happier than its opening is 0.998, with a dip and recovery in the middle, close to the "man in a hole" and "Cinderella" shapes. Frankenstein is the clearest fall (+0.15 to −0.30; probability of a happier ending 0.000), a tragedy. The others are much flatter: Pride and Prejudice stays positive throughout, and Dracula, Moby-Dick and Alice in Wonderland drift gently downward, with probabilities of a happier ending between 0.07 and 0.28. Two lessons for the project follow. Most books are not cleanly one of six shapes; and a window of a few dozen to a few hundred sentiment words measures tone with a standard error comparable to the arcs' own rise and fall, so the measurement model (lexicon, window length, the noise level estimated for each book) matters as much as the clustering model. Both are good reasons for the Bayesian treatment the project proposes.

Publishable result

A probabilistic reanalysis of a widely cited claim: Journal of Cultural Analytics or the Computational Humanities Research conference. If the answer is "far fewer than six" or "a continuum rather than types", that is the headline.

Risks and fallback

Sentiment lexicons measure tone imperfectly, especially in older prose; treat measurement as part of the model and report its sensitivity.

Question

When people write stories with help from a language model, or models write them alone, is the collection of stories less diverse in its themes than stories written by people from the same prompts, and which themes are over- and under-represented?

Why it is new

Doshi and Hauser (2024) found in an experiment that access to AI ideas made individual stories rated more creative but made the stories more similar to each other. A Bayesian topic model with explicit diversity measures, prompts as a random effect and the author type as the treatment, can locate where the narrowing happens and how large it is, with intervals.

Data

Human stories written to the same prompts, from the public WritingPrompts collection (Fan, Lewis and Dauphin, 2018) or from Doshi and Hauser's published data; stories generated by several models from the same prompts, with settings recorded; optionally a small writing experiment with students, with ethical approval.

Model skeleton

An LDA or Dirichlet-multinomial topic model (Chapter 12) with document-topic proportions depending on author type and prompt through a hierarchical logistic-normal prior; diversity as the expected entropy of the collection's topic distribution and the expected distance between stories from the same prompt, both as posterior quantities.

Checks

Shortcut checks (Chapter 14): length, formatting and stock phrases differ between human and model text and must not drive the topics; robustness to the number of topics and to the choice of models; human rating of a sample of topics for interpretability.

Chapters

6 (prompts as random effects), 11 (mixtures and latent structure), 12 (topic models), 13 (author type as a treatment in a designed comparison), 14 (shortcuts and invariance).

Publishable result

A quantitative map of thematic narrowing: Journal of Cultural Analytics, an ACL workshop, or a general-interest venue if the effect is large and well identified.

Risks and fallback

Models change quickly; record versions and dates, and frame results as a measurement protocol others can rerun.

Question

Shakespeare's Henry VIII is generally believed to be a collaboration with John Fletcher. Plecháč (2021) used frequent words and rhythmic patterns with a classifier on rolling windows to locate the boundaries between the authors' shares, which need not coincide with scene boundaries. Can a Bayesian model give each passage a posterior probability of authorship, find the change points with intervals, and say where the evidence is genuinely ambiguous?

Why it is new

Stylometric attributions usually report classifier scores without calibrated probabilities. A hidden Markov model, whose hidden state is the author and whose emissions are Dirichlet-multinomial counts of function words and metrical features, gives exactly that, and extends the Bayesian authorship tradition of Mosteller and Wallace (Chapter 12).

Data

Public-domain texts of the plays of both authors (for training) and of Henry VIII (and other suspected collaborations, such as The Two Noble Kinsmen), from Project Gutenberg or the Folger editions.

Model skeleton

Texts divided into windows of fixed length; author-specific Dirichlet-multinomial distributions over the most frequent words, with hierarchical priors pooling plays by the same author; a two-state hidden Markov model along the play with a sticky transition matrix, so that authorship changes rarely; posterior probabilities from the forward–backward algorithm, inside Stan.

Checks

Leave-one-play-out validation on single-author plays, including artificial "collaborations" spliced from two known plays, so that the change-point accuracy is measured against known truth; calibration of the posterior probabilities; invariance to genre and date (Chapter 14), since late plays differ from early ones for reasons other than authorship.

Chapters

8 (hidden Markov and change-point models), 10 (shrinkage over many words), 11 (latent states), 12 (Dirichlet-multinomial text models, calibration), 14 (invariance to genre).

Publishable result

Calibrated scene-by-scene attribution with uncertainty, validated on spliced texts: Digital Scholarship in the Humanities or the Computational Humanities Research conference.

Risks and fallback

Revision and collaboration may be finer-grained than any model can resolve; the spliced-text validation shows the resolution limit, which is itself a useful, publishable result.