Chapter 0
Before we start

Not lectured · how these notes work, the calculus you'll use, and your first hour with R

what you need before the course starts

This chapter is the ground the course stands on. We won't spend class time on it. Read it once at the start of the semester, and come back whenever something later on feels shaky: a derivative you've forgotten how to take, an R error you don't understand, a box in the notes you're not sure how to read.

0.1 What this course is about

Every day, people and machines make decisions without knowing what will happen next:

an airline decides how many tickets to sell;
a hospital decides how many nurses to schedule;
your phone decides which word you're about to type;
a forensic analyst decides whether two glass fragments could have come from the same window.

None of them can see the future. All of them use patterns in past data to make a good guess, and all of them have to live with being wrong some of the time.

This course is about doing that well. It has two big ideas.

Prediction. Using what we have seen to make a good guess about what we haven't, with errors small enough to live with.
Inference. Using data to learn about the mechanism that produced it: how strong a drug is, how often a test is wrong, whether a policy changed anything. Knowing the mechanism tells us how the world behaved in the past and how it will behave in the future.

The language for both is probability, so a large part of the course builds that language: events, random variables, distributions. Then we use it to estimate unknown quantities, say how sure we are, and test claims.

By the end of the course you should be able to:

choose a sensible probability model for a real situation, and say what has to be true for it to work;
describe and visualize data honestly, in R;
compute and interpret probabilities, expected values and standard deviations;
estimate a quantity from data, attach a margin of error, and test a claim, explaining in plain words what your answer means;
recognize when a pattern in data does, and does not, tell you about cause and effect.

The examples come from fields where these ideas matter every day: forensic science, toxicology, criminal justice, computer science and machine learning, and experimental science in general. Wherever we can, they use real data, with the source given so you can check it yourself.

Think About It: Prediction or Inference?

For each question, decide whether it is mainly about prediction (a good guess about a particular unseen case) or inference (learning about the mechanism that produces the data). (a) Will it rain in Manhattan tomorrow? (b) Does a new breath-test device read higher, on average, than the old one? (c) Which of this week's emails are spam? (d) Does adding street lighting reduce burglaries?

(a) Prediction: one unseen outcome, and a forecaster is judged by how often she's right, not by why. (b) Inference: the question is about the device's systematic behaviour, not any one reading. (c) Prediction: a spam filter only has to sort emails correctly; it can do that using patterns (such as the word "winner") without any theory of why spammers write what they write. (d) Inference, and a causal one: we want to know what lighting does, which is harder than noticing that well-lit streets have fewer burglaries (they may also be richer streets). The two goals often blend: knowing the mechanism usually improves prediction, and a good prediction rule can hint at a mechanism. But they need different kinds of evidence, and this course keeps asking which one a question is after.

0.2 How these notes work

Each chapter starts with a cartoon and a short motivation, then builds ideas in the same order: a question we care about, intuition and pictures, then precise definitions and results, then worked examples.

The boxes

Different kinds of content live in differently coloured boxes, and the colour tells you at a glance what kind of reading you are about to do.

colour	boxes	what to do with them
blue	definitions, theorems	read slowly; every word matters
green	examples, check yourself, exercises	work through them actively
orange	key takeaways	remember these
red	watch out	common mistakes to avoid
purple	interludes	optional, for the curious
teal, pink, sand, grey	causal inference, Bayesian inference, history, think about it	the threads that run through the course

Here is one of each.

Definition: What a Definition Looks Like

Definitions are stated precisely, because later arguments depend on exactly what they say. Read every word. Bold terms are the ones being defined. The header always names the thing being defined, so you can find it again quickly: for example "Definition: Random Variable".

Theorem: What a Theorem Looks Like

Theorems are results that follow from the definitions. They are stated precisely too. Most have a proof hidden behind a button: try to see why the result is true before you open it.

A proof is the argument that the theorem is true. You won't be examined on reproducing proofs, but reading them is the best way to understand what a theorem really says.∎

Example: What a Worked Example Looks Like ★ lecture example

Numbered examples are worked in full. Examples marked ★ lecture example are the ones we go through together in class. The others are for practice: try them yourself first, then open the solution.

Solutions are hidden until you click. The button at the top of each chapter opens all of them at once (handy for revision).

Check Yourself

Check yourself boxes are quick questions to test whether an idea has sunk in. Choose or type an answer and press check. A correct answer turns green and explains why; a wrong one turns amber and gives you a hint, not the answer. After two wrong tries you can reveal the answer, but try first. Have a go:

What is the best way to use a check-yourself question? Reveal the answer straight away and read it.Skip it if the section made sense.Commit to an answer before checking, and use the hint if you're wrong.

Watch Out: A Common Mistake

Watch out boxes flag the mistakes students most often make with an idea, and explain why they're mistakes. If you've made one of them, you're in good company; the box is there so you only make it once.

Think About It: A Question to Chew on

Think about it boxes ask an open question with no single right answer, or a question that makes you apply an idea somewhere new. Stop and try to answer before reading on. Most have a show discussion button with some thoughts to compare with your own.

Like this one. The point of these boxes isn't the answer; it's the habit of asking "is that really true?", "how would I know?", and "what would change my mind?" every time you meet a claim backed by data.

Causal Inference: The Causal Viewpoint

Causal inference boxes ask: did one thing actually cause another? Most of statistics is about association. These boxes show how to think about cause, usually by drawing a diagram of what influences what. They form a running thread through the whole course.

Bayesian Inference: The Bayesian Viewpoint

Bayesian inference boxes show the same question answered from the other major school of statistics, where probability describes our degree of belief and data update it. The main text teaches the classical ("frequentist") approach; these boxes show you the alternative.

Interlude (Optional): A Deeper Idea

Interludes are short, completely optional excursions into deeper or more modern ideas: measure theory, information theory, machine learning, the philosophy of science. They are never examined. Read them if you're curious.

History: Where Ideas Come from

History boxes tell the story of where an idea came from and who argued about it.

Some of that story is uncomfortable. Several of the people who built modern statistics, including Francis Galton, Karl Pearson and Ronald Fisher, were leading supporters of eugenics: the movement to "improve" the human population by controlling who had children. They developed some of their most famous tools while studying heredity for that purpose. These notes don't hide that history.

The tools themselves are among the most valuable ideas in science. Correlation, regression, the standard deviation and significance tests are now used to test vaccines, to free wrongly convicted people and to detect discrimination. The same tools were once used to give racist and classist policies a look of scientific authority, and they can be misused in the same way today. That is one of the best reasons to learn statistics well: people who understand how a number was produced are much harder to mislead with it.

One small change follows from this history. These notes call the bell-shaped distribution the Gaussian distribution, not by its more common name, the normal distribution; Section 5.3 explains why. Every other term is the standard one. Where a standard name has a troubling or confusing origin, a history box says so.

Key Takeaways

Takeaways close each section with what you should remember.

Exercises

Each chapter ends with exercises like this one.

A sample exercise. What is $2 + 2$ ?

The show hint button gives a nudge in the right direction without giving the game away. Try the exercise again after reading it.

The show answer button gives the final answer, and often a sentence of explanation, so that you can check your own work. Here: $4$ .

R code

R code appears in blocks like the one below, with the real output underneath, produced by running the code. Press copy code to copy it, then paste it into RStudio and run it yourself.

2 + 2
sqrt(16)

output

[1] 4
[1] 4

The plots in these notes are produced by R and then restyled to match the notes, so they follow the light or dark theme. Your plots will look like R's default style, which is fine.

Widgets

Many sections contain interactive widgets: sliders, buttons, simulations. They are there to make you think: change something, predict what will happen, then check. Every widget works with a keyboard (Tab to move, arrow keys for sliders, Enter or Space for buttons), and has a text read-out for screen readers. The button in the top corner switches between dark and light themes.

How to study from these notes

Read the section before the lecture. Don't worry if it doesn't all make sense; you'll know what to listen for.
Try examples before opening the solutions. Struggling for five minutes is how learning happens. Reading a solution feels like understanding but usually isn't.
Run the R code yourself. Change a number and predict the result before you press Enter.
Use the widgets actively. Ask "what happens if…", then check.
Do the check-yourself questions. They are the fastest way to find out what you don't know yet.

0.3 How the course runs

The course has nine chapters, roughly two to four lectures each:

Chapter	What it's about
1 Prediction, inference and data	why statistics exists; samples, variables, where data come from
2 Describing data	summaries and pictures of data
3 Probability	the language of uncertainty; conditional probability and Bayes' theorem
4 Discrete random variables	counts: binomial, Poisson and friends
5 Continuous random variables	measurements: Gaussian, exponential and friends
6 Joint distributions	two or more variables at once; correlation
7 Sampling distributions	how statistics vary from sample to sample; the Central Limit Theorem
8 Estimation	best guesses, maximum likelihood, confidence intervals
9 Hypothesis testing	deciding between claims, and the errors we can make

Assessments

The notes are designed for a one-semester course. A typical plan of assessments looks like this:

Assessment	Covers
Homework 1	Chapters 1–2
Homework 2	Chapter 3
Exam 1	Chapters 1–3
Homework 3	Chapter 4
Homework 4	Chapter 5
Exam 2	Chapters 4–5
Homework 5	Chapters 6–7
Homework 6	Chapter 8 (and 9)
Exam 3	Chapters 6–8
Final exam	everything
Capstone case study	everything, applied to real data

If you are taking a course that uses these notes, your syllabus gives the dates and grade weights.

Homework mixes calculation by hand with short R tasks and questions that ask you to interpret results in words. You'll be asked to explain, not just compute.

Exams are taken in class without a computer. They may show you R code and output and ask what it means, or ask what code you would write. A formula sheet is provided.

The capstone case study is a short report on a real, public dataset that you choose from a list. It brings the whole course together. Your report will:

state a question and say whether it is about prediction, inference, or cause and effect;
explain how the data were collected, and draw a diagram of what might influence what;
describe the data with summaries and plots in R;
estimate one quantity with a confidence interval, test one claim, and discuss honestly what the data can't tell you.

Using AI tools

You may use AI tools to help you study, as the syllabus allows. You must be able to explain and justify everything you submit, and solve problems without AI in exams. An AI can produce a confident, fluent wrong answer about statistics, and this course will teach you how to catch it.

0.4 What you should know from calculus

This is a calculus-based course, but the calculus we use is a small, friendly part of it. Here is what you need, why you need it, and a worked example of each. If anything feels rusty, do the check-yourself questions at the end of this section before Chapter 3.

Exponentials and logarithms

The exponential function $e^{x}$ and the natural logarithm $ln x$ are inverses: $e^{ln x} = x$ for $x > 0$ , and $ln (e^{x}) = x$ for every $x$ .

Theorem: Rules of Exponentials and Logarithms

For $a, b > 0$ and any real $r$ , $s$ : $\begin{matrix} e^{r + s} & = e^{r} e^{s}, & ln (a b) & = ln a + ln b, \\ ln (\frac{a}{b}) & = ln a - ln b, & ln (a^{r}) & = r ln a . \end{matrix}$ $e^{r + s}$ $= e^{r} e^{s},$ $ln (a b)$ $= ln a$ $+ ln b,$ $ln (\frac{a}{b})$ $= ln a$ $- ln b,$ $ln (a^{r})$ $= r ln a .$

Where it shows up. The logarithm turns products into sums. In Chapter 8, the probability of a whole dataset is a product of many small probabilities. Taking logs turns it into a sum, which is far easier to work with.

Example 0.1: Logs Turn Products Into Sums

Simplify $ln (p^{3} {(1 - p)}^{7})$ for $0 < p < 1$ .

The expression inside the log is a product of two factors, $p^{3}$ and ${(1 - p)}^{7}$ , so the product rule splits it into a sum of two logs. Then the power rule brings each exponent down in front: $\begin{matrix} ln (p^{3} {(1 - p)}^{7}) & = ln (p^{3}) + ln ({(1 - p)}^{7}) & (log of a product) \\ = 3 ln p + 7 ln (1 - p) . & (log of a power) \end{matrix}$ $ln (p^{3} {(1 - p)}^{7})$ $= ln (p^{3})$ $+ ln ({(1 - p)}^{7})$ $(log of a product)$ $= 3 ln p$ $+ 7 ln (1$ $- p) .$ $(log of a power)$ Check with numbers: at $p = 0.5$ , the left side is $ln ({0.5}^{10})$ $= ln (0.000977)$ $\approx - 6.93$ , and the right side is $10 ln 0.5$ $\approx 10$ $\times (- 0.693)$ $= - 6.93$ . In Chapter 8 this expression is the log-likelihood for a dataset with 3 successes and 7 failures. Without logs we would have to differentiate a product of powers; with logs, each piece is differentiated on its own.

Sums

We write long sums with sigma notation: $\sum_{i = 1}^{n} x_{i}$ $= x_{1}$ $+ x_{2}$ $+ \dots$ $+ x_{n}$ . Read it as "add up $x_{i}$ as $i$ runs from 1 to $n$ ". Two rules do most of the work: $\begin{matrix} \sum_{i = 1}^{n} (a x_{i} + b) & = a \sum_{i = 1}^{n} x_{i} + n b, \\ \sum_{i = 1}^{n} (x_{i} + y_{i}) & = \sum_{i = 1}^{n} x_{i} + \sum_{i = 1}^{n} y_{i} . \end{matrix}$ The first rule says constants can be pulled out of a sum, and that adding $b$ to each of $n$ terms adds $n b$ in total. For example, with $x_{1},$ $x_{2},$ $x_{3}$ $= 1,$ $2,$ $3$ , we get $\sum_{i = 1}^{3} (2 x_{i}$ $+ 5)$ $= 7$ $+ 9$ $+ 11$ $= 27$ , and the rule gives the same answer, $2$ $\times 6$ $+ 3$ $\times 5$ $= 27$ . We will use these rules in Chapter 2 to show how the average changes when every value is rescaled, for example converting temperatures from °C to °F.

Two infinite sums appear again and again.

Theorem: Two Series You Will Meet Often

Geometric series. If $| r | < 1$ , then $\sum_{k = 0}^{\infty} r^{k}$ $= 1$ $+ r$ $+ r^{2}$ $+ \dots$ $= \frac{1}{1 - r}$ .
Exponential series. For every real $λ$ , $\sum_{k = 0}^{\infty} \frac{λ^{k}}{k!}$ $= 1$ $+ λ$ $+ \frac{λ^{2}}{2!}$ $+ \dots$ $= e^{λ}$ .

For the geometric series, let $S_{n}$ $= 1$ $+ r$ $+ \dots$ $+ r^{n}$ . Then $r S_{n}$ $= r$ $+ r^{2}$ $+ \dots$ $+ r^{n + 1}$ , and subtracting, almost everything cancels: $S_{n}$ $- r S_{n}$ $= 1$ $- r^{n + 1}$ , so $S_{n} = \frac{1 - r^{n + 1}}{1 - r} .$ If $| r | < 1$ then $r^{n + 1} \to 0$ as $n \to \infty$ , so $S_{n} \to \frac{1}{1 - r}$ .

The exponential series is the Taylor series of $e^{λ}$ at 0: every derivative of $e^{x}$ is $e^{x}$ , which equals 1 at $x = 0$ . Taylor's theorem shows the series converges to $e^{λ}$ for every $λ$ .∎

Where it shows up. Probabilities of all possible outcomes must add up to 1. To check that for distributions with infinitely many outcomes (Chapter 4), we need these two series.

Example 0.2: Probabilities That Add to One

In Chapter 4 we'll model the number of calls to a 911 dispatch centre in a minute by saying the chance of exactly $k$ calls is $e^{- λ} λ^{k} / k!$ for $k = 0, 1, 2, \dots$ . Show that these chances add up to 1.

The factor $e^{- λ}$ doesn't depend on $k$ , so it comes out of the sum (the first sum rule, with $a = e^{- λ}$ ), and what's left is the exponential series: $\begin{matrix} \sum_{k = 0}^{\infty} \frac{e^{- λ} λ^{k}}{k!} & = e^{- λ} \sum_{k = 0}^{\infty} \frac{λ^{k}}{k!} & (pull out the constant) \\ = e^{- λ} e^{λ} & (exponential series) \\ = 1 . \end{matrix}$ $\sum_{k = 0}^{\infty} \frac{e^{- λ} λ^{k}}{k!}$ $= e^{- λ} \sum_{k = 0}^{\infty} \frac{λ^{k}}{k!}$ $(pull out the constant)$ $= e^{- λ} e^{λ}$ $(exponential series)$ $= 1 .$ A numerical check helps make this feel real. With $λ = 2$ , the first few terms are $e^{- 2} (1$ $+ 2$ $+ 2$ $+ 1.333$ $+ 0.667$ $+ \dots)$ $\approx 0.135$ $+ 0.271$ $+ 0.271$ $+ 0.180$ $+ 0.090$ $+ \dots$ , and they already add to about 0.95 after five terms. The remaining terms make up the last 0.05.

Derivatives and finding the best value

You need the usual rules (power, product, quotient, chain) and the derivatives of $x^{n}$ , $e^{x}$ and $ln x$ : $\frac{d}{d x} x^{n}$ $= n x^{n - 1},$ $\frac{d}{d x} e^{a x}$ $= a e^{a x},$ $\frac{d}{d x} ln x$ $= \frac{1}{x} .$

Where it shows up. In statistics we constantly look for the best value of something: the guess that makes the data most probable (Chapter 8), or the decision with the smallest expected cost. To find the highest point of a smooth curve, set its derivative to zero.

Example 0.3: Finding the Best Value

For $0 < p < 1$ , find the value of $p$ that maximizes $L (p)$ $= p^{3} {(1 - p)}^{7}$ .

Step 1: take logs. Maximizing $L (p)$ is the same as maximizing $ln L (p)$ , because $ln$ is an increasing function: whichever $p$ makes $L$ biggest also makes $ln L$ biggest. The log is much easier to differentiate (Example 0.1): $ℓ (p)$ $= ln L (p)$ $= 3 ln p$ $+ 7 ln (1$ $- p) .$

Step 2: differentiate. The derivative of $ln p$ is $1 / p$ . For $ln (1 - p)$ , the chain rule gives $\frac{1}{1 - p} \times (- 1)$ . So $ℓ' (p) = \frac{3}{p} - \frac{7}{1 - p} .$

Step 3: set the derivative to zero and solve. $\begin{matrix} \frac{3}{p} - \frac{7}{1 - p} = 0 & \Rightarrow \frac{3}{p} = \frac{7}{1 - p} \\ \Rightarrow 3 (1 - p) = 7 p & (cross-multiply) \\ \Rightarrow 3 = 10 p \\ \Rightarrow p = \frac{3}{10} . \end{matrix}$ $\frac{3}{p}$ $- \frac{7}{1 - p}$ $= 0$ $\Rightarrow$ $\frac{3}{p}$ $= \frac{7}{1 - p}$ $\Rightarrow$ $3 (1$ $- p)$ $= 7 p$ $(cross-multiply)$ $\Rightarrow$ $3$ $= 10 p$ $\Rightarrow$ $p$ $= \frac{3}{10} .$

Step 4: check it's a maximum, not a minimum. Differentiate again: $ℓ ″ (p) = - \frac{3}{p^{2}} - \frac{7}{{(1 - p)}^{2}} .$ Both terms are negative for every $p$ between 0 and 1, so the curve bends downwards everywhere, and the flat point must be a peak.

A picture confirms it:

p <- seq(0, 1, by = 0.001)
L <- p^3 * (1 - p)^7
ggplot(data.frame(p, L), aes(x = p, y = L)) +
  geom_line() +
  geom_vline(xintercept = 0.3, linetype = "dashed") +
  labs(x = "p", y = "L(p)")

So the best value is $p = 0.3$ : exactly the fraction of successes in the data, 3 out of 10. That is no coincidence, as Chapter 8 will show. Try the widget below with other numbers of successes and failures.

The top of a hill has slope zero

The curve is ℓ(p) = a ln p + b ln(1 − p) from Example 0.3, for a dataset with a successes and b failures. The straight line touches the curve at the point you choose; its steepness is the derivative ℓ′(p). Slide p along the curve. Where the line is uphill (positive slope) the top is further right; where it is downhill the top is to the left. The highest point is exactly where the line goes flat.

touching point p =

successes a = failures b =

Integrals and areas

The integral $\int_{a}^{b} f (x) d x$ is the area under the curve $y = f (x)$ between $a$ and $b$ (counting area below the axis as negative). The Fundamental Theorem of Calculus says that if $F' (x) = f (x)$ , then $\int_{a}^{b} f (x) d x = F (b) - F (a) .$

Where it shows up. For measurements like time, weight or concentration, probabilities are areas under a curve (Chapter 5). The probability that a measurement lands between $a$ and $b$ is $\int_{a}^{b} f (x) d x$ .

An integral is a sum of thin rectangles

Pick a curve and an interval. The integral is the area under the curve, and we approximate it with rectangles. Add more rectangles and the approximation converges to the exact value. In this course, areas under curves will be probabilities: the first two curves are probability densities you'll meet in Chapter 5, so their total area is exactly 1.

curve from a = to b = rectangles =

You need three techniques: antiderivatives of the basic functions, substitution, and integration by parts, $\int u d v = u v - \int v d u,$ plus improper integrals, where a limit is infinite: $\int_{0}^{\infty} f (x) d x$ $= {lim}_{b \to \infty} \int_{0}^{b} f (x) d x$ .

Example 0.4: A Total Area of One

In Chapter 5, the time (in hours) until a lab instrument fails is modelled by the curve $f (x) = λ e^{- λ x}$ for $x \geq 0$ , where $λ > 0$ is a constant. Show that the total area under the curve is 1.

Find an antiderivative. An antiderivative of $λ e^{- λ x}$ is $- e^{- λ x}$ . Check by differentiating: the chain rule gives $\frac{d}{d x} (- e^{- λ x})$ $= - e^{- λ x}$ $\times (- λ)$ $= λ e^{- λ x}$ .

Handle the infinite upper limit. An integral up to $\infty$ means: integrate up to some large number $b$ , then let $b$ grow without bound. $\begin{matrix} \int_{0}^{\infty} λ e^{- λ x} d x & = lim_{b \to \infty} [- e^{- λ x}]_{0}^{b} \\ = lim_{b \to \infty} (- e^{- λ b} - (- e^{0})) \\ = lim_{b \to \infty} (1 - e^{- λ b}) = 1, \end{matrix}$ $\int_{0}^{\infty} λ e^{- λ x} d x$ $= lim_{b \to \infty} [- e^{- λ x}]_{0}^{b}$ $= lim_{b \to \infty} (- e^{- λ b}$ $- (- e^{0}))$ $= lim_{b \to \infty} (1$ $- e^{- λ b})$ $= 1,$ because $e^{- λ b} \to 0$ as $b \to \infty$ (for $λ > 0$ ). Notice that the answer doesn't depend on $λ$ : whatever the failure rate, the total area is 1. That is what makes this curve a valid probability model, as Chapter 5 will explain. In the widget above, choose the exponential curve and move the to b slider all the way to the right to watch the area approach 1.

Example 0.5: Integration by Parts

Compute $\int_{0}^{\infty} x λ e^{- λ x} d x$ . (In Chapter 5 this will be the average time until failure.)

Choose the parts. Integration by parts helps when the integrand is a product of something that gets simpler when differentiated ( $x$ ) and something easy to integrate ( $λ e^{- λ x}$ ). So take $u$ $= x,$ $d v$ $= λ e^{- λ x} d x,$ $so$ $d u$ $= d x,$ $v$ $= - e^{- λ x} .$

Apply the formula $\int u d v$ $= u v$ $- \int v d u$ : $\begin{matrix} \int_{0}^{\infty} x λ e^{- λ x} d x & = [- x e^{- λ x}]_{0}^{\infty} - \int_{0}^{\infty} (- e^{- λ x}) d x \\ = [- x e^{- λ x}]_{0}^{\infty} + \int_{0}^{\infty} e^{- λ x} d x . \end{matrix}$ $\int_{0}^{\infty} x λ e^{- λ x} d x$ $= [- x e^{- λ x}]_{0}^{\infty}$ $- \int_{0}^{\infty} (- e^{- λ x}) d x$ $= [- x e^{- λ x}]_{0}^{\infty}$ $+ \int_{0}^{\infty} e^{- λ x} d x .$

Evaluate each piece. The first term is 0: it is 0 at $x = 0$ , and $x e^{- λ x} \to 0$ as $x \to \infty$ because the exponential shrinks faster than $x$ grows. The second integral is Example 0.4 divided by $λ$ : $\int_{0}^{\infty} e^{- λ x} d x$ $= \frac{1}{λ} \int_{0}^{\infty} λ e^{- λ x} d x$ $= \frac{1}{λ}$ $\times 1$ $= \frac{1}{λ} .$ So the answer is $\frac{1}{λ}$ . If an instrument fails at rate $λ = 0.01$ per hour, it lasts $1 / 0.01 = 100$ hours on average. That makes sense: a lower failure rate means a longer average lifetime.

Double integrals

When two measurements are uncertain at once (Chapter 6), probabilities become volumes under a surface, and we integrate twice: once in each variable, treating the other as a constant.

Example 0.6: Integrating Twice

Compute $\int_{0}^{1} \int_{0}^{1} (x$ $+ y) d x d y$ .

Work from the inside out. The inner integral is in $x$ , so treat $y$ as a fixed number, just like a constant $c$ in $\int (x + c) d x$ : $\begin{matrix} \int_{0}^{1} (x + y) d x & = {[\frac{x^{2}}{2} + x y]}_{x = 0}^{x = 1} \\ = (\frac{1}{2} + y) - (0 + 0) = \frac{1}{2} + y . \end{matrix}$ $\int_{0}^{1} (x$ $+ y) d x$ $= {[\frac{x^{2}}{2} + x y]}_{x = 0}^{x = 1}$ $= (\frac{1}{2} + y)$ $- (0$ $+ 0)$ $= \frac{1}{2}$ $+ y .$ The answer depends on $y$ , as it should: we have integrated out $x$ only.

The outer integral is now an ordinary integral in $y$ : $\int_{0}^{1} (\frac{1}{2} + y) d y$ $= {[\frac{y}{2} + \frac{y^{2}}{2}]}_{0}^{1}$ $= \frac{1}{2}$ $+ \frac{1}{2}$ $= 1 .$ Geometrically, this is the volume under the tilted roof $z = x + y$ above the unit square. The roof's height ranges from 0 (at one corner) to 2 (at the opposite corner), and its average height is 1, so a volume of 1 makes sense.

Check Yourself

Test yourself on the whole section.

Simplify

ln (e^{- 2 λ} λ^{5})

- 2 λ \cdot 5 ln λ

- 2 λ + 5 ln λ

- 2 + 5 λ

e^{- 2 λ} + 5 ln λ

Find the value of $λ > 0$ that maximizes $e^{- 2 λ} λ^{5}$ .

λ =

Compute $\sum_{k = 0}^{\infty} {(1 / 3)}^{k}$ .

sum =

Compute $\int_{0}^{2} \frac{x}{2} d x$ .

integral =

Think About It: Why Bother with Calculus at All?

Many statistics courses use no calculus. Why might a course that wants you to understand where statistical methods come from need it? Think about Example 0.3: what would you have to do to find the best $p$ without the derivative?

Without calculus you could only try values of $p$ one by one (0.1, 0.2, 0.3, ...) and look for the biggest $L (p)$ . That works for one number, but not for proving that the answer is always the fraction of successes, whatever the data. Calculus turns "try lots of values" into a general formula, and a general formula is what lets us prove that a method works in every case, not just the ones we happened to try.

Key Takeaways

Logs turn products into sums; we'll use this to handle the probability of a whole dataset.
The geometric and exponential series let us add up infinitely many probabilities.
Setting a derivative to zero finds the best value of something; we'll use it to find best estimates.
For continuous measurements, probabilities are areas under curves, computed with integrals.

0.5 A first session with R and RStudio

R is a free programming language built for statistics. It is used in forensic laboratories, pharmaceutical companies, public-health agencies, banks and tech companies, and most new statistical methods appear in R first. RStudio is a free program that makes working with R much more comfortable. We use R for every calculation in this course: no statistical tables.

Getting set up

Install R from the Comprehensive R Archive Network, cran.r-project.org. Choose your operating system and the latest release.
Install RStudio Desktop from posit.co/download/rstudio-desktop. Install R first.
No laptop, or can't install software? Use Posit Cloud, which runs RStudio in a web browser. It has a free plan with a monthly usage limit; check its plans page for the current allowance.

When you open RStudio you'll see four panes:

the Console (bottom left), where code runs and results appear;
the Source editor (top left), where you write scripts: files of code you can save and rerun;
the Environment (top right), which lists the objects you have created;
Files / Plots / Help (bottom right).

Write your code in a script (File → New File → R Script), not straight into the console. Run the current line with Ctrl+Enter (Cmd+Enter on a Mac). A saved script is a record of exactly what you did, which anyone (including you, in three weeks) can rerun.

R as a calculator

3 * (4 + 2)       # arithmetic
2^10              # powers
exp(1)            # the number e
log(100)          # natural log
log10(100)        # log base 10
choose(5, 2)      # number of ways to choose 2 things from 5

output

[1] 18
[1] 1024
[1] 2.718282
[1] 4.60517
[1] 2
[1] 10

Anything after # is a comment: R ignores it. Use comments to explain what your code does.

Objects and vectors

You store a value by giving it a name with the assignment arrow <-. The most important kind of object is a vector: an ordered list of values, created with c() ("combine"). Most R functions work on a whole vector at once.

# toxin concentrations (ng/mL) in five samples
conc <- c(12.1, 9.8, 15.3, 11.0, 10.4)
conc
length(conc)
mean(conc)
sort(conc)
conc * 1000        # convert to pg/mL: every value at once
conc > 11          # which samples exceed 11 ng/mL?
sum(conc > 11)     # how many? (TRUE counts as 1)

output

[1] 12.1  9.8 15.3 11.0 10.4
[1] 5
[1] 11.72
[1]  9.8 10.4 11.0 12.1 15.3
[1] 12100  9800 15300 11000 10400
[1]  TRUE FALSE  TRUE FALSE FALSE
[1] 2

The last line is a trick you'll use all semester: comparisons give TRUE/FALSE, and summing them counts the TRUEs.

Functions and help

A function takes inputs (called arguments) inside parentheses and returns an output: mean(conc). Arguments can be named: round(3.14159, digits = 2). To read the help page of any function, type ? and its name in the console, for example ?mean. Help pages are terse, but the Examples section at the bottom is always worth running.

Data frames

Real data come as tables: one row per individual, one column per measurement. In R this is a data frame. R comes with many real datasets built in. Here is one we'll use in Chapter 1: airquality, daily ozone and weather measurements in New York City from May to September 1973.

head(airquality, 4)    # the first 4 rows
dim(airquality)        # number of rows and columns
str(airquality)        # each column and its type

output

  Ozone Solar.R Wind Temp Month Day
1    41     190  7.4   67     5   1
2    36     118  8.0   72     5   2
3    12     149 12.6   74     5   3
4    18     313 11.5   62     5   4
[1] 153   6
'data.frame': 153 obs. of  6 variables:
 $ Ozone  : int  41 36 12 18 NA 28 23 19 8 NA ...
 $ Solar.R: int  190 118 149 313 NA NA 299 99 19 194 ...
 $ Wind   : num  7.4 8 12.6 11.5 14.3 14.9 8.6 13.8 20.1 8.6 ...
 $ Temp   : int  67 72 74 62 56 66 65 59 61 69 ...
 $ Month  : int  5 5 5 5 5 5 5 5 5 5 ...
 $ Day    : int  1 2 3 4 5 6 7 8 9 10 ...

NA means not available: a missing value. Ozone wasn't recorded on some days. Use $ to pull out one column, and square brackets to pick rows:

temp <- airquality$Temp          # one column, as a vector
mean(temp)
summary(airquality$Ozone)   # quartiles, mean, missing
mean(airquality$Ozone, na.rm = TRUE)   # ignore missing values
hot <- airquality[airquality$Temp > 90, ]   # rows: Temp > 90
nrow(hot)

output

[1] 77.88235
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
   1.00   18.00   31.50   42.13   63.25  168.00      37 
[1] 42.12931
[1] 14

Notice that mean() of a column containing NA would give NA: R refuses to guess. The argument na.rm = TRUE tells it to remove missing values first.

Think About It: Is It Safe to Ignore the Missing Days?

37 of the 153 days have no ozone reading. na.rm = TRUE simply drops them. When might that give a misleading average? Imagine why a reading might be missing.

Dropping missing values is harmless only if the missing days are like the others. If the monitor tended to fail on very hot days, or was switched off for maintenance at weekends when traffic is low, the remaining days would not be typical, and the average would be biased. This is the same question as "who ends up in the sample?" from Chapter 1: always ask why data are missing before deciding what to do about it.

Reading your own data

Most datasets you meet will be CSV files (comma-separated values), which any spreadsheet program can export. Read one with

mydata <- read.csv("lab_results.csv")

R looks for the file in its working directory. The easiest way to avoid "file not found" errors is to use an RStudio project (File → New Project) and keep your data in the project folder.

Packages and a first plot

Packages add functions to R. Install a package once with install.packages("name"), then load it in every session with library(name). The most popular plotting package is ggplot2.

install.packages("ggplot2")   # once per computer

library(ggplot2)
ggplot(airquality, aes(x = Ozone)) +
  geom_histogram(binwidth = 10) +
  labs(x = "ozone (parts per billion)", y = "number of days")

output

Warning message:
Removed 37 rows containing non-finite values (`stat_bin()`).

R warns that it removed the 37 days with missing ozone; that's expected. (The base-R function hist(airquality$Ozone) draws a similar plot without any package.) Chapter 2 explains how to read a histogram. For now, notice that most days had modest ozone, but a few had much more.

When things go wrong

Everyone gets errors. Most are one of these:

Error message	Usual cause
`object 'x' not found`	a typo, wrong capitalization (R is case-sensitive), or you haven't yet run the line that creates `x`
`could not find function "ggplot"`	the package isn't loaded: run `library(ggplot2)`
`unexpected ')'` or `unexpected symbol`	a missing or extra bracket, comma or quote
`cannot open file 'data.csv'`	the file isn't in the working directory
the result is `NA`	missing values in the data: add `na.rm = TRUE`

When you're stuck: read the error message slowly, check the line before the one that failed, and search the exact error message online.

Check Yourself

Use R and the airquality data to answer these. Type the numbers R gives you.

What was the highest temperature recorded?

highest temperature =°F

On how many days was ozone above 100 parts per billion?

number of days =

Key Takeaways

Write code in a script, and run lines with Ctrl+Enter.
<- stores a value; c() makes a vector; most functions work on whole vectors.
A data frame has one row per individual and one column per variable; $ picks a column.
NA is a missing value; many functions need na.rm = TRUE.
Install a package once, then library() it every session.

0.6 Where else to learn

These notes are self-contained, but everyone learns better from more than one explanation. Everything in the first three lists is free.

Free textbooks

OpenIntro Statistics, D. Diez, M. Çetinkaya-Rundel and C. Barr, 4th edition. A friendly, applied introduction with lots of real data. openintro.org/book/os
Introduction to Modern Statistics, M. Çetinkaya-Rundel and J. Hardin, 2nd edition. A modern, simulation-first take on the same material. openintro-ims.netlify.app
Introduction to Probability, Statistics, and Random Processes, H. Pishro-Nik. Close to the level of this course, with solved problems and videos. probabilitycourse.com
Introduction to Probability, J. Blitzstein and J. Hwang, 2nd edition. The best book on probability intuition, with superb stories and examples. Free online via stat110.hsites.harvard.edu
Introduction to Probability, C. Grinstead and J. L. Snell. A classic, with many simulations. PDF
Online Statistics Education, D. Lane. An interactive multimedia course. onlinestatbook.com

Videos and visual tools

Seeing Theory (Brown University). Beautiful interactive visualizations of probability and statistics. seeing-theory.brown.edu
StatQuest with Josh Starmer. Clear short videos on statistics and machine learning. youtube.com/@statquest
3Blue1Brown. Visual explanations, including Bayes' theorem and the Central Limit Theorem. youtube.com/@3blue1brown
Harvard Stat 110 (Joe Blitzstein). A full probability course on video, to go with the Blitzstein and Hwang book. stat110.hsites.harvard.edu
Khan Academy: Statistics and probability. Short lessons and practice for anything you're missing. khanacademy.org

Learning R

R for Data Science, H. Wickham, M. Çetinkaya-Rundel and G. Grolemund, 2nd edition. The standard modern introduction to working with data in R. r4ds.hadley.nz
Hands-On Programming with R, G. Grolemund. Learn R by building dice and slot machines. rstudio-education.github.io/hopr
Telling Stories with Data, R. Alexander. Real-world data analysis in R, from collection to communication. tellingstorieswithdata.com

Going further

All free, except where noted.

An Introduction to Statistical Learning, G. James, D. Witten, T. Hastie and R. Tibshirani. The standard first book on machine learning, with R labs. statlearning.com
Causal Inference: What If, M. Hernán and J. Robins. Causal inference for health and social science. miguelhernan.org/whatifbook
Causal Inference: The Remix, S. Cunningham. The second edition of The Mixtape: causal inference with code, aimed at social scientists. The free online version is still being polished. mixtape.scunning.com
Statistical Rethinking, R. McElreath. A Bayesian course with causal diagrams throughout. The book is not free, but the full lecture series is free online. github.com/rmcelreath

Traditional textbooks

Listed from most to least accessible.

Probability and Statistics for Engineering and the Sciences, J. L. Devore, 9th edition.
Probability and Statistics, M. DeGroot and M. Schervish, 4th edition.
Statistical Inference, G. Casella and R. Berger, 2nd edition.
Mathematical Statistics, J. Shao, 2nd edition.

Reading for pleasure

The Lady Tasting Tea, D. Salsburg (2001). The history of statistics through the people who made it.
The Art of Statistics, D. Spiegelhalter (2019). How to learn from data, for everyone.
The Book of Why, J. Pearl and D. Mackenzie (2018). The science of cause and effect.
The Theory That Would Not Die, S. B. McGrayne (2011). The history of Bayes' theorem.
Bernoulli's Fallacy, A. Clayton (2021). A passionate argument for the Bayesian view.
Everything Is Predictable, T. Chivers (2024). Bayes' theorem in the modern world.

0.7 Warm-up exercises

These are for checking your own readiness. If any of them feels hard, reread the matching part of Section 0.4 or 0.5 and try again.

Exercises

Logs. Simplify $ln (λ^{n} e^{- n λ})$ for $λ > 0$ .

Split the log of the product into two logs, then use $ln (a^{r})$ $= r ln a$ and $ln (e^{r}) = r$ .

$ln (λ^{n})$ $+ ln (e^{- n λ})$ $= n ln λ$ $- n λ$ .
Best value. Find the value of $λ > 0$ that maximizes $g (λ) = λ^{4} e^{- 2 λ}$ , and show it is a maximum.

Maximize $ln g (λ)$ $= 4 ln λ$ $- 2 λ$ instead. Differentiate, set to zero, then check the sign of the second derivative.

$\frac{d}{d λ} (4 ln λ$ $- 2 λ)$ $= \frac{4}{λ}$ $- 2$ $= 0$ gives $λ = 2$ . The second derivative is $- 4 / λ^{2} < 0$ , so it is a maximum.
Series. Compute (a) $\sum_{k = 1}^{\infty} (\frac{1}{2})^{k}$ and (b) $\sum_{k = 0}^{\infty} 0.2$ $\times {0.8}^{k}$ .

Both are geometric series. In (a) the sum starts at $k = 1$ , so subtract the $k = 0$ term from $\frac{1}{1 - r}$ . In (b), pull the constant 0.2 out of the sum first.

(a) $\frac{1}{1 - 1 / 2}$ $- 1$ $= 2$ $- 1$ $= 1$ . (b) $0.2$ $\times \frac{1}{1 - 0.8}$ $= 0.2$ $\times 5$ $= 1$ . Both add up to 1, so both could be lists of probabilities: we will meet (b) as the geometric distribution in Chapter 4.
Areas. Let $f (x) = 3 x^{2}$ for $0 \leq x \leq 1$ . Compute (a) $\int_{0}^{1} f (x) d x$ and (b) $\int_{0}^{1} x f (x) d x$ .

An antiderivative of $3 x^{2}$ is $x^{3}$ . For (b), the integrand is $3 x^{3}$ .

(a) $[x^{3}]_{0}^{1} = 1$ . (b) $\int_{0}^{1} 3 x^{3} d x$ $= [\frac{3}{4} x^{4}]_{0}^{1} = \frac{3}{4}$ . In Chapter 5, (a) says $f$ is a valid density and (b) is its average value.
Parts twice. Show that $\int_{0}^{\infty} x^{2} λ e^{- λ x} d x$ $= \frac{2}{λ^{2}}$ .

Use parts with $u = x^{2}$ and $d v = λ e^{- λ x} d x$ . The boundary term vanishes, and what is left is $2 \int_{0}^{\infty} x e^{- λ x} d x$ . Relate that to Example 0.5.

Parts gives $[- x^{2} e^{- λ x}]_{0}^{\infty}$ $+ \int_{0}^{\infty} 2 x e^{- λ x} d x = 0$ $+ \frac{2}{λ} \int_{0}^{\infty} x λ e^{- λ x} d x$ , which equals $\frac{2}{λ} \times \frac{1}{λ} = \frac{2}{λ^{2}}$ , using Example 0.5.
R practice. Using airquality, find (a) the average temperature in July (Month 7), and (b) the number of days on which the wind speed was above 15 mph.

For (a), pick out the July temperatures with square brackets: airquality$Temp[airquality$Month == 7], then take the mean(). For (b), use sum() on a TRUE/FALSE comparison.

(a) mean(airquality$Temp[airquality$Month == 7]) gives 83.9 °F. (b) sum(airquality$Wind > 15) gives 10 days. (Note the double equals sign == for "is equal to"; a single = would be read as setting an argument.)

Chapter 0Before we start

0.1 What this course is about

Think About It: Prediction or Inference?

0.2 How these notes work

The boxes

Definition: What a Definition Looks Like

Theorem: What a Theorem Looks Like

Example: What a Worked Example Looks Like ★ lecture example

Check Yourself

Watch Out: A Common Mistake

Think About It: A Question to Chew on

Causal Inference: The Causal Viewpoint

Bayesian Inference: The Bayesian Viewpoint

Interlude (Optional): A Deeper Idea

History: Where Ideas Come from

Key Takeaways

Exercises

R code

Widgets

How to study from these notes

0.3 How the course runs

Assessments

Using AI tools

0.4 What you should know from calculus

Exponentials and logarithms

Theorem: Rules of Exponentials and Logarithms

Example 0.1: Logs Turn Products Into Sums

Sums

Theorem: Two Series You Will Meet Often

Example 0.2: Probabilities That Add to One

Derivatives and finding the best value

Example 0.3: Finding the Best Value

Integrals and areas

Example 0.4: A Total Area of One

Example 0.5: Integration by Parts

Double integrals

Example 0.6: Integrating Twice

Check Yourself

Think About It: Why Bother with Calculus at All?

Key Takeaways

0.5 A first session with R and RStudio

Getting set up

R as a calculator

Objects and vectors

Functions and help

Data frames

Think About It: Is It Safe to Ignore the Missing Days?

Reading your own data

Packages and a first plot

When things go wrong

Check Yourself

Key Takeaways

0.6 Where else to learn

Free textbooks

Videos and visual tools

Learning R

Going further

Traditional textbooks

Reading for pleasure

0.7 Warm-up exercises

Exercises

Chapter 0
Before we start