Lectures 6–11 | the chapter calculus is built on
There are two kinds of function in the world. Some change at a constant rate, and those are easy: one number describes them completely, and everything about them can be worked out with arithmetic. The rest change at a rate that itself changes, and those are hard.
This chapter is the strategy for dealing with the hard ones, and the strategy is always the same: zoom in until they look easy, do the easy thing, and control the error. Every technique here — slopes of secant lines, rectangles under curves, polynomial approximations, root-finding by iteration — is that one move in a different costume. Calculus, next year, is what happens when you take the zoom to its limit.
A function is linear when equal steps in the input always produce equal steps in the output. That single sentence is the whole content of the topic; the formulas are bookkeeping.
The slope of the line through \((x_1,y_1)\) and \((x_2,y_2)\) is $$m=\frac{y_2-y_1}{x_2-x_1}=\frac{\text{change in output}}{\text{change in input}}.$$ It is the same number whichever two points on the line you pick — and that invariance is exactly what "linear" means.
Useful forms: point-slope \(y-y_1=m(x-x_1)\), which you can write down the instant you know one point and the slope; and slope-intercept \(y=mx+b\).
What is the slope of the line through \((1,4)\) and \((5,12)\)?
decimals, fractions, pi and sqrt( ) are all fine
Slope always carries units, and reading them is the difference between an answer and an understanding. If \(x\) is in hours and \(y\) is in kilometres, the slope is in kilometres per hour, and it is a speed. If \(y\) is in dollars and \(x\) is in items, the slope is the cost of one more item — what an economist calls the marginal cost, and what calculus will eventually call the derivative.
A 3D print job costs \$14.50 for a part using 60 g of filament and \$21.70 for one using 108 g. Assuming the cost is linear in the mass, find the model, and say what each of its two numbers means.
Slope: \(m=\dfrac{21.70-14.50}{108-60}=\dfrac{7.20}{48}=0.15\) dollars per gram.
Point-slope from \((60,14.50)\): \(C-14.50=0.15(m-60)\), so $$C(m)=0.15m+5.50.$$
The slope is the material cost: every extra gram costs 15 cents. The intercept is the \$5.50 you pay before any plastic is used at all — machine time, setup, the operator walking over. It is the cost of a print job of zero grams, which is a thing you would never order but a number that is perfectly real.
Note also which question the model cannot answer: at 5,000 g it predicts \$755.50, and by then you would not be using this printer at all. A linear model is a claim about a range, never about all of \(\mathbb{R}\).
A robot vacuum travels along \(y=\tfrac23x+1\). Find the line through \((6,0)\) parallel to its path, and the line through \((6,0)\) that meets the path at a right angle.
Parallel means identical slope: \(y=\tfrac23(x-6)=\tfrac23x-4\).
Perpendicular means the slopes multiply to \(-1\), so the slope is \(-\tfrac32\): \(y=-\tfrac32(x-6)=-\tfrac32x+9\).
Why \(m_1m_2=-1\)? Rotating a direction by a quarter turn swaps the run and the rise and flips one sign: a step of \((3,2)\) becomes \((-2,3)\). We will prove this properly in Chapter 7, where rotation becomes a matrix and this fact becomes a one-line computation with the dot product.
Real measurements never sit exactly on a line. So we stop asking for the line through the points, which usually does not exist, and start asking for the line that misses them by as little as possible. To do that we have to decide what "as little as possible" means — and that decision is the birth of every fitting procedure in statistics and machine learning.
Given data \((x_i,y_i)\) and a candidate line \(y=mx+b\), the \(i\)th residual is $$e_i=y_i-(mx_i+b),$$ the vertical gap between what actually happened and what the line predicted. The sum of squared residuals is $$\mathrm{SSE}=\sum_i e_i^{2},$$ and the least-squares line is the one line making that total as small as possible.
The data point \((3,10)\) and the line \(y=2x+3\). What is the residual at this point?
decimals, fractions, pi and sqrt( ) are all fine
Two questions are worth asking immediately. Why square the residuals rather than just adding them? Because positive and negative misses would cancel, and a line that is wildly too high here and wildly too low there would score perfectly. Why square rather than take absolute values? Partly because squaring punishes one big miss more than several small ones, and partly — honestly — because squares are smooth and absolute values have a corner, which makes the squared version far easier to minimise. That convenience has shaped the whole subject.
Five measurements: \((1,3),(2,5),(3,4),(4,8),(5,9)\). Two lines are proposed: \(y=1.5x+1.5\) and \(y=2x+0.5\). Which fits better by the least-squares criterion?
| \(x\) | \(y\) | line A pred. | \(e_A\) | line B pred. | \(e_B\) |
|---|---|---|---|---|---|
| 1 | 3 | 3.0 | 0.0 | 2.5 | 0.5 |
| 2 | 5 | 4.5 | 0.5 | 4.5 | 0.5 |
| 3 | 4 | 6.0 | −2.0 | 6.5 | −2.5 |
| 4 | 8 | 7.5 | 0.5 | 8.5 | −0.5 |
| 5 | 9 | 9.0 | 0.0 | 10.5 | −1.5 |
\(\mathrm{SSE}_A=0+0.25+4+0.25+0=4.50\). \(\mathrm{SSE}_B=0.25+0.25+6.25+0.25+2.25=9.25\).
Line A wins. Notice that the third point contributes most of both totals: one bad miss, squared, outweighs four good fits. That sensitivity to outliers is a genuine weakness of least squares, and knowing about it is part of using it honestly.
Neither line is optimal, as it happens. The true least-squares line here is \(y=1.5x+1.3\), with \(\mathrm{SSE}=4.30\) — line A was close but not best. Finding the optimum, rather than testing candidates one at a time, is a minimisation problem. Lecture 8 shows you how to hunt for a minimum, and Lecture 9 shows you how to go straight to it.
You have just met, in full, the structure of supervised machine learning. There is a model with adjustable numbers (here, \(m\) and \(b\)). There is a loss function measuring how badly it does (here, SSE). And there is the problem of choosing the numbers that make the loss smallest, which is called training.
A modern neural network differs in exactly one respect: it has billions of adjustable numbers rather than two, so the loss cannot be minimised by a formula and has to be hunted down by rolling downhill. That hunt is Lecture 8. Everything else — model, loss, training — is already on this page, and you have done it by hand with a slider.
On 1 January 1801 the asteroid Ceres was spotted, tracked for 41 days, and then lost behind the Sun. Predicting where it would re-emerge meant fitting an orbit to a short, noisy arc of observations. Carl Friedrich Gauss, then 24, did it with a method he had not published, and Ceres was recovered in December within half a degree of his prediction.
The method was least squares. Legendre published it first in 1805; Gauss claimed he had been using it since 1795, and the priority dispute was bitter. Either way, the technique that runs inside every spreadsheet's trendline button was invented to find a lost rock.
Ice cream sales and drowning deaths rise and fall together with a strong linear relationship. Ice cream does not cause drowning. Both are caused by summer.
A fitted line measures association and says nothing whatever about cause. No amount of data, and no improvement in the fit, ever upgrades one to the other — that upgrade requires an experiment, or an argument about mechanism, and it comes from outside the mathematics.
A line has one number in it, the slope, and that number never changes. A curved function is precisely one for which no such number exists: the rate of change is different at different places. That is not a vague description, it is the definition, and everything difficult and interesting follows from it.
Before we can approximate curves we should know a few by name.
For \(x>1\) the largest of \(\sqrt x\), \(x\), \(x^{3}\) is ; for \(0<x<1\) the largest is .
Signal strength from a router falls off as \(S(d)=k/d^{2}\). If the signal is 100 units at 2 m, what is it at 4 m, at 6 m, and at 1 m? At what distance has it dropped below 5 units?
From \(S(2)=k/4=100\) we get \(k=400\). Then \(S(4)=25\), \(S(6)=11.1\), \(S(1)=400\).
Below 5 units: \(400/d^{2}<5\) gives \(d^{2}>80\), so \(d>8.9\) m.
Doubling the distance quarters the signal — not halves it. That is the signature of a square in the denominator, and the same shape governs brightness, sound intensity and gravity, all for the same geometric reason: whatever is emitted spreads over a sphere, and the area of a sphere grows with the square of its radius. The physics is a consequence of the geometry.
Compare \(\sqrt{x}\), \(x\), \(x^{2}\) and \(x^{3}\) at \(x=0.01\), \(x=1\) and \(x=100\). Which is largest in each case, and what is the general rule?
| \(\sqrt x\) | \(x\) | \(x^{2}\) | \(x^{3}\) | |
|---|---|---|---|---|
| \(x=0.01\) | 0.1 | 0.01 | 0.0001 | \(10^{-6}\) |
| \(x=1\) | 1 | 1 | 1 | 1 |
| \(x=100\) | 10 | 100 | \(10^{4}\) | \(10^{6}\) |
Below 1 the order is reversed; above 1 the higher power always wins; at \(x=1\) everything ties. That crossing point is why every one of these curves passes through \((1,1)\).
The practical version: for a polynomial, the highest-degree term decides the behaviour far from the origin, and the lowest-degree term decides it near zero. \(x^{3}+500x\) looks like a line near the origin and like a cubic beyond \(x\approx22\).
These are the first graphs to commit to memory. From here on, every new function in the course gets added to this list, and every one of them is reshaped by the same five transformations of Lecture 4.
| function | domain | range | features |
|---|---|---|---|
| \(y=x\) | all reals | all reals | a line; the only member with no bend |
| \(y=x^{2}\) | all reals | \([0,\infty)\) | even; a single lowest point |
| \(y=x^{3}\) | all reals | all reals | odd; flattens at the origin then climbs |
| \(y=\sqrt{x}\) | \([0,\infty)\) | \([0,\infty)\) | the right half of a sideways parabola |
| \(y=\dfrac{1}{x}\) | all reals except 0 | all reals except 0 | two branches; asymptotes \(x=0\) and \(y=0\) |
| \(y=|x|\) | all reals | \([0,\infty)\) | even; a corner at the origin |
Get into the habit of writing the domain and range down every time you sketch. They are the two questions a grader always asks, and for the reciprocal they are the two places the graph does something dramatic.
Signal strength falls off as \(k/d^{2}\). If you move three times as far away, the signal is multiplied by what fraction?
decimals, fractions, pi and sqrt( ) are all fine
Slope tells you which way a curve is going. The next question — the one that separates a curve from a line — is how the slope itself is changing.
A function is concave up on an interval if its slope is increasing there, so the curve bends upward and lies above its own tangent lines. It is concave down if the slope is decreasing. A point where the concavity changes is an inflection point.
Language is a trap here, so it is worth being slow. A concave-down curve can still be rising: think of a rocket that is still gaining altitude but more slowly every second. "Increasing" is about the function; "concave" is about its slope. A quantity can grow while its growth decelerates, and confusing those two statements is how a great deal of bad news gets reported as good.
A country's total cases are recorded weekly: 100, 260, 500, 790, 1090, 1370. Are cases rising? Is the situation improving?
Weekly increases: \(160,\ 240,\ 290,\ 300,\ 280\).
Cases are rising every week, so the function is increasing throughout. But the increases grew, then peaked around week 4, and have started to fall. The curve is concave up at first and concave down at the end, with an inflection around week 4.
And that inflection is the good news, arriving well before any total starts to fall. It is the moment the growth stopped accelerating. Reporters call it "the curve flattening"; mathematically, it is the second difference changing sign, and it is visible in the table long before it is visible in the headline number.
Notice what we just did: we took differences of the data to learn about the slope, and then differences of those to learn about the bending. Do that twice with calculus and you have the first and second derivative. The idea is already here, in arithmetic.
"Bends a lot" ought to be measurable, and it is. Take three nearby points on a curve. Unless they happen to be collinear, exactly one circle passes through all three. Slide the points together and that circle settles down to a definite limiting circle, the one that hugs the curve best at that spot.
The radius of curvature \(R\) at a point is the radius of that best-fitting circle. The curvature is \(\kappa=1/R\).
A tight bend is a small circle, so a large curvature. A straight line is the extreme case: the best circle has infinite radius, and curvature zero.
A tighter bend has a radius of curvature and a curvature.
Find the circle through the three points \((-0.1,\,0.01)\), \((0,\,0)\) and \((0.1,\,0.01)\) on the parabola \(y=x^{2}\), and compare its radius with the one for the closer points \((\pm0.01,\,0.0001)\).
By symmetry the centre is on the \(y\)-axis, at \((0,c)\). Equal distances to \((0,0)\) and \((0.1,0.01)\) give $$c^{2}=0.1^{2}+(c-0.01)^{2}=0.01+c^{2}-0.02c+0.0001,$$ so \(0.02c=0.0101\) and \(c=0.505\). The radius is \(0.505\).
Repeating with \(x=\pm0.01\): \(0.0002c=0.0001+10^{-8}\), giving \(c=0.50005\) and radius \(0.50005\).
The radii are heading to \(0.5\). This is a limit — the third one in this course, after \(e\) and the difference quotient — and it is completely computable by hand. Calculus will later give the shortcut \(\kappa=|y''|/(1+(y')^{2})^{3/2}\), which at the vertex of \(y=x^{2}\) reads \(\kappa=2\). You already have the answer; the formula is a convenience.
Hold a slice flat by the crust and the tip droops. Fold the crust into a gentle U and the slice goes rigid and stays level. Everyone knows the trick. Almost nobody knows that it is a theorem.
A surface has a curvature that can be measured entirely from inside it, without ever stepping off — by comparing the circumference of a small circle drawn on the surface with the \(2\pi r\) you would expect, or by checking whether the angles of a small triangle add to \(180^{\circ}\). Gauss proved in 1827 that this intrinsic curvature is unchanged by bending, as long as you do not stretch or tear. He called it the theorema egregium, the remarkable theorem, and he was not usually given to superlatives.
A flat pizza slice has intrinsic curvature zero. Folding it does not stretch it, so the curvature stays zero. But a surface curving downward along its length and curving across its width would have non-zero curvature at that point — and it is not allowed to. Curve it one way and it is forbidden from curving the other. The slice stays out straight because geometry will not let it do otherwise.
The same theorem says no flat map of the Earth can get both areas and shapes right, that an orange peel will never lie flat, and that a cylinder — surprisingly — is intrinsically flat, which is why you can roll a sheet of paper into one and unroll it undamaged. And it is the doorway to the study of curved spaces in their own right: manifolds, the objects general relativity is written on, where "straight line" is replaced by "the straightest available path" and the shortest route between two cities bends across the map. You will meet the first honest example in Chapter 5, on a sphere, where triangles have too many degrees.
A curved function has no single slope. But it has a slope between any two of its points, and that is something we can compute with nothing but arithmetic.
The average rate of change of \(f\) from \(a\) to \(b\) is $$\frac{f(b)-f(a)}{b-a},$$ the slope of the straight line joining \((a,f(a))\) to \((b,f(b))\). That line is called a secant line.
Writing \(b=a+h\) gives the same quantity as the difference quotient $$\frac{f(a+h)-f(a)}{h}.$$
What is the average rate of change of \(f(x)=x^{2}\) from \(x=1\) to \(x=3\)?
decimals, fractions, pi and sqrt( ) are all fine
A drone's height is \(h(t)=40t-5t^{2}\) metres, \(t\) in seconds. Find its average velocity over \([0,4]\), over \([0,8]\), and over \([3,5]\). Interpret each.
\(h(0)=0\), \(h(3)=75\), \(h(4)=80\), \(h(5)=75\), \(h(8)=0\).
Over \([0,4]\): \(\dfrac{80-0}{4}=20\) m/s upward.
Over \([0,8]\): \(\dfrac{0-0}{8}=0\) m/s. The drone went up 80 m and came back; the average velocity over the whole flight is zero, even though it was moving the entire time.
Over \([3,5]\): \(\dfrac{75-75}{2}=0\) as well — a window straddling the peak.
So "average rate of change is zero" does not mean "nothing happened". It means the endpoints agree. Average speed (total distance over total time) is a different quantity entirely: 160 m in 8 s, so 20 m/s. Which one you want depends on the question, and conflating them is a standard way to get an exam problem wrong.
For \(f(x)=3x^{2}-x\), simplify \(\dfrac{f(a+h)-f(a)}{h}\), then evaluate it at \(a=2\) for \(h=0.1\) and for \(h=0.001\).
\(f(a+h)=3(a+h)^{2}-(a+h)=3a^{2}+6ah+3h^{2}-a-h\), and subtracting \(f(a)=3a^{2}-a\) leaves \(6ah+3h^{2}-h\). Dividing by \(h\): $$\frac{f(a+h)-f(a)}{h}=6a+3h-1.$$
At \(a=2\): with \(h=0.1\) it is \(11.3\); with \(h=0.001\) it is \(11.003\).
Look at what the algebra did. Before dividing, every term had an \(h\) in it, so the quotient was \(\tfrac00\) at \(h=0\) — meaningless. After cancelling, the expression \(6a+3h-1\) is perfectly happy at \(h=0\), where it equals \(6a-1\). That cancellation is the entire trick of differential calculus, performed here with Chapter 0 algebra and nothing else. The number \(6a-1\) is the derivative of \(3x^{2}-x\), and you have just computed it.
Fix a point and let the second point slide towards it. The secant line pivots, and its slope settles down.
| \(h\) | secant slope of \(y=x^{2}\) from \(1\) to \(1+h\) |
|---|---|
| 1 | 3 |
| 0.5 | 2.5 |
| 0.25 | 2.25 |
| 0.1 | 2.1 |
| 0.01 | 2.01 |
| 0.001 | 2.001 |
The pattern is not subtle: the slopes are \(2+h\), heading for \(2\). We cannot reach \(h=0\) — the quotient has no meaning there — but the destination is unmistakable.
Zoom far enough into a smooth curve at a point and it becomes indistinguishable from a straight line. Try it on any graphing tool: pick a point on a parabola, zoom in ten times, and the parabola disappears — what is left is a line, and its slope is the number the secant slopes were heading for.
This is the single most important fact in calculus, and it is a geometric fact, not an algebraic one. Curved things are locally flat. It is why a small enough patch of the Earth can be surveyed with plane geometry, why the tangent line predicts well nearby and badly far away, and why the whole strategy of this chapter — replace the hard curved thing with the easy flat thing, locally — is legitimate rather than lazy.
It is also exactly the property that defines a manifold: a space that looks flat if you stand close enough, however wildly it curves overall. The surface of the Earth is one. So is the space-time of general relativity. A curve that is locally a line is the one-dimensional case, and you are looking at it.
For \(y=x^{2}\), the secant slopes from \(2\) to \(2+h\) are \(4.1\) when \(h=0.1\) and \(4.01\) when \(h=0.01\). What number are they heading for?
decimals, fractions, pi and sqrt( ) are all fine
Estimate \(\sqrt{9.2}\) using only the value \(\sqrt9=3\) and one secant slope.
Take a nearby point: \(\sqrt{9.1}=3.016621\), so the secant slope from 9 to 9.1 is $$\frac{3.016621-3}{0.1}=0.16621.$$ Treat the curve as a line through \((9,3)\) with that slope: $$\sqrt{9.2}\approx 3+0.2(0.16621)=3.03324.$$
The true value is \(3.03315\). We are off by \(9\times10^{-5}\), from one subtraction and one multiplication.
This is a preview of Lecture 11 in miniature. There, instead of replacing the curve with a line, we will replace it with a parabola or a cubic and do far better. And the error is not mysterious: it is the amount by which the curve pulled away from its tangent over that distance — which is to say, it is caused by curvature.
Now the payoff. Suppose you want the lowest point of a curve and cannot solve for it. The slope tells you which way is downhill: where the slope is positive the curve rises to the right, so you should move left; where it is negative, move right. Either way, step against the slope.
Choose a starting point \(x_0\) and a step size \(\alpha>0\). Repeat: $$x_{\text{new}}=x_{\text{old}}-\alpha\cdot(\text{slope at }x_{\text{old}}),$$ estimating the slope with a difference quotient using a small \(h\). Stop when the slope is close enough to zero.
You are at \(x=2\), the estimated slope there is \(3\), and the step size is \(\alpha=0.1\). Where does one step of gradient descent take you?
decimals, fractions, pi and sqrt( ) are all fine
A delivery route's total cost, in hundreds of dollars, as a function of the departure hour \(x\) measured from midday, is modelled by $$C(x)=0.05x^{4}-0.5x^{2}+0.2x+3.$$ Start at \(x_0=3\) with step size \(\alpha=0.4\), estimating each slope with \(h=0.01\). Run six steps.
| step | \(x\) | \(C(x)\) | estimated slope |
|---|---|---|---|
| 0 | 3.0000 | 3.1500 | 2.6001 |
| 1 | 1.9600 | 2.2091 | −0.2541 |
| 2 | 2.0616 | 2.1904 | −0.1091 |
| 3 | 2.1053 | 2.1872 | −0.0391 |
| 4 | 2.1209 | 2.1868 | −0.0128 |
| 5 | 2.1260 | 2.1867 | −0.0041 |
| 6 | 2.1276 | 2.1867 | −0.0013 |
The first step overshoots past the bottom and lands on the far side — the slope was steep and \(\alpha\) was generous. After that the steps shrink automatically, because the slope shrinks as the bottom approaches. The method brakes on its own.
Now run it again from \(x_0=-3\). It converges to \(x=-2.3300\) with \(C=1.2932\) — a lower value than the one we just found. The first run landed in the wrong valley and had no way of knowing.
This is how neural networks are trained, with no embellishment. The loss is a function of billions of adjustable numbers instead of one; the slope in each of those directions is estimated (by a bookkeeping method called backpropagation rather than by difference quotients, but the quantity is the same); and the whole vector of numbers takes a small step against it. Then again, millions of times.
The local-minimum problem in the picture above is entirely real, and the honest answer from the field is that in very high dimensions it matters less than you would expect — with a billion directions to choose from, it is rare for all of them to point uphill at once. But the warning stands, the step size is still chosen by hand, and it still goes wrong in the ways this one-dimensional picture suggests. You now know the whole algorithm.
Chapter 0 taught you to solve equations by factoring. It is worth being honest about how narrow that skill is. \(x^{2}-5x+6=0\) factors. \(x^{5}+x-1=0\) does not, and never will: it has a perfectly ordinary real root near \(0.75\), and it can be proved that no formula built from \(+,-,\times,\div\) and roots can ever express it.
That is not an exotic failure. Almost every equation that arises from a real problem is in this second class. So the practical question is not how to solve equations exactly — usually impossible — but how to compute a root to as many decimal places as you need.
If \(f\) is continuous, \(f(a)<0\) and \(f(b)>0\), then a root lies between. Check the midpoint, keep whichever half still has a sign change, repeat. Each step halves the uncertainty.
Apply bisection to \(f(x)=x^{5}+x-1\) on \([0,1]\) and report six steps.
| step | midpoint | sign of \(f\) | interval kept |
|---|---|---|---|
| 1 | 0.500000 | − | [0.5, 1] |
| 2 | 0.750000 | − | [0.75, 1] |
| 3 | 0.875000 | + | [0.75, 0.875] |
| 4 | 0.812500 | + | [0.75, 0.8125] |
| 5 | 0.781250 | + | [0.75, 0.78125] |
| 6 | 0.765625 | + | [0.75, 0.765625] |
After six steps we know the root to within \(1/2^{6}=0.0156\). To get ten decimal places would take about 34 steps. It always works, it never surprises you, and it is slow — each step buys exactly one binary digit, no matter how nicely the function behaves.
Bisection ignores everything except the sign of \(f\). It never asks how negative the function was. That discarded information is what the next method exploits.
Take two points on the curve, draw the secant line through them — the same line as Lecture 8 — and ask where that line crosses zero. A line's root is trivial to find. Since the line hugs the curve nearby, its root should be close to the curve's root, and we can iterate.
Given two estimates \(x_{n-1},x_n\), the secant line through \((x_{n-1},f(x_{n-1}))\) and \((x_n,f(x_n))\) crosses zero at $$x_{n+1}=x_n-f(x_n)\cdot\frac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})}.$$ The fraction is the reciprocal of the average rate of change. Nothing beyond Lecture 8 is needed.
\(f(1)=-2\) and \(f(3)=2\). Where does the secant line through these two points cross zero?
decimals, fractions, pi and sqrt( ) are all fine
Run the secant method on \(x^{5}+x-1\) from \(x_0=0\), \(x_1=1\), and compare with the bisection table above, step for step.
| step | bisection | secant |
|---|---|---|
| 1 | 0.500000 | 0.500000 |
| 2 | 0.750000 | 0.659574 |
| 3 | 0.875000 | 0.795474 |
| 4 | 0.812500 | 0.748472 |
| 5 | 0.781250 | 0.754456 |
| 6 | 0.765625 | 0.754882 |
The true root is \(0.7548777\). After six steps bisection has two correct digits and the secant method has five. The gap widens from there: bisection gains a fixed amount per step, the secant method gains an increasing amount, because the closer it gets the better a line approximates the curve.
Note that the secant estimates are not trapped in a shrinking interval — step 3 overshoots past the root, step 4 comes back. Speed was bought by giving up the guarantee.
In the secant method the two points can be anywhere. Put them very close together and the secant line becomes the tangent line, whose slope is the number the difference quotients were converging to in Lecture 8. That is Newton's method:
$$x_{n+1}=x_n-\frac{f(x_n)}{m(x_n)},\qquad m(x_n)\approx\frac{f(x_n+h)-f(x_n-h)}{2h}\ \text{for small } h.$$In words: stand on the curve, slide down the tangent line to the axis, and start again from there. Calculus will give you \(m\) exactly and call it \(f'\); until then a small \(h\) gives it to you accurately enough.
For \(f(x)=x^{2}-2\), your guess is \(x=1.5\) and the slope there is \(3\). What is the next Newton guess?
decimals, fractions, pi and sqrt( ) are all fine
Newton's method on \(x^{5}+x-1\) from \(x_0=1\) gives \(0.8333333333\), \(0.7643821157\), \(0.7550248672\), \(0.7548777018\), \(0.7548776662\). The true root is \(0.7548776662\). How many correct digits does each step produce?
| step | estimate | correct digits |
|---|---|---|
| 1 | 0.8333333333 | 1 |
| 2 | 0.7643821157 | 2 |
| 3 | 0.7550248672 | 4 |
| 4 | 0.7548777018 | 7 |
| 5 | 0.7548776662 | 10+ |
The count roughly doubles each step: 1, 2, 4, 7, 10. This is called quadratic convergence, and it is why Newton's method is inside almost every piece of numerical software ever written. Four steps from a mediocre guess gives more precision than most measurements deserve.
Compare the cost honestly, though: bisection needed one function evaluation per step and could not fail. Newton needs a slope estimate as well, and it can fail.
Three failure modes, all worth seeing once:
None of these makes the method bad. They make it a method that needs a decent starting guess — often supplied by a few bisection steps first, which is exactly what good software does.
Lecture 8 hunted for a minimum by rolling downhill. Here is a sharper way to think about it. At the bottom of a valley the slope is zero. So finding a minimum of \(C\) means solving the equation \(m(x)=0\), where \(m\) is the slope function — and solving equations is what this lecture is about.
For the delivery cost \(C(x)=0.05x^{4}-0.5x^{2}+0.2x+3\) of Lecture 8, estimate the slope function with \(h=0.01\) and run the secant method on it from \(x_0=1.5\), \(x_1=3\).
| step | \(x\) | slope there |
|---|---|---|
| 1 | 1.790681 | −0.442 |
| 2 | 1.966482 | −0.246 |
| 3 | 2.185914 | +0.103 |
| 4 | 2.121031 | −0.0126 |
| 5 | 2.128089 | −0.00053 |
Five steps land on \(x=2.1281\), against the true minimum at \(2.1285\). Gradient descent in Lecture 8 needed six steps to reach \(2.1276\) — and that was with a step size chosen by hand, where this needed no such choice.
But notice what has not changed: it found the same valley, the one nearer the starting points, and the deeper minimum at \(x=-2.33\) is still invisible to it. Solving \(m(x)=0\) finds a place where the ground is level. Level ground can be the bottom of a valley, the top of a hill, or a flat shoulder, and the equation cannot tell them apart. Checking which is which is a job for curvature — concave up means a minimum — and that is Lecture 7 coming back to be useful.
The method is older than Newton and was not his alone. Sharaf al-Din al-Tusi, in twelfth-century Persia, gave a numerically identical procedure for cubics; Chinese root-extraction algorithms of comparable power are older still. Newton's version, written around 1669, contained no derivatives and no tangent lines — it was a purely algebraic scheme for improving an approximation, and he applied it to a cubic.
Joseph Raphson published a simplified iterative form in 1690, and Thomas Simpson in 1740 gave the version with derivatives that everyone now recognises, and extended it to systems of equations. The name "Newton–Raphson" records half the story at best, which is normal.
Look at what these two lectures have in common. Gradient descent uses the slope and takes a cautious step. Newton's method uses the slope and takes the exact step that a straight line would require. The first is robust and slow; the second is fast and brittle.
That trade-off has not gone away. Training a large model is done with gradient descent, in variants with names like Adam and SGD, precisely because the honest Newton step is unaffordable when there are a billion directions. Methods that do use curvature information are called second-order, and building affordable ones is an active research problem. When you meet that phrase, you will know exactly what it means: somebody is trying to use the bend, not just the slope.
Rates have occupied us for three lectures. This one asks the opposite question, and it turns out to be the deeper of the two.
If you drive at a steady 60 km/h for 2 hours you cover 120 km, and you got that by multiplying. On a graph of speed against time, that product is literally the area of a rectangle: 60 tall, 2 wide.
Now let the speed vary. The multiplication no longer works — which speed would you use? But the area still does. Area under a speed curve is distance. Area under a flow-rate curve is volume. Area under a power curve is energy; under a rate-of-infection curve, total infections. Whenever you have a rate and want a total, you want an area, and this is not an analogy. It is the same arithmetic, cut finer.
Under a linear function the region is a trapezoid, and you already know its area: average height times width. A tank filling at a rate rising steadily from 2 to 8 litres per minute over 5 minutes receives \(\tfrac{2+8}{2}\times 5=25\) litres. Nothing new is needed, precisely because the rate changes at a constant rate.
Water flows into a tank at a rate rising steadily from 2 L/min to 6 L/min over 4 minutes. How many litres arrive?
decimals, fractions, pi and sqrt( ) are all fine
A car accelerates from rest at a constant rate, reaching 30 m/s after 12 s, then holds that speed for 8 s. Sketch the speed graph and find the total distance.
The graph is a straight climb from \((0,0)\) to \((12,30)\), then a horizontal line to \((20,30)\).
Triangle: \(\tfrac12(12)(30)=180\) m. Rectangle: \(8\times30=240\) m. Total \(420\) m.
The whole calculation is primary-school geometry, and it is exact. Everything hard about integration comes from the moment the top edge stops being straight.
Under a curve there is no formula. So we do the only thing available: cover the region with rectangles whose areas we can compute, and accept an approximation. Divide \([a,b]\) into \(n\) strips of width \(h=(b-a)/n\), and on each strip build a rectangle whose height is the function's value at the left edge, the right edge, or the middle.
Estimate the area under \(y=x^{2}\) on \([0,2]\) with 4 strips, using left endpoints and then right endpoints. What can you conclude about the true area?
Here \(h=0.5\) and the sample points are \(0,0.5,1,1.5\) on the left and \(0.5,1,1.5,2\) on the right.
Left: \(0.5(0+0.25+1+2.25)=1.75\). Right: \(0.5(0.25+1+2.25+4)=3.75\).
Because the function is increasing on this interval, every left rectangle is too short and every right one too tall. So the true area is trapped: $$1.75 < A < 3.75.$$ That is a poor bracket, but it is a guarantee, and it costs eight multiplications. With \(n=50\) the bracket is \(2.587<A<2.747\), and with \(n=200\) it is \(2.647<A<2.687\).
Notice the gap: left and right differ by exactly \(h\left(f(b)-f(a)\right)=h\cdot 4\), which shrinks in proportion to \(h\). Halving the strip width halves the uncertainty — the same slow, dependable behaviour as bisection.
Redo the estimate with midpoints, \(n=4\), and compare the error with the left and right rules.
Midpoints \(0.25,0.75,1.25,1.75\): \(0.5(0.0625+0.5625+1.5625+3.0625)=2.625\).
The true area is \(2.6667\). Errors: left \(-0.917\), right \(+1.083\), midpoint \(-0.042\) — about twenty times better than either endpoint rule, from the same four evaluations.
The reason is visible in a sketch: a midpoint rectangle cuts the curve, losing a sliver on one side and gaining one on the other, and the two very nearly cancel. Averaging the left and right rules instead gives the trapezoid rule, \(2.75\), which is also good but not as good here. Choosing where to sample is free, and it buys accuracy that more strips would cost you dearly.
A car's speedometer is read every 10 seconds: 0, 8, 17, 23, 26, 27, 25 m/s. Estimate the distance travelled in the first minute, and give an honest bracket.
Six strips of width 10 s. Left sum: \(10(0+8+17+23+26+27)=1010\) m. Right sum: \(10(8+17+23+26+27+25)=1260\) m.
The speed is not increasing throughout — it drops in the last interval — so the left and right sums do not strictly bracket the answer. The trapezoid rule, which averages them, gives \(1135\) m and is the sensible estimate.
And here is the point of the example: there is no formula here at all. There is a table of numbers from an instrument. No amount of calculus will give an exact answer, because the exact answer depends on what the car did between readings, and nobody recorded it. Real measurement looks like this far more often than it looks like a tidy formula, and the rectangle method is not a teaching device — it is the tool people actually use.
Archimedes found the area of a parabolic segment around 240 BCE, by filling it with infinitely many triangles and summing the series — getting \(4/3\) of the inscribed triangle, exactly right. He worked from a method of Eudoxus called exhaustion: to prove an area equals \(A\), show it cannot be more and cannot be less.
So area was being computed nineteen centuries before anyone worked out slopes. The hard problem was solved first, one shape at a time, by geniuses. What Newton and Leibniz added was not the idea of area but a shocking discovery about the connection between the two halves of this chapter: finding an area and finding a rate of change are opposite operations, so that solving either one solves the other. That is the Fundamental Theorem of Calculus, and it is the reason calculus is a subject rather than a collection of tricks.
Watch the pattern across this chapter. Slope was a quotient that got better as \(h\) shrank. Area is a sum that gets better as \(h\) shrinks. In both cases we can compute any particular approximation exactly, and in both cases the answer we actually want is the thing those approximations approach.
Calculus supplies the missing word: limit, the value a sequence of ever-better approximations is heading towards. It also supplies two symbols for the two destinations, and both are worth recognising on sight: \(\dfrac{dy}{dx}\) for where the slope quotients are heading, and \(\int\) for where the rectangle sums are heading — a stretched letter S, chosen because it is a sum. You are not being prepared for those symbols. You are already computing the things they stand for.
Here is a question with an uncomfortable answer. Your calculator computes \(\sqrt{2}\), \(\sin 37^{\circ}\) and \(e^{1.4}\) in microseconds. The processor inside it can do four things: add, subtract, multiply and divide. There is no square-root circuit and no sine circuit. So how?
The answer is that it does not compute those functions at all. It computes a polynomial that agrees with them closely enough, because polynomials are made of exactly the four operations available. This lecture builds one by hand.
Suppose we want a polynomial that behaves like \(f\) near \(x=0\). We have three things we can ask it to match, and each costs one coefficient:
A polynomial that gets all three right cannot help hugging the curve nearby: it starts in the same place, sets off in the same direction, and bends the same way. And every one of those three quantities can be measured from sampled values with the arithmetic of Lecture 8.
Sample \(f\) at \(-h\), \(0\) and \(h\). Then
$$c_0=f(0),\qquad c_1\approx\frac{f(h)-f(-h)}{2h},\qquad c_2\approx\frac{f(h)-2f(0)+f(-h)}{2h^{2}},$$and \(p(x)=c_0+c_1x+c_2x^{2}\) is the quadratic that matches \(f\) at that point. The middle expression is the symmetric difference quotient; the last is a difference of differences, which is why it detects bending.
\(f(-0.1)=0.9\), \(f(0)=1\) and \(f(0.1)=1.2\). Estimate the slope coefficient \(c_1\).
decimals, fractions, pi and sqrt( ) are all fine
A function is sampled at three points and you are told nothing else: $$f(-0.2)=0.980067,\qquad f(0)=1,\qquad f(0.2)=0.980067.$$ Build the quadratic approximation, then use it to predict \(f(0.1)\) and \(f(0.5)\).
\(c_0=1\).
\(c_1=\dfrac{0.980067-0.980067}{0.4}=0\). The function is flat at the origin — a peak or a trough.
\(c_2=\dfrac{0.980067-2+0.980067}{2(0.04)}=\dfrac{-0.039866}{0.08}=-0.49833\).
So \(p(x)=1-0.49833x^{2}\), which is suspiciously close to \(1-\tfrac12x^{2}\).
Predictions: \(p(0.1)=0.995017\) and \(p(0.5)=0.875418\).
The mystery function was \(\cos x\). The true values are \(\cos(0.1)=0.995004\) and \(\cos(0.5)=0.877583\). Near the sample points the approximation is good to five decimal places; two-tenths further out it has already lost two of them.
You have just derived the first two terms of the cosine series from three numbers and some division. Madhava of Sangamagrama and the Kerala school had the full series — and \(\pi\) to eleven places — by about 1400, some 250 years before Newton, Gregory and Taylor, working in a tradition that reached Europe by a route historians still argue about.
The same idea, carried to three terms, gives one of the most useful approximations in practice:
valid for \(x\) near zero, with each extra term shrinking the error.
Use the first two terms, \(1+\tfrac{x}{2}\), to estimate \(\sqrt{1.06}\).
decimals, fractions, pi and sqrt( ) are all fine
Approximate \(\sqrt{1.1}\), \(\sqrt{1.5}\) and \(\sqrt{0.7}\) with one, two and three terms, and compare with the true values.
| true | 1 term | 2 terms | 3 terms | |
|---|---|---|---|---|
| \(x=0.1\) | 1.048809 | 1.050000 | 1.048750 | 1.048813 |
| \(x=0.5\) | 1.224745 | 1.250000 | 1.218750 | 1.226562 |
| \(x=-0.3\) | 0.836660 | 0.850000 | 0.838750 | 0.837063 |
At \(x=0.1\) three terms give six correct digits — better than most measurements deserve, from two multiplications and two additions. At \(x=0.5\) the same three terms give only two. The approximation is local: it is built at zero and it degrades as you walk away.
That is not a defect to be apologised for, it is the deal. And it is worth noticing which of the three terms did what: the first term is a flat guess, the second adds the slope, and the third adds the bend — the correction that curvature makes to a straight-line estimate.
Show that a 6% increase followed by a 6% decrease leaves you slightly worse off, and quantify it with the approximation \((1+x)(1-x)=1-x^{2}\).
\(1.06\times0.94=0.9964\), a loss of \(0.36\%\). The identity says the result is \(1-0.06^{2}=1-0.0036\), which is exactly it.
The general lesson: small percentage changes almost cancel, and what survives is a second-order term — a term in \(x^{2}\). Ignore it and you have the useful rule that percentages add; keep it and you know the size of the error you are making.
This is the whole philosophy of approximation in one line. First-order thinking gets you the trend. Second-order thinking — curvature again — tells you how wrong the trend is.
Everything in this lecture was done with sampled values and division. Our estimate of the slope came from two nearby values; our estimate of the bend came from three. Calculus replaces those estimates with exact ones, and once it does, a clean pattern appears in the coefficients: the \(x\) term is built from the exact slope, the \(x^{2}\) term from the exact bend divided by 2, the \(x^{3}\) term from the next quantity divided by 6, and so on down the line. The result is called a Taylor series, and those divisors are doing exactly the job our \(2h^{2}\) denominator was doing.
Then it goes further than we can. It proves how large the error is, rather than letting you observe it. It says which functions are equal to their infinite series and which merely resemble them. And it explains the wall at \(x=-1\) in the picture above — the radius of convergence, a boundary you cannot cross no matter how many terms you take, and which turns out to be governed by behaviour in the complex plane that has nothing visibly to do with the real graph at all.
That is the shape of the whole subject. You arrive already able to compute the objects. Calculus tells you why they work, how well, and where they stop.
One idea, six ways. A curve is hard, so replace it locally with something flat and easy, and control the error: a secant line to measure a rate, a step against that slope to find a valley, a tangent line to chase a root, rectangles to accumulate a total, and a polynomial to stand in for a function a processor cannot compute. Every one of those is exact arithmetic producing an approximate answer, and every one of them gets better as a step size shrinks.
You have also met curvature twice: as the number that measures how hard a curve bends, and as the quantity Gauss proved cannot be changed by bending a surface — the reason a folded slice of pizza holds itself up and a flat map of the world cannot be honest.
Chapter 3 leaves polynomials behind for functions that grow in proportion to themselves, where a limit appears again and produces a number nobody chose.
Next: Chapter 3 — Growth: exponentials and logarithms.