Part IV · Unsupervised learning | why correlation is not dependence, the probability integral transform, Sklar's theorem and the Fréchet–Hoeffding bounds, Kendall's tau, Spearman's rho and tail dependence, elliptical and Archimedean copulas, rank-based fitting, joint crashes in industry returns, synthetic tabular data, the nonparanormal graphical model | reading: Nelsen, An Introduction to Copulas (2006); McNeil, Frey & Embrechts, Quantitative Risk Management (2015), ch. 7; Genest, Ghoudi & Rivest (1995); Liu, Lafferty & Wasserman (2009) | mixture models are in Lecture 17a
Lecture 14 estimated densities and Lecture 17a built them from mixtures. Both treated a multivariate distribution as one object. This lecture splits it in two. Every continuous joint distribution can be written as its one-dimensional margins, which say how each variable behaves on its own, glued together by a copula, which says how they move together and nothing else. The split is useful in practice. Margins are easy to estimate one at a time, even nonparametrically; dependence is where models go wrong, and a copula lets us model it separately, test it, and see where it lives: near the middle of the distribution, or in the tails, where joint crashes and joint floods happen.
By the end you should be able to state and use Sklar's theorem and the probability integral transform; compute Kendall's tau, Spearman's rho and tail dependence from a copula; choose between Gaussian, \(t\) and Archimedean copulas and fit them by maximum pseudo-likelihood on ranks; and use copulas in machine learning to generate synthetic tabular data and to estimate graphical models that are robust to unknown monotone transformations of the data.
Pearson's correlation measures linear association. It is the right summary for a bivariate normal distribution, where it determines the whole dependence structure, and a poor one almost everywhere else. Three facts make this concrete. It changes when one variable is put through an increasing transformation (a change of units from returns to cubed returns, or from income to log income), although the "amount of dependence" has not changed. It can be zero for variables that are perfectly dependent, such as \(X\) and \(X^2\) for a symmetric \(X\). And very different joint distributions share the same correlation, which matters most when the difference is in the tails.
The last point is the subject of this lecture's first experiment. We draw from five joint distributions that all have standard normal margins and all have the same Kendall rank correlation (Definition 17.8), and ask how often the two variables are extreme together.
# Same correlation, different dependence: five bivariate copulas with the same Kendall tau, on standard normal margins import numpy as np, pandas as pd, time, warnings; warnings.filterwarnings('ignore') from scipy import stats, optimize, special, integrate Phi, Phinv = stats.norm.cdf, stats.norm.ppf # --- a small copula library: samplers, log-densities, Kendall's tau and tail dependence for each family ------------------- def debye1(t): # D_1(t) = (1/t) int_0^t s / (e^s - 1) ds, for Frank's tau return integrate.quad(lambda s: s / np.expm1(s), 0, t)[0] / t if t != 0 else 1.0 FAM = { 'Gaussian': dict(tau=lambda r: 2 / np.pi * np.arcsin(r), bounds=(-0.99, 0.99), lam=lambda r: (0.0, 0.0)), 't (4 d.f.)': dict(tau=lambda r: 2 / np.pi * np.arcsin(r), bounds=(-0.99, 0.99), lam=lambda r, nu=4: (2 * stats.t.cdf(-np.sqrt((nu + 1) * (1 - r) / (1 + r)), nu + 1),) * 2), 'Clayton': dict(tau=lambda th: th / (th + 2), bounds=(0.01, 30), lam=lambda th: (2 ** (-1 / th), 0.0)), 'Gumbel': dict(tau=lambda th: 1 - 1 / th, bounds=(1.001, 20), lam=lambda th: (0.0, 2 - 2 ** (1 / th))), 'Frank': dict(tau=lambda th: 1 - 4 / th * (1 - debye1(th)), bounds=(0.01, 60), lam=lambda th: (0.0, 0.0)), } def logc(fam, th, u, v): # log copula density at (u, v) in (0,1)^2 if fam == 'Gaussian': x, y = Phinv(u), Phinv(v); return -0.5 * np.log(1 - th ** 2) - (th ** 2 * (x ** 2 + y ** 2) - 2 * th * x * y) / (2 * (1 - th ** 2)) if fam.startswith('t'): nu = 4; x, y = stats.t.ppf(u, nu), stats.t.ppf(v, nu) q = (x ** 2 + y ** 2 - 2 * th * x * y) / (1 - th ** 2) lj = special.gammaln((nu + 2) / 2) - special.gammaln(nu / 2) - np.log(nu * np.pi) - 0.5 * np.log(1 - th ** 2) - (nu + 2) / 2 * np.log1p(q / nu) return lj - stats.t.logpdf(x, nu) - stats.t.logpdf(y, nu) if fam == 'Clayton': return np.log1p(th) - (th + 1) * (np.log(u) + np.log(v)) - (2 + 1 / th) * np.log(u ** -th + v ** -th - 1) if fam == 'Gumbel': a, b = -np.log(u), -np.log(v); A = a ** th + b ** th; A1 = A ** (1 / th) return -A1 - np.log(u * v) + (th - 1) * np.log(a * b) + (1 / th - 2) * np.log(A) + np.log(A1 + th - 1) if fam == 'Frank': e = -np.expm1(-th) return np.log(th * e) - th * (u + v) - 2 * np.log(np.abs(e - (-np.expm1(-th * u)) * (-np.expm1(-th * v)))) def sample(fam, th, n, g): # n draws (U, V) from the copula u, w = g.random(n), g.random(n) if fam == 'Gaussian': z = g.multivariate_normal([0, 0], [[1, th], [th, 1]], n); return Phi(z[:, 0]), Phi(z[:, 1]) if fam.startswith('t'): z = g.multivariate_normal([0, 0], [[1, th], [th, 1]], n) / np.sqrt(g.chisquare(4, n) / 4)[:, None] return stats.t.cdf(z[:, 0], 4), stats.t.cdf(z[:, 1], 4) if fam == 'Clayton': # invert the conditional distribution C(v | u) return u, (u ** -th * (w ** (-th / (1 + th)) - 1) + 1) ** (-1 / th) if fam == 'Frank': return u, -np.log1p(w * -np.expm1(-th) / (w * (np.exp(-th * u) - 1) - np.exp(-th * u))) / th if fam == 'Gumbel': # Marshall-Olkin: a positive stable frailty S, then exp(-(E/S)^(1/theta)) al = 1 / th; U, E = np.pi * g.random(n), g.exponential(size=n) S = np.sin(al * U) / np.sin(U) ** (1 / al) * (np.sin((1 - al) * U) / E) ** ((1 - al) / al) return tuple(np.exp(-(g.exponential(size=n) / S) ** al) for _ in range(2)) def theta_for_tau(fam, tau): # the parameter that gives a chosen Kendall's tau lo, hi = FAM[fam]['bounds']; return optimize.brentq(lambda t: FAM[fam]['tau'](t) - tau, lo, hi) # --- the experiment --------------------------------------------------------------------------------------------------------- g = np.random.default_rng(0); n = 100_000; tau = 0.5 print(f"{n:,} draws from each copula with Kendall's tau = {tau}, put on standard normal margins") print("copula parameter Pearson r Kendall tau P(Y below its 1% quantile | X below its) the same, above 99%") for fam in FAM: th = theta_for_tau(fam, tau); u, v = sample(fam, th, n, g); x, y = Phinv(u), Phinv(v) qx, qy = np.quantile(x, [0.01, 0.99]), np.quantile(y, [0.01, 0.99]) lo = np.mean(y[x < qx[0]] < qy[0]); hi = np.mean(y[x > qx[1]] > qy[1]) print(f"{fam:12s} {th:10.3f} {np.corrcoef(x, y)[0, 1]:11.3f} {stats.kendalltau(x[:5000], y[:5000])[0]:13.3f} {lo:30.3f} {hi:28.3f}") print("independent copula: either conditional probability would be 0.010")
100,000 draws from each copula with Kendall's tau = 0.5, put on standard normal margins copula parameter Pearson r Kendall tau P(Y below its 1% quantile | X below its) the same, above 99% Gaussian 0.707 0.708 0.495 0.258 0.277 t (4 d.f.) 0.707 0.701 0.499 0.419 0.420 Clayton 2.000 0.682 0.495 0.701 0.036 Gumbel 2.000 0.701 0.493 0.134 0.590 Frank 5.736 0.659 0.510 0.051 0.056 independent copula: either conditional probability would be 0.010
All five samples have Kendall's \(\tau\) of 0.5, and their Pearson correlations are nearly the same too (0.659 to 0.708). Yet they behave very differently where it matters for risk. When \(X\) has one of its worst 1% of values, \(Y\) has one of its own worst 1% with probability 0.258 under the Gaussian copula, 0.419 under the \(t\) copula, 0.701 under Clayton, 0.134 under Gumbel and 0.051 under Frank, against 0.010 for independence. In the upper tail the order changes: Gumbel 0.590, Clayton 0.036. The Gaussian and Frank copulas do have joint extremes at the 1% level, but, as Section 3 shows, the probability tends to zero further out; the \(t\) copula keeps it positive in both tails, Clayton in the lower and Gumbel in the upper. One correlation coefficient cannot tell these apart.
The construction rests on one transformation. Applying a variable's own distribution function to it produces a uniform variable, whatever the distribution was.
Let \(X\) have a continuous distribution function \(F\). Then \(F(X)\) is uniformly distributed on \((0,1)\). Conversely, if \(U\) is uniform on \((0,1)\) and
$$F^{-}(u)=\inf\{x:F(x)\ge u\}$$is the generalised inverse of any distribution function \(F\), then \(F^{-}(U)\) has distribution function \(F\).
For the converse, \(F^{-}(u)\le x\) holds exactly when \(u\le F(x)\), because \(F\) is non-decreasing and right-continuous. So
$$P\big(F^{-}(U)\le x\big)=P\big(U\le F(x)\big)=F(x).$$For the first part, take \(u\in(0,1)\). By continuity there is an \(x\) with \(F(x)=u\), and \(F(X)\le u\) holds exactly when \(X\le F^{-}(u)\) up to an event of probability zero (where \(F\) is flat). Hence
$$P\big(F(X)\le u\big)=P\big(X\le F^{-}(u)\big)=F\big(F^{-}(u)\big)=u.\qquad\square$$The converse is how computers sample from any distribution with a known quantile function, and it is how we shall put new margins on a copula.
A \(d\)-dimensional copula is the distribution function of a random vector \((U_1,\dots,U_d)\) whose components are each uniformly distributed on \((0,1)\):
$$C(u_1,\dots,u_d)=P(U_1\le u_1,\dots,U_d\le u_d).$$Three bivariate copulas are landmarks: the independence copula \(\Pi(u,v)=uv\); the comonotone copula \(M(u,v)=\min(u,v)\), the copula of \((U,U)\); and the countermonotone copula \(W(u,v)=\max(u+v-1,0)\), the copula of \((U,1-U)\). When \(C\) has a density, we write it \(c(u,v)=\partial^2C/\partial u\,\partial v\).
Let \(H\) be the joint distribution function of \((X_1,\dots,X_d)\), with margins \(F_1,\dots,F_d\). Then there is a copula \(C\) such that, for all \(x\),
$$H(x_1,\dots,x_d)=C\big(F_1(x_1),\dots,F_d(x_d)\big).$$If the margins are continuous, \(C\) is unique: it is the distribution function of \(\big(F_1(X_1),\dots,F_d(X_d)\big)\). Conversely, for any copula \(C\) and any distribution functions \(F_1,\dots,F_d\), the formula defines a joint distribution with those margins. With densities, the joint density factorises into a copula part and a margin part:
$$h(x_1,\dots,x_d)=c\big(F_1(x_1),\dots,F_d(x_d)\big)\prod_{j=1}^df_j(x_j).$$Let \(U_j=F_j(X_j)\). By Theorem 17.6 each \(U_j\) is uniform, so the distribution function \(C\) of \((U_1,\dots,U_d)\) is a copula. Because \(F_j\) is continuous and non-decreasing, \(X_j\le x_j\) holds exactly when \(F_j(X_j)\le F_j(x_j)\), up to an event of probability zero. Therefore
$$H(x)=P\big(F_1(X_1)\le F_1(x_1),\dots,F_d(X_d)\le F_d(x_d)\big)=C\big(F_1(x_1),\dots,F_d(x_d)\big).$$Any other copula with this property agrees with \(C\) on \(\operatorname{Ran}F_1\times\dots\times\operatorname{Ran}F_d=(0,1)^d\), so \(C\) is unique. For the converse, let \((U_1,\dots,U_d)\sim C\) and \(X_j=F_j^{-}(U_j)\). By Theorem 17.6 each \(X_j\) has distribution \(F_j\), and, using \(F_j^-(u)\le x\iff u\le F_j(x)\),
$$P(X_1\le x_1,\dots,X_d\le x_d)=P\big(U_1\le F_1(x_1),\dots,U_d\le F_d(x_d)\big)=C\big(F_1(x_1),\dots,F_d(x_d)\big).$$The density formula follows by differentiating once in each argument and using the chain rule. \(\square\)
The density formula is why the split pays off in estimation: the log-likelihood of a sample is the sum of one term for each margin and one term for the copula, so they can be fitted separately. Two simple facts complete the picture. Every copula lies between the two extreme cases, and the copula does not see changes of units.
Every bivariate copula satisfies
$$W(u,v)=\max(u+v-1,0)\;\le\;C(u,v)\;\le\;\min(u,v)=M(u,v).$$The upper bound is attained when \(V\) is an increasing function of \(U\), the lower when it is a decreasing one.
\(C(u,v)=P(U\le u,V\le v)\) is at most \(P(U\le u)=u\) and at most \(P(V\le v)=v\). For the lower bound,
$$C(u,v)=P(U\le u)+P(V\le v)-P(U\le u\text{ or }V\le v)\ge u+v-1,$$and \(C\ge0\). The copula of \((U,U)\) is \(P(U\le\min(u,v))=\min(u,v)\); that of \((U,1-U)\) is \(P(1-v\le U\le u)=\max(u+v-1,0)\). \(\square\)
If \(g_1,\dots,g_d\) are strictly increasing, then \(\big(g_1(X_1),\dots,g_d(X_d)\big)\) has the same copula as \((X_1,\dots,X_d)\). In particular, anything computed from ranks alone is a property of the copula.
The distribution function of \(g_j(X_j)\) is \(F_j\circ g_j^{-1}\), and applying it to \(g_j(X_j)\) gives back \(F_j(X_j)\). So the vectors of probability integral transforms, whose distribution is the copula by Theorem 17.7, are identical. Ranks are preserved by increasing maps, so rank statistics are functions of the copula. \(\square\)
In data we do not know the margins, but we have a natural estimate of \(F_j(X_{ij})\): the rank of observation \(i\) among the \(n\) values of variable \(j\), scaled into the unit interval. The values
$$\hat U_{ij}=\frac{\operatorname{rank}(X_{ij})}{n+1}$$are called pseudo-observations; dividing by \(n+1\) rather than \(n\) keeps them strictly inside \((0,1)\), where copula densities are finite. Here they are on real data.
Kenneth French's data library at Dartmouth publishes daily returns of 30 value-weighted US industry portfolios, built from all NYSE, AMEX and NASDAQ stocks in CRSP and classified by SIC code, from July 1926 (file built from the September 2026 CRSP database). We use 9,254 trading days from January 1990 to September 2026 and two industries: Fin (banks, insurance, real estate, trading) and BusEq (computers, software, electronic equipment). The years 1990–2006 are used for fitting; 2007–2026, which contain the financial crisis of 2008 and the pandemic crash of 2020, are held out.
# Daily returns of two US industry portfolios: the margins are not normal, and ranks fix that (Kenneth French's data library) import io txt = open('data/30_Industry_Portfolios_Daily.csv').read() blk = txt.split('Average Value Weighted Returns -- Daily')[1].split('Average Equal Weighted')[0] ind = pd.read_csv(io.StringIO(blk.strip()), index_col=0); ind.columns = ind.columns.str.strip() ind.index = pd.to_datetime(ind.index.astype(str).str.strip(), format='%Y%m%d'); ind = ind['1990':] train, test = ind[:'2006'], ind['2007':] # fit on 1990-2006; the 2007-2026 period, with 2008 and 2020, is held out print(f"{len(ind):,} trading days, {ind.index[0]:%d %b %Y} to {ind.index[-1]:%d %b %Y}; training {len(train):,}, test {len(test):,}") pair = ['Fin', 'BusEq'] # financial firms; business equipment (computers, software, electronics) X = train[pair].to_numpy() for j, name in enumerate(pair): x = X[:, j]; sd = x.std() print(f"{name:6s} mean {x.mean():+.3f}% sd {sd:.3f}% skewness {stats.skew(x):+.2f} excess kurtosis {stats.kurtosis(x):5.1f} " f"days beyond 4 sd: {np.mean(np.abs(x - x.mean()) > 4 * sd):.4f} (normal: {2 * stats.norm.sf(4):.5f})") # the probability integral transform with a fitted normal margin, and with the ranks pseudo = lambda A: stats.rankdata(A, axis=0) / (len(A) + 1) # pseudo-observations: ranks scaled into (0, 1) U_norm = Phi((X - X.mean(0)) / X.std(0)); U_rank = pseudo(X) for j, name in enumerate(pair): print(f"{name:6s} Kolmogorov-Smirnov distance from uniform: normal margin {stats.kstest(U_norm[:, j], 'uniform').statistic:.3f}" f" (p = {stats.kstest(U_norm[:, j], 'uniform').pvalue:.1e}), ranks {stats.kstest(U_rank[:, j], 'uniform').statistic:.4f}") print(f"Pearson r {np.corrcoef(X.T)[0, 1]:.3f}; Kendall tau {stats.kendalltau(*X.T)[0]:.3f}; Spearman rho {stats.spearmanr(*X.T)[0]:.3f}") print(f"the same rank correlations after cubing one series (a strictly increasing transform): Kendall {stats.kendalltau(X[:, 0] ** 3, X[:, 1])[0]:.3f}, " f"Spearman {stats.spearmanr(X[:, 0] ** 3, X[:, 1])[0]:.3f}; Pearson becomes {np.corrcoef(X[:, 0] ** 3, X[:, 1])[0, 1]:.3f}")
9,254 trading days, 02 Jan 1990 to 30 Sep 2026; training 4,287, test 4,967 Fin mean +0.063% sd 1.124% skewness +0.13 excess kurtosis 4.0 days beyond 4 sd: 0.0047 (normal: 0.00006) BusEq mean +0.059% sd 1.888% skewness +0.36 excess kurtosis 5.6 days beyond 4 sd: 0.0056 (normal: 0.00006) Fin Kolmogorov-Smirnov distance from uniform: normal margin 0.065 (p = 2.2e-16), ranks 0.0036 BusEq Kolmogorov-Smirnov distance from uniform: normal margin 0.069 (p = 5.7e-18), ranks 0.0023 Pearson r 0.584; Kendall tau 0.420; Spearman rho 0.571 the same rank correlations after cubing one series (a strictly increasing transform): Kendall 0.420, Spearman 0.571; Pearson becomes 0.357
Both margins are far from normal. Their excess kurtosis is 4.0 and 5.6 (0 for a normal distribution), and moves beyond four standard deviations happen on 0.47% and 0.56% of days, roughly 74 and 88 times as often as a normal distribution allows. So the probability integral transform with a fitted normal margin does not give uniform values: the Kolmogorov–Smirnov distance is 0.065 and 0.069, with \(p\)-values below \(10^{-15}\). The ranks are uniform by construction (distances 0.0036 and 0.0023, only the discreteness of \(i/(n+1)\)). Pearson's correlation is 0.584, Kendall's \(\tau\) 0.420 and Spearman's \(\rho\) 0.571. Cube one of the series, a strictly increasing change of units, and the two rank correlations do not move while Pearson's falls to 0.357: by Proposition 17.9 the copula, and everything computed from ranks, ignores the margins.
You fit a copula to the pseudo-observations of two variables, then discover that one variable was recorded in logarithms. What do you need to refit?
If dependence is the copula, a measure of dependence should be a property of the copula alone. Pearson's correlation is not (it changed when we cubed a series), but two older rank correlations are.
Let \((X_1,Y_1)\) and \((X_2,Y_2)\) be independent copies of \((X,Y)\). The pair is concordant if \((X_1-X_2)(Y_1-Y_2)>0\) and discordant if the product is negative. Kendall's tau is
$$\tau=P\big[(X_1-X_2)(Y_1-Y_2)\gt0\big]-P\big[(X_1-X_2)(Y_1-Y_2)\lt0\big],$$and Spearman's rho is the Pearson correlation of the probability integral transforms,
$$\rho_S=\operatorname{corr}\big(F(X),G(Y)\big).$$Their sample versions use the observed pairs and the ranks (Kendall, 1938; Spearman, 1904). Both lie in \([-1,1]\), equal \(1\) for the copula \(M\), \(-1\) for \(W\) and \(0\) for \(\Pi\).
For continuous \((X,Y)\) with copula \(C\),
$$\tau=4\int_0^1\!\!\int_0^1C(u,v)\,dC(u,v)-1,\qquad\rho_S=12\int_0^1\!\!\int_0^1\big(C(u,v)-uv\big)\,du\,dv.$$For the Gaussian copula with correlation parameter \(r\) (the copula of a bivariate normal with correlation \(r\)),
$$\tau=\frac2\pi\arcsin r,\qquad\rho_S=\frac6\pi\arcsin\frac r2.$$By Proposition 17.9 we may work with \((U,V)=(F(X),G(Y))\). Concordance has probability \(2P(U_2\lt U_1,V_2\lt V_1)=2\,\mathbb E[C(U_1,V_1)]\), and discordance has probability one minus that, so \(\tau=4\,\mathbb E[C(U,V)]-1\), which is the first formula. For Spearman's rho, \(\operatorname{var}U=\operatorname{var}V=1/12\), and Hoeffding's covariance identity gives
$$\operatorname{cov}(U,V)=\int_0^1\!\!\int_0^1\big(C(u,v)-uv\big)\,du\,dv.$$For the Gaussian copula take \((X,Y)\) bivariate normal with correlation \(r\). The differences \((X_1-X_2,\,Y_1-Y_2)\) are again bivariate normal with correlation \(r\), and for such a pair the probability that both have the same sign is \(\tfrac12+\tfrac1\pi\arcsin r\) (Sheppard, 1899). So \(\tau=2\big(\tfrac12+\tfrac1\pi\arcsin r\big)-1=\tfrac2\pi\arcsin r\). The formula for \(\rho_S\) follows in the same way from the correlation of \(\Phi(X)\) and \(\Phi(Y)\). \(\square\)
A Gaussian copula has correlation parameter \(r=0.5\). What is its Kendall's \(\tau\)? (Three decimals.)
Rank correlations summarise the whole copula. Risk depends on its corners: the probability that both variables are extreme together, as the probability of the event shrinks.
The lower and upper tail-dependence coefficients of a bivariate copula are the limits, when they exist,
$$\lambda_L=\lim_{q\to0}P\big(V\le q\mid U\le q\big)=\lim_{q\to0}\frac{C(q,q)}q,$$ $$\lambda_U=\lim_{q\to1}P\big(V\gt q\mid U\gt q\big)=\lim_{q\to1}\frac{1-2q+C(q,q)}{1-q}.$$A copula with \(\lambda_L\gt0\) makes joint extremes in the lower tail stay likely however extreme the threshold; with \(\lambda_L=0\) they become, relative to one extreme, ever rarer.
For the Gaussian copula with \(r\lt1\), \(\lambda_L=\lambda_U=0\). For the \(t\) copula with correlation \(r\) and \(\nu\) degrees of freedom,
$$\lambda_L=\lambda_U=2\,t_{\nu+1}\!\left(-\sqrt{\frac{(\nu+1)(1-r)}{1+r}}\right),$$where \(t_{\nu+1}\) is the Student \(t\) distribution function. The Clayton copula (Definition 17.10) has \(\lambda_L=2^{-1/\theta}\) and \(\lambda_U=0\); the Gumbel copula has \(\lambda_L=0\) and \(\lambda_U=2-2^{1/\theta}\); the Frank copula has neither.
For Clayton, \(C(q,q)=(2q^{-\theta}-1)^{-1/\theta}\), so
$$\frac{C(q,q)}q=\big(2-q^{\theta}\big)^{-1/\theta}\to2^{-1/\theta}\qquad(q\to0).$$For Gumbel, \(C(q,q)=\exp\!\big(-(2(-\ln q)^\theta)^{1/\theta}\big)=q^{2^{1/\theta}}\). Writing \(a=2^{1/\theta}\), l'Hôpital's rule gives
$$\lambda_U=\lim_{q\to1}\frac{1-2q+q^{a}}{1-q}=\lim_{q\to1}\frac{-2+a\,q^{a-1}}{-1}=2-a.$$The Gaussian and \(t\) results need the asymptotics of the bivariate normal and \(t\) distributions; see McNeil, Frey and Embrechts (2015, section 7.2.4). \(\square\)
A Clayton copula has \(\theta=2\), which gives Kendall's \(\tau=0.5\). What is its lower tail-dependence coefficient? (Three decimals.)
Two kinds of parametric copula cover most applications. Elliptical copulas are taken from elliptical distributions by Sklar's theorem: the Gaussian copula from the multivariate normal, the \(t\) copula from the multivariate \(t\). They extend to any dimension with a correlation matrix as parameter, and the \(t\) copula adds one parameter, \(\nu\), that controls joint extremes in both tails symmetrically. Archimedean copulas are built from a single function.
Let \(\varphi:[0,1]\to[0,\infty]\) be continuous, strictly decreasing and convex with \(\varphi(1)=0\). The Archimedean copula with generator \(\varphi\) is
$$C(u,v)=\varphi^{-1}\big(\varphi(u)+\varphi(v)\big).$$(For a generator with \(\varphi(0)\lt\infty\), \(\varphi^{-1}\) is replaced by the pseudo-inverse, equal to \(\varphi^{-1}\) on \([0,\varphi(0)]\) and to 0 beyond; the three families below have \(\varphi(0)=\infty\).) Three standard choices, each with one parameter \(\theta\):
Rotating a copula by 180 degrees, \((U,V)\mapsto(1-U,1-V)\), swaps its tails: the survival Gumbel copula has dependence in the lower tail.
How should a copula be fitted when the margins are unknown? The density factorisation of Sklar's theorem suggests fitting the margins first and then the copula to the transformed data, but a wrong margin then contaminates the copula. Replacing the margins by ranks avoids that.
Given pseudo-observations \((\hat U_i,\hat V_i)\), the maximum pseudo-likelihood estimate of a copula parameter \(\theta\) is
$$\hat\theta=\arg\max_\theta\sum_{i=1}^n\log c\big(\hat U_i,\hat V_i;\theta\big).$$It needs no model for the margins, and it is consistent and asymptotically normal (Genest, Ghoudi and Rivest, 1995). Families with different numbers of parameters are compared by AIC, \(2k-2\log L\), as in Lecture 17a.
We fit six families to the 1990–2006 returns of the two industries, and then ask the question a risk manager asks: in the held-out years, on a day when financial stocks had one of their worst days, how likely was business equipment to have one too?
# Fit six copula families to the training years by maximum pseudo-likelihood; test them on 2007-2026 def logc_t(r, nu, u, v): x, y = stats.t.ppf(u, nu), stats.t.ppf(v, nu); q = (x ** 2 + y ** 2 - 2 * r * x * y) / (1 - r ** 2) lj = special.gammaln((nu + 2) / 2) - special.gammaln(nu / 2) - np.log(nu * np.pi) - 0.5 * np.log(1 - r ** 2) - (nu + 2) / 2 * np.log1p(q / nu) return lj - stats.t.logpdf(x, nu) - stats.t.logpdf(y, nu) def loglik(fam, par, u, v): if fam == 't': return logc_t(par[0], par[1], u, v) if fam == 'survival Gumbel': return logc('Gumbel', par[0], 1 - u, 1 - v) # Gumbel rotated by 180 degrees: a lower tail return logc(fam, par[0], u, v) def fit(fam, u, v): if fam == 't': # profile likelihood over the degrees of freedom best = None for nu in [2, 2.5, 3, 3.5, 4, 5, 6, 8, 10, 15, 30]: r = optimize.minimize_scalar(lambda r: -np.sum(logc_t(r, nu, u, v)), bounds=(-0.99, 0.99), method='bounded') if best is None or r.fun < best[1]: best = ((r.x, nu), r.fun) return best[0], -best[1] lo, hi = FAM['Gumbel' if fam == 'survival Gumbel' else fam]['bounds'] r = optimize.minimize_scalar(lambda t: -np.sum(loglik(fam, (t,), u, v)), bounds=(lo, hi), method='bounded'); return (r.x,), -r.fun def C_diag(fam, par, q): # C(q, q): the probability that both are below their q-quantiles if fam == 'Gaussian': return stats.multivariate_normal([0, 0], [[1, par[0]], [par[0], 1]]).cdf([Phinv(q)] * 2) if fam == 't': return stats.multivariate_t([0, 0], [[1, par[0]], [par[0], 1]], df=par[1]).cdf([stats.t.ppf(q, par[1])] * 2) if fam == 'Clayton': return (2 * q ** -par[0] - 1) ** (-1 / par[0]) if fam == 'Gumbel': return q ** (2 ** (1 / par[0])) if fam == 'survival Gumbel': return 2 * q - 1 + (1 - q) ** (2 ** (1 / par[0])) if fam == 'Frank': th = par[0]; return -np.log1p(np.expm1(-th * q) ** 2 / np.expm1(-th)) / th def tails(fam, par): if fam == 't': return (2 * stats.t.cdf(-np.sqrt((par[1] + 1) * (1 - par[0]) / (1 + par[0])), par[1] + 1),) * 2 if fam == 'survival Gumbel': return FAM['Gumbel']['lam'](par[0])[::-1] return FAM[fam]['lam'](par[0]) def tau_of(fam, par): return FAM['Gumbel' if fam == 'survival Gumbel' else 'Gaussian' if fam == 't' else fam]['tau'](par[0]) u, v = U_rank.T; Ut = pseudo(test[pair].to_numpy()); ut, vt = Ut.T fams = ['Gaussian', 't', 'Clayton', 'Gumbel', 'survival Gumbel', 'Frank']; fits = {} print(f"Fin and BusEq, {len(u):,} training days; empirical Kendall tau {stats.kendalltau(u, v)[0]:.3f} (test years: {stats.kendalltau(ut, vt)[0]:.3f})") print("family parameters log-lik AIC implied tau lambda_L lambda_U test log-lik per day") for fam in fams: par, ll = fit(fam, u, v); fits[fam] = par; k = len(par) ps = ', '.join(f'{p:.3g}' for p in par); lL, lU = tails(fam, par) print(f"{fam:16s} {ps:16s} {ll:10.1f} {2 * k - 2 * ll:9.1f} {tau_of(fam, par):12.3f} {lL:10.3f} {lU:10.3f} {np.mean(loglik(fam, par, ut, vt)):17.4f}") print("\nP(BusEq has one of its worst q of days | Fin does), test years 2007-2026, against each copula fitted on 1990-2006:") print("q observed " + " ".join(f"{f[:10]:>10s}" for f in fams)) for q in [0.05, 0.01]: obs = np.mean(vt[ut < q] < q) print(f"{q:<8} {obs:9.3f} " + " ".join(f"{C_diag(f, fits[f], q) / q:10.3f}" for f in fams))
Fin and BusEq, 4,287 training days; empirical Kendall tau 0.420 (test years: 0.469) family parameters log-lik AIC implied tau lambda_L lambda_U test log-lik per day Gaussian 0.588 906.7 -1811.4 0.400 0.000 0.000 0.2876 t 0.611, 3.5 1105.3 -2206.5 0.418 0.350 0.350 0.3405 Clayton 0.974 747.0 -1492.0 0.328 0.491 0.000 0.2601 Gumbel 1.68 980.6 -1959.2 0.406 0.000 0.491 0.2939 survival Gumbel 1.67 941.9 -1881.8 0.401 0.485 0.000 0.3137 Frank 4.62 921.3 -1840.6 0.432 0.000 0.000 0.2686 P(BusEq has one of its worst q of days | Fin does), test years 2007-2026, against each copula fitted on 1990-2006: q observed Gaussian t Clayton Gumbel survival G Frank 0.05 0.546 0.302 0.420 0.505 0.217 0.505 0.190 0.01 0.490 0.180 0.378 0.494 0.096 0.489 0.045
The \(t\) copula wins on every likelihood measure. Its AIC is −2206.5, against −1959.2 for the runner-up (Gumbel) and −1811.4 for the Gaussian; on the held-out years it also has the highest log-likelihood per day (0.3405, Gaussian 0.2876). Its fitted degrees of freedom, \(\nu=3.5\), are small, and the tail dependence they imply, \(\lambda=0.350\) in each tail, is far from the Gaussian copula's zero. The crash table shows what the Gaussian copula misses. In 2007–2026, on the days when financial stocks had one of their worst 1% of days, business equipment had one of its own worst 1% 49% of the time. The Gaussian copula fitted to 1990–2006 predicts 18%, the \(t\) copula 38%, and the two lower-tail families, Clayton and survival Gumbel, 49%. There is a trade-off. Clayton and survival Gumbel describe the lower tail well and the rest of the distribution less well (Clayton has the worst AIC of all, and survival Gumbel trails the \(t\) and Gumbel copulas), because they have no upper tail dependence; the \(t\) copula is the best single description but treats booms like crashes. Part of the miss is also the data's fault: Kendall's \(\tau\) rose from 0.420 in the training years to 0.469 in the test years. Dependence is not constant, and it tends to rise in crises, which is the case for models whose copula parameters change over time.
Wassily Hoeffding (1940) and Maurice Fréchet (1951) studied joint distributions with given margins and found the bounds of Theorem 17.8. Answering Fréchet's question of which joint distributions are possible, Abe Sklar (1959) proved the theorem that bears his name and coined the word "copula", Latin for a link or bond. For forty years copulas were mostly a topic in probability theory, until Paul Embrechts, Alexander McNeil and Daniel Straumann (1999, published 2002) showed risk managers how badly correlation can mislead them outside the normal world. In 2000 David X. Li proposed using a Gaussian copula to model the joint default times of companies, and the method became the industry standard for pricing collateralised debt obligations: a single correlation parameter summarised the dependence of hundreds of mortgages or loans. The Gaussian copula has no tail dependence (Proposition 17.11), so it treated simultaneous defaults as far rarer than they turned out to be in 2007–2008. Felix Salmon's 2009 article in Wired called it "the formula that killed Wall Street"; the model did not cause the crisis, but its users had relied on one number to describe a dependence that lived in the tails.
Banks and insurers use copulas to combine the risks of business lines, assets or perils into one loss distribution, in capital models built for regulators. Hydrologists use them for floods, where the peak flow, the volume and the duration of a flood are dependent and the design of a dam depends on their joint extremes. In machine learning, Gaussian copulas generate synthetic tabular data (Section 5), model high-dimensional forecasts (Salinas et al., 2019, use a low-rank Gaussian copula process for thousands of related time series), estimate graphical models robustly (Section 6), and estimate mutual information, which equals the negative entropy of the copula (Ma and Sun, 2011). For many variables, vine copulas build a \(d\)-dimensional copula from \(d(d-1)/2\) bivariate ones arranged on a sequence of trees (Bedford and Cooke, 2002; Aas et al., 2009), which allows different tail behaviour for different pairs.
Organisations often want to share data they cannot share: patient records, tax returns, customer histories. One response is to release synthetic data, records drawn from a model fitted to the real ones, which should be useful for analysis and model building while matching no real person. Sklar's theorem gives the simplest such model: keep each column's own distribution exactly, and glue the columns together with a Gaussian copula. This was the original model of the Synthetic Data Vault (Patki, Wedge and Veeramachaneni, 2016), and it is a standard baseline in later benchmarks of deep generative models for tables, such as CTGAN (Xu et al., 2019).
Given a table with \(n\) rows and \(d\) columns:
Each synthetic column then has the real column's distribution, and pairs of columns have, approximately, the real Spearman correlations.
The usual test of synthetic data is train on synthetic, test on real: fit a model to the synthetic records and measure its error on real held-out records. We use the California housing data of Lectures 3 and 8.
# Synthetic tabular data from a Gaussian copula (the California housing data of Lectures 3 and 8): train on synthetic, test on real from sklearn.model_selection import train_test_split from sklearn.ensemble import HistGradientBoostingRegressor from sklearn.neighbors import NearestNeighbors raw = pd.read_csv('data/housing.csv').dropna() cal = pd.DataFrame({'MedInc': raw.median_income, 'HouseAge': raw.housing_median_age, 'AveRooms': raw.total_rooms / raw.households, 'AveBedrms': raw.total_bedrooms / raw.households, 'Population': raw.population, 'AveOccup': raw.population / raw.households, 'Latitude': raw.latitude, 'Longitude': raw.longitude, 'Value': raw.median_house_value / 1e5}) tr, te = train_test_split(cal, test_size=0.2, random_state=0) # Lecture 3's split R = tr.to_numpy(); n, d = R.shape; g = np.random.default_rng(0) def inv_ecdf(Rcol, w): # map uniforms back through the empirical quantiles of a column return np.quantile(Rcol, w, method='inverted_cdf') # every synthetic value is a real value of that column def gaussian_copula_synth(R, m, g): Z = Phinv(pseudo(R)); S = np.corrcoef(Z.T) # normal scores and their correlation: the fitted Gaussian copula W = Phi(g.multivariate_normal(np.zeros(R.shape[1]), S, m)) # draws from that copula return np.column_stack([inv_ecdf(R[:, j], W[:, j]) for j in range(R.shape[1])]), S def independent_synth(R, m, g): # the same margins, no dependence at all (the independence copula) return np.column_stack([inv_ecdf(R[:, j], g.random(m)) for j in range(R.shape[1])]) S_gc, Sig = gaussian_copula_synth(R, n, g); S_ind = independent_synth(R, n, g) Xte, yte = te.to_numpy()[:, :-1], te.to_numpy()[:, -1] def tstr(D): # train on (synthetic) D, test on the real test set m = HistGradientBoostingRegressor(max_iter=500, learning_rate=0.05, random_state=0).fit(D[:, :-1], D[:, -1]) return np.sqrt(np.mean((m.predict(Xte) - yte) ** 2)) sp = lambda D: stats.spearmanr(D).statistic nn = NearestNeighbors(n_neighbors=1).fit((R - R.mean(0)) / R.std(0)) dcr = lambda D: np.median(nn.kneighbors((D - R.mean(0)) / R.std(0))[0]) # median distance to the closest real training record print(f"{n:,} real training districts, {len(te):,} real test districts; gradient boosting predicts the median house value ($100k)") print("training data test RMSE largest error in a Spearman correlation median distance to closest real record") print(f"{'real':34s} {tstr(R):9.3f} {0.0:24.3f} {dcr(te.to_numpy()):36.3f} (real test districts)") for name, D in [('Gaussian copula, empirical margins', S_gc), ('independent margins', S_ind)]: print(f"{name:34s} {tstr(D):9.3f} {np.abs(sp(D) - sp(R)).max():24.3f} {dcr(D):36.3f}") k_lat, k_lon = cal.columns.get_loc('Latitude'), cal.columns.get_loc('Longitude') def on_land(D, eps=0.05): # share of synthetic districts within 0.05 degrees of some real district nb = NearestNeighbors(n_neighbors=1).fit(R[:, [k_lat, k_lon]]); return np.mean(nb.kneighbors(D[:, [k_lat, k_lon]])[0][:, 0] < eps) print(f"share of districts within 0.05 degrees (about 5 km) of a real district: Gaussian copula {on_land(S_gc):.1%}, " f"independent {on_land(S_ind):.1%}, real test {on_land(te.to_numpy()):.1%}")
16,346 real training districts, 4,087 real test districts; gradient boosting predicts the median house value ($100k) training data test RMSE largest error in a Spearman correlation median distance to closest real record real 0.452 0.000 0.356 (real test districts) Gaussian copula, empirical margins 0.672 0.090 0.540 independent margins 1.159 0.881 0.955 share of districts within 0.05 degrees (about 5 km) of a real district: Gaussian copula 60.9%, independent 34.2%, real test 95.6%
Trained on real districts, gradient boosting has a test RMSE of 0.452 ($100,000s). Trained on the same number of synthetic districts from the Gaussian copula it gets 0.672, much better than the 1.159 of synthetic data with the right margins and no dependence, but clearly worse than real data. The pairwise rank correlations are kept (the largest error in a Spearman correlation is 0.090), so what is lost is dependence that a correlation matrix cannot express. Geography shows it plainly: only 60.9% of the Gaussian copula's districts lie within about 5 km of a real district, against 95.6% of real test districts, because California's coastline and its cities are not an ellipse in normal scores. Synthetic records are also no closer to real ones than real test records are (median distance to the closest training record 0.540 against 0.356), which is reassuring but is not a proof of privacy: an outlier copied from the training data would be invisible in a median.
Why does the Gaussian copula synthesiser put many districts in the sea, although every synthetic latitude and every synthetic longitude is a real one?
Two cautions apply to all synthetic data. Utility is measured on the questions one thought to ask: the TSTR error above says nothing about rare subgroups or about interactions that the regression model did not use. And distance to the closest real record is a weak privacy check; formal guarantees need methods designed for them, such as differentially private synthesisers, whose noise costs further utility.
Lecture 20a will estimate the conditional-independence graph of many variables with the graphical lasso, which assumes multivariate normal data. Real variables are skewed, bounded or heavy-tailed, and a monotone transformation that would make them normal is unknown. A Gaussian copula with arbitrary margins is exactly the model that is normal "up to unknown monotone transformations".
A random vector \(X\in\mathbb R^d\) follows a nonparanormal distribution, \(X\sim\mathrm{NPN}(\Sigma,f)\), if there are strictly increasing functions \(f_1,\dots,f_d\) such that
$$\big(f_1(X_1),\dots,f_d(X_d)\big)\sim\mathcal N(0,\Sigma),\qquad\Sigma_{jj}=1.$$Equivalently, \(X\) has a Gaussian copula with correlation matrix \(\Sigma\) and arbitrary continuous margins.
If \(X\sim\mathrm{NPN}(\Sigma,f)\), then \(X_j\) and \(X_k\) are conditionally independent given the other variables exactly when \((\Sigma^{-1})_{jk}=0\), and \(\Sigma\) is determined by the rank correlations:
$$\Sigma_{jk}=\sin\Big(\frac\pi2\,\tau_{jk}\Big)=2\sin\Big(\frac\pi6\,\rho_{S,jk}\Big).$$Each \(f_j\) is invertible, so conditioning on \(X_k\) is the same as conditioning on \(f_k(X_k)\), and conditional independence of the \(X_j\) is conditional independence of the \(f_j(X_j)\). For a multivariate normal vector the latter holds exactly when the corresponding entry of the precision matrix is zero (Lecture 20a). The rank correlations of \(X\) are those of \(f(X)\) (Proposition 17.9), and for a normal vector Proposition 17.10 gives \(\tau=\tfrac2\pi\arcsin\Sigma_{jk}\) and \(\rho_S=\tfrac6\pi\arcsin(\Sigma_{jk}/2)\); inverting gives the formulas. \(\square\)
So the graphical lasso can be run on a correlation matrix estimated from ranks, either the correlation of the normal scores \(\Phi^{-1}(\hat U_{ij})\) or Kendall's \(\tau\) mapped through \(\sin(\pi\tau/2)\) (the SKEPTIC estimator of Liu et al., 2012). We test both on a known graph, observed through unknown distortions.
# The nonparanormal (a Gaussian copula graphical model): recover a sparse graph after unknown monotone distortions of the variables from sklearn.covariance import graphical_lasso p, nobs = 30, 300; g = np.random.default_rng(1) Theta = np.eye(p) # a sparse precision matrix: a chain plus a few random edges for j in range(p - 1): Theta[j, j + 1] = Theta[j + 1, j] = 0.4 for a, b in g.choice(p, (8, 2)): if abs(a - b) > 1: Theta[a, b] = Theta[b, a] = 0.3 Theta += np.eye(p) * max(0, 0.2 - np.linalg.eigvalsh(Theta).min()); Sigma = np.linalg.inv(Theta) d_ = np.sqrt(np.diag(Sigma)); Sigma = Sigma / np.outer(d_, d_); true = (np.abs(np.linalg.inv(Sigma)) > 1e-8) & ~np.eye(p, dtype=bool) Z = g.multivariate_normal(np.zeros(p), Sigma, nobs) f = [lambda z: np.exp(2 * z), lambda z: z ** 5, lambda z: np.sign(z) * np.abs(z) ** 0.2, lambda z: 1 / (1 + np.exp(-4 * z)), lambda z: z] Xd = np.column_stack([f[j % 5](Z[:, j]) for j in range(p)]) # what we observe: each variable bent by an unknown increasing function def best_f1(S): # graphical lasso along a path of penalties; best F1 for the edges out = [] for lam in np.geomspace(0.02, 0.6, 25): try: P = graphical_lasso(S, alpha=lam, max_iter=200)[1] except Exception: continue est = (np.abs(P) > 1e-6) & ~np.eye(p, dtype=bool); tp = np.sum(est & true) / 2 out.append(2 * tp / ((est.sum() + true.sum()) / 2) if est.any() else 0.0) return max(out) pearson = lambda X: np.corrcoef(X.T) normal_scores = lambda X: np.corrcoef(Phinv(pseudo(X)).T) # normal scores: a simplified form of Liu, Lafferty and Wasserman (2009) def kendall_sin(X): # SKEPTIC: Sigma_jk = sin(pi tau_jk / 2) (Liu et al., 2012) T = np.eye(X.shape[1]) for a in range(X.shape[1]): for b in range(a + 1, X.shape[1]): T[a, b] = T[b, a] = np.sin(np.pi / 2 * stats.kendalltau(X[:, a], X[:, b])[0]) w, V = np.linalg.eigh(T); return V @ np.diag(np.maximum(w, 1e-3)) @ V.T # project to a positive definite matrix print(f"{p} variables, {int(true.sum() / 2)} true edges, {nobs} observations; best F1 of the recovered edge set along the penalty path") print(f" graphical lasso on the Gaussian data before distortion (the ideal): {best_f1(pearson(Z)):.3f}") print(f" graphical lasso on Pearson correlations of the distorted data: {best_f1(pearson(Xd)):.3f}") print(f" nonparanormal, normal-score correlations: {best_f1(normal_scores(Xd)):.3f}") print(f" nonparanormal, Kendall's tau mapped by sin(pi tau / 2): {best_f1(kendall_sin(Xd)):.3f}")
30 variables, 36 true edges, 300 observations; best F1 of the recovered edge set along the penalty path graphical lasso on the Gaussian data before distortion (the ideal): 0.958 graphical lasso on Pearson correlations of the distorted data: 0.784 nonparanormal, normal-score correlations: 0.958 nonparanormal, Kendall's tau mapped by sin(pi tau / 2): 0.958
On the undistorted Gaussian data the graphical lasso recovers the 36 edges with an F1 of 0.958 at its best penalty. After the distortions, Pearson correlations mislead it, and the best F1 drops to 0.784. Both nonparanormal estimators recover 0.958, the same as with the undistorted data. The invariance is exact: ranks are unchanged by increasing transformations, so the rank-based matrices are the same whether computed from the distorted or the undistorted data. That they also match the Pearson ideal to three decimals reflects how coarse F1 is on 36 edges. The comparison is generous to all methods, since each is scored at its own best penalty, and it is one simulated graph; but the conclusion, that a rank-based estimate costs almost nothing when the data are Gaussian and protects against distortions when they are not, matches Liu et al.'s theory and experiments.
A Bayesian treatment of a copula with unknown margins seems to need priors for the margins too. Hoff (2007) avoids them with the extended rank likelihood: the observed data imply only that the latent normal scores \(Z_{ij}\) are in the same order as the \(X_{ij}\) within each column, and the likelihood of the correlation matrix is the probability of that ordering. A Gibbs sampler alternates between drawing the latent scores, each constrained to an interval set by its neighbours in rank, and drawing the correlation matrix from its conditional posterior. The margins never need to be modelled, and the method works for ordinal and mixed discrete–continuous data, where ranks have ties; it is widely used for imputing missing values in surveys.
Whether \(X\) and \(Y\) are independent, or conditionally independent given \(Z\), is a property of the copula alone: independence means \(C=\Pi\). Constraint-based causal discovery, such as the PC algorithm of Lecture 20b, uses only such statements, so it can run on rank-based tests that are valid for any continuous margins; Harris and Drton (2013) proved that PC with Kendall or Spearman correlations recovers the Markov equivalence class of a nonparanormal model consistently, even in high dimensions. The methods that go beyond equivalence classes, such as LiNGAM or additive-noise models, need more than the copula: they rely on the shapes of the distributions, which the copula has deliberately thrown away.
Next: Lecture 18 turns from tables to signals, where the dependence of a series on its own past is the subject.
\(C_\theta(u,1)=u\) and \(C_\theta(1,v)=v\), and the density \(c=1+\theta(1-2u)(1-2v)\ge0\) when \(|\theta|\le1\). By Proposition 17.10,
$$\rho_S=12\int_0^1\!\!\int_0^1\theta\,uv(1-u)(1-v)\,du\,dv=12\theta\Big(\frac16\Big)^2=\frac\theta3,$$and a similar calculation gives \(\tau=2\theta/9\). So \(|\rho_S|\le1/3\) and \(|\tau|\le2/9\): the family can only represent weak dependence, which is why it is used for perturbation arguments rather than for data.
No. A bivariate normal distribution has a Gaussian copula, which by Sklar's theorem is the only copula giving that joint distribution with normal margins. The Clayton copula has lower tail dependence \(2^{-1/2}\approx0.71\) (Proposition 17.11) while every Gaussian copula with \(r\lt1\) has none. Normal margins are not enough for joint normality.
\(\tau=\tfrac2\pi\arcsin0.9\approx0.713\) and \(\rho_S=\tfrac6\pi\arcsin0.45\approx0.891\). Spearman's rho is always close to the Pearson correlation under a Gaussian copula (within about 0.02), while Kendall's tau is noticeably smaller; the measures are on different scales and should not be compared across one another.
\(\lambda=2t_5\big(-\sqrt{5\cdot0.4/1.6}\big)=2t_5(-1.118)\approx0.31\) for \(\nu=4\), and \(2t_{31}\big(-\sqrt{31\cdot0.4/1.6}\big)=2t_{31}(-2.78)\approx0.009\) for \(\nu=30\). As \(\nu\to\infty\) the \(t\) copula tends to the Gaussian copula and \(\lambda\to0\): heavy tails need small degrees of freedom.
Use the pseudo-observations within each window and the profile-likelihood fit of the code box. Expect dependence to rise in crises (for example 2000–2002, 2008–2009 and 2020), with the fitted correlation increasing and \(\nu\) tending to fall, so that joint extremes become more likely exactly when they matter; the test-period rise in Kendall's \(\tau\) in Section 4 points the same way. A model fitted on calm years under-predicts joint losses in the next crisis, as the test-period table showed; practice uses time-varying copulas or margins filtered by volatility models (GARCH) and refits frequently.
A Gaussian mixture in normal-score space is no longer a Gaussian copula, and it can bend around the coastline and concentrate on the cities; to keep the real margins, map each synthetic coordinate through the mixture's own marginal distribution function before the empirical quantile function. Expect the share of synthetic districts near real ones and the TSTR error both to move towards the real-data values, with more components helping until the mixture starts to memorise the training rows, which the distance-to-closest-record check will reveal.
Each project uses this lecture's methods on public data, and has a clear result.
pyvinecopulib package) to the standardised residuals in rolling windows; backtest one-day 99% value-at-risk of an equal-weighted portfolio with Kupiec's test.Data. Industry portfolios: K. R. French, Data Library, "30 Industry Portfolios" (daily, value-weighted), mba.tuck.dartmouth.edu, Data Library, built from CRSP data. California housing: R. K. Pace, R. Barry, "Sparse spatial autoregressions", Statistics & Probability Letters 33(3), 291–297 (1997), in the copy from A. Géron's Hands-On Machine Learning repository, as in Lectures 3 and 8. The graph in Section 6 is simulated.
numbers in the text not printed by a code box are computed in verify_L17b.py, figures and widget data in figs_L17b.py; code-box outputs are pasted from actual runs (numpy, SciPy and scikit-learn on two CPU cores)