# Lecture 1: Introduction and Mathematical Foundations

Chapter 1 of the notes is the toolbox the other six chapters draw on, and it works every piece of
it on a small example by hand. We run each of those examples here, one script at a time. Each one
prints the chapter's hand-worked value, the value the script computes for it, and the gap between
the two, so you can watch the chapter's arithmetic come out of the code as you read it. The six
scripts are worked examples, and `tests/test_lecture_1.py` asserts every number they print against
the chapter that states it.

We run them from the repository root, with `PYTHONPATH=.` on the front of each command.

```
cd cis6270
PYTHONPATH=. python lecture_1/probability.py
PYTHONPATH=. python -m pytest tests/test_lecture_1.py -q
```

## The files

| File | Command | What it works through |
| --- | --- | --- |
| `probability.py` | `python lecture_1/probability.py` | Sections 1.3 to 1.11. The two-token DNA distribution, the expected adenine count against a Monte Carlo ladder, the worked variance, the three peptides and their covariance matrix, the noising equation, the motif-and-binding table with its marginals and conditionals, Bayes' rule, and the independence check. |
| `linear_algebra.py` | `python lecture_1/linear_algebra.py` | Sections 1.12, 1.13, 1.15, 1.16, 1.17 and 1.18. The simplex and a mixture, the softmax of `z = (2, 1, 0)`, one step of the two-state chain, the eigenvalues of `A = [[2, 1], [1, 2]]`, powers along an eigendirection, the telescoping `A^n = V Lambda^n V^{-1}`, the stationary distribution and its rate, and `e^{tA}` by both the series and the eigenvalues. |
| `calculus.py` | `python lecture_1/calculus.py` | Sections 1.22 to 1.30. The Jacobian by automatic differentiation against the chapter's closed form, the determinant as local volume change, change of variables in one and in two dimensions, the log-determinant of a composition, the two inverse branches of the squaring map, the gradient and directional derivative, the divergence of three fields, and the continuity equation checked numerically on `v(x) = x`. |
| `fields.py` | `python lecture_1/fields.py` | Sections 1.31 and 1.32. The score of the standard Gaussian and of `N(1, 4)`, the normalizing constant differentiated away, the Laplacian of the bowl `x_1^2 + 3 x_2^2` and of the saddle `x_1^2 - x_2^2`, the heat equation at `D = 1/2` on the widening Gaussian `N(0, t)`, and the Fokker-Planck equation with its two reductions, to the continuity equation at `sigma_t = 0` and to the heat equation at `b_t = 0`. The same drift and noise drive an Euler-Maruyama walk, whose sample variance tracks the elapsed time. Chapter 3 starts from both sections. |
| `information.py` | `python lecture_1/information.py` | Sections 1.33 to 1.35. Information content, entropy, the cross-entropy of `p = (0.8, 0.2)` scored by `q = (0.6, 0.4)`, the split `H(p, q) = H(p) + KL(p \|\| q)`, nonnegativity and asymmetry, and maximum likelihood on ten DNA bases as a KL minimization. |
| `optimization.py` | `python lecture_1/optimization.py` | Sections 1.36 to 1.38. Gradient descent on `(theta - 3)^2`, an equality constraint through its Lagrangian, normalization as a constraint, the maximum-entropy derivation that produces the exponential family at a GC target of 0.6, the KKT conditions on a bounded quadratic, and the same likelihood problem solved on logits through the softmax. |

## The flags

Every script takes `--steps`, `--seed`, `--out` and `--quiet`, so you can learn the flags once and
use them across the course. `--steps` sets the iteration count of each script, and every flag a
script takes changes a line of its report.

| Flag | Where it changes the report |
| --- | --- |
| `--steps` | `probability.py`, repeats of the ten-sample Monte Carlo estimate, default 5000. `linear_algebra.py`, iterations of `p_{t+1} = P^T p_t`, default 10. `calculus.py`, Euler steps along the worked flow, default 200. `fields.py`, Euler-Maruyama steps over `[0, 1]`, default 200. `information.py`, repeats of the ten-sample cross-entropy estimate, default 5000. `optimization.py`, gradient-descent steps, default 600. |
| `--samples` | `probability.py`, the largest Monte Carlo sample size, default 1000. `fields.py`, paths simulated along the SDE, default 20000. `information.py`, the draws behind the sampled cross-entropy and the random distribution pairs, default 20000. `optimization.py`, the competing distributions drawn at the same GC target, default 4000. |
| `--grid` | `calculus.py` and `fields.py`, the points used in the numerical checks, default 1000. |
| `--seed` | The Monte Carlo draws, the simulated paths and the random distributions. The same seed gives the same report. |
| `--out` | A directory for `config.json` and `report.json`, written through `cis6270.runner.save_run`. |
| `--quiet` | Print the closing report and nothing else. |

## What a default run prints

| Script | Headline numbers |
| --- | --- |
| `probability.py` | `E[h(X)] = 1.1000`, Monte Carlo estimate `1.1050` at `N = 1000`, `Var(h(X)) = 0.6900`, spread of 5000 ten-sample averages `0.2616` against the predicted `0.2627`, `rho_XY = 0.9449`, `p(X = 1 \| Y = 1) = 0.8182`, association `0.0960`. |
| `linear_algebra.py` | `softmax(z)_1 = 0.6652`, `lambda_1 = 3.0000`, `x_10` first entry `118099`, `p*_1 = 0.6000`, second eigenvalue `0.5000`, `e^{tA}_11 = 1.2275` at `t = 0.1`, truncation error `0.0002`. |
| `calculus.py` | `cos(1) = 0.5403`, `det J_phi = 4.0000` at `x_1 = 2`, `p_Y = 0.1667` inside the image of `A`, `log\|det J\| = 3.1781` for the composition, continuity residual `0.0000`, `log p = -1.0000` along the flow at `t = 1`. |
| `fields.py` | `s(3) = -3.0000`, `s(5) = -1.0000` under `N(1, 4)`, Laplacian `8.0000` on the bowl and `0.0000` on the saddle, `p_1(0) = 0.3989`, `d p_t(0)/dt = -0.1995`, `D` times the Laplacian `-0.1995`, every residual `0.0000`, sample variance `1.0130` at `t = 1` over 20000 paths. |
| `information.py` | `H(p) = 0.5004`, `H(p, q) = 0.5919`, `KL(p \|\| q) = 0.0915`, `KL(q \|\| p) = 0.1046`, spread of 5000 ten-sample estimates `0.0502` against the predicted `0.0513`, empirical loss `1.2799` nats, uniform loss `1.3863` nats, `KL(p_hat \|\| uniform) = 0.1064`. |
| `optimization.py` | `theta = 3.0000` after 600 steps, `lambda = -1.0000` on the equality constraint, `lambda = 0.4055` for a GC target of 0.6, `H(p*) = 1.3662` nats, best drawn competitor `1.3661` nats, `lambda = 4.0000` on the active bound, softmax descent loss `1.2799` nats. |

Every one of these runs in a few seconds on one core, and only the sampled rows move with
`--seed`. `probability.py --steps 50000` closes the measured spread to `0.2624` and
`information.py --steps 50000` closes it to `0.0510`, at the cost of a second each.

## What is simplified

- Every script runs in double precision. Chapter 1 quotes five decimal places in places, and the
  training lectures run in single precision, which cannot reach them.
- The Monte Carlo sections redraw their samples, so the sampled values move with `--seed`. The
  chapter's own ten-sample draw gave 1.2 against the exact 1.1, and each run draws its own ten
  samples, reproducing the exact value and the `1/sqrt(N)` error scale.
- We check the continuity equation by central differences at interior points of the support of
  `p_t`, away from the two ends, where a one-sided difference would be needed. The Fokker-Planck
  residual of `fields.py` uses the same differences, with a second difference for the Laplacian, so
  it sits near `1e-8` and not at zero.
- `fields.py` takes the Brownian increment of `dX_t = b_t(X_t) dt + sigma_t dW_t` as a scaled
  Gaussian draw per step. Chapter 3 defines Brownian motion properly. With a constant drift and a
  constant noise magnitude the Euler-Maruyama walk has the exact law of the chapter's `N(bt, t)`, so
  the gap the script prints is Monte Carlo error alone.
- We integrate the branch density `1/(2 sqrt(y))` numerically over `[0.01, 1]`, because the
  integrand grows without bound at the origin. Its value over the full `[0, 1]` comes from the
  antiderivative the chapter gives.
- The random competitors in the maximum-entropy check and the random pairs in the KL check confirm
  the chapter's inequalities on a finite sample, and the chapter itself gives the proof in full.
  The competitors cover the two free splits of the GC-constrained family, one inside `{A, T}` and
  one inside `{C, G}`, drawn independently.
- Each script has its own three-line `show` helper for the side-by-side printing and its own
  `WORKED` dictionary of computed values. Both stay in the script and out of `cis6270/`, because
  Lecture 1 alone prints a chapter number beside a computed one.