lecture_1/README.md

Lecture 1: Introduction and Mathematical Foundations

Chapter 1 of the notes is the toolbox the other six chapters draw on, and it works every piece of it on a small example by hand. We run each of those examples here, one script at a time. Each one prints the chapter's hand-worked value, the value the script computes for it, and the gap between the two, so you can watch the chapter's arithmetic come out of the code as you read it. The six scripts are worked examples, and tests/test_lecture_1.py asserts every number they print against the chapter that states it.

We run them from the repository root, with PYTHONPATH=. on the front of each command.

cd cis6270
PYTHONPATH=. python lecture_1/probability.py
PYTHONPATH=. python -m pytest tests/test_lecture_1.py -q

The files

File	Command	What it works through
probability.py	python lecture_1/probability.py	Sections 1.3 to 1.11. The two-token DNA distribution, the expected adenine count against a Monte Carlo ladder, the worked variance, the three peptides and their covariance matrix, the noising equation, the motif-and-binding table with its marginals and conditionals, Bayes' rule, and the independence check.
linear_algebra.py	python lecture_1/linear_algebra.py	Sections 1.12, 1.13, 1.15, 1.16, 1.17 and 1.18. The simplex and a mixture, the softmax of z = (2, 1, 0), one step of the two-state chain, the eigenvalues of A = [[2, 1], [1, 2]], powers along an eigendirection, the telescoping A^n = V Lambda^n V^{-1}, the stationary distribution and its rate, and e^{tA} by both the series and the eigenvalues.
calculus.py	python lecture_1/calculus.py	Sections 1.22 to 1.30. The Jacobian by automatic differentiation against the chapter's closed form, the determinant as local volume change, change of variables in one and in two dimensions, the log-determinant of a composition, the two inverse branches of the squaring map, the gradient and directional derivative, the divergence of three fields, and the continuity equation checked numerically on v(x) = x.
fields.py	python lecture_1/fields.py	Sections 1.31 and 1.32. The score of the standard Gaussian and of N(1, 4), the normalizing constant differentiated away, the Laplacian of the bowl x_1^2 + 3 x_2^2 and of the saddle x_1^2 - x_2^2, the heat equation at D = 1/2 on the widening Gaussian N(0, t), and the Fokker-Planck equation with its two reductions, to the continuity equation at sigma_t = 0 and to the heat equation at b_t = 0. The same drift and noise drive an Euler-Maruyama walk, whose sample variance tracks the elapsed time. Chapter 3 starts from both sections.
information.py	python lecture_1/information.py	Sections 1.33 to 1.35. Information content, entropy, the cross-entropy of p = (0.8, 0.2) scored by q = (0.6, 0.4), the split H(p, q) = H(p) + KL(p || q), nonnegativity and asymmetry, and maximum likelihood on ten DNA bases as a KL minimization.
optimization.py	python lecture_1/optimization.py	Sections 1.36 to 1.38. Gradient descent on (theta - 3)^2, an equality constraint through its Lagrangian, normalization as a constraint, the maximum-entropy derivation that produces the exponential family at a GC target of 0.6, the KKT conditions on a bounded quadratic, and the same likelihood problem solved on logits through the softmax.

The flags

Every script takes --steps, --seed, --out and --quiet, so you can learn the flags once and use them across the course. --steps sets the iteration count of each script, and every flag a script takes changes a line of its report.

Flag	Where it changes the report
--steps	probability.py, repeats of the ten-sample Monte Carlo estimate, default 5000. linear_algebra.py, iterations of p_{t+1} = P^T p_t, default 10. calculus.py, Euler steps along the worked flow, default 200. fields.py, Euler-Maruyama steps over [0, 1], default 200. information.py, repeats of the ten-sample cross-entropy estimate, default 5000. optimization.py, gradient-descent steps, default 600.
--samples	probability.py, the largest Monte Carlo sample size, default 1000. fields.py, paths simulated along the SDE, default 20000. information.py, the draws behind the sampled cross-entropy and the random distribution pairs, default 20000. optimization.py, the competing distributions drawn at the same GC target, default 4000.
--grid	calculus.py and fields.py, the points used in the numerical checks, default 1000.
--seed	The Monte Carlo draws, the simulated paths and the random distributions. The same seed gives the same report.
--out	A directory for config.json and report.json, written through cis6270.runner.save_run.
--quiet	Print the closing report and nothing else.

What a default run prints

Script	Headline numbers
probability.py	E[h(X)] = 1.1000, Monte Carlo estimate 1.1050 at N = 1000, Var(h(X)) = 0.6900, spread of 5000 ten-sample averages 0.2616 against the predicted 0.2627, rho_XY = 0.9449, p(X = 1 | Y = 1) = 0.8182, association 0.0960.
linear_algebra.py	softmax(z)_1 = 0.6652, lambda_1 = 3.0000, x_10 first entry 118099, p*_1 = 0.6000, second eigenvalue 0.5000, e^{tA}_11 = 1.2275 at t = 0.1, truncation error 0.0002.
calculus.py	cos(1) = 0.5403, det J_phi = 4.0000 at x_1 = 2, p_Y = 0.1667 inside the image of A, log|det J| = 3.1781 for the composition, continuity residual 0.0000, log p = -1.0000 along the flow at t = 1.
fields.py	s(3) = -3.0000, s(5) = -1.0000 under N(1, 4), Laplacian 8.0000 on the bowl and 0.0000 on the saddle, p_1(0) = 0.3989, d p_t(0)/dt = -0.1995, D times the Laplacian -0.1995, every residual 0.0000, sample variance 1.0130 at t = 1 over 20000 paths.
information.py	H(p) = 0.5004, H(p, q) = 0.5919, KL(p || q) = 0.0915, KL(q || p) = 0.1046, spread of 5000 ten-sample estimates 0.0502 against the predicted 0.0513, empirical loss 1.2799 nats, uniform loss 1.3863 nats, KL(p_hat || uniform) = 0.1064.
optimization.py	theta = 3.0000 after 600 steps, lambda = -1.0000 on the equality constraint, lambda = 0.4055 for a GC target of 0.6, H(p*) = 1.3662 nats, best drawn competitor 1.3661 nats, lambda = 4.0000 on the active bound, softmax descent loss 1.2799 nats.

Every one of these runs in a few seconds on one core, and only the sampled rows move with --seed. probability.py --steps 50000 closes the measured spread to 0.2624 and information.py --steps 50000 closes it to 0.0510, at the cost of a second each.

What is simplified

• Every script runs in double precision. Chapter 1 quotes five decimal places in places, and the training lectures run in single precision, which cannot reach them.
• The Monte Carlo sections redraw their samples, so the sampled values move with --seed. The chapter's own ten-sample draw gave 1.2 against the exact 1.1, and each run draws its own ten samples, reproducing the exact value and the 1/sqrt(N) error scale.
• We check the continuity equation by central differences at interior points of the support of p_t, away from the two ends, where a one-sided difference would be needed. The Fokker-Planck residual of fields.py uses the same differences, with a second difference for the Laplacian, so it sits near 1e-8 and not at zero.
• fields.py takes the Brownian increment of dX_t = b_t(X_t) dt + sigma_t dW_t as a scaled Gaussian draw per step. Chapter 3 defines Brownian motion properly. With a constant drift and a constant noise magnitude the Euler-Maruyama walk has the exact law of the chapter's N(bt, t), so the gap the script prints is Monte Carlo error alone.
• We integrate the branch density 1/(2 sqrt(y)) numerically over [0.01, 1], because the integrand grows without bound at the origin. Its value over the full [0, 1] comes from the antiderivative the chapter gives.
• The random competitors in the maximum-entropy check and the random pairs in the KL check confirm the chapter's inequalities on a finite sample, and the chapter itself gives the proof in full. The competitors cover the two free splits of the GC-constrained family, one inside {A, T} and one inside {C, G}, drawn independently.
• Each script has its own three-line show helper for the side-by-side printing and its own WORKED dictionary of computed values. Both stay in the script and out of cis6270/, because Lecture 1 alone prints a chapter number beside a computed one.