Advanced Track

ML Research Readiness

Use this when the next step is reading stronger ML papers with less friction. The broader product can start simpler: choose a goal, find gaps, save notes, and review weak spots.

Find gaps for this track View checkpoints

Track length

8 weeks

Designed for focused preparation, with review and repair between checkpoints.

Checkpoints

Each checkpoint has topics, review targets, and a concrete evidence standard.

Final artifact

Dossier

A compact record of diagnostics, review, labs, and written technical reasoning.

Checkpoint 1

Probability and Concentration

Random variables, tails, and finite-sample guarantees.

Diagnostic question

Can you move from expectation and variance to Hoeffding or Bernstein without hand-waving?

Review deck seeds

expectation-variance-covariance-momentssubgaussian-random-variablesconcentration-inequalities

State the random object, choose the right concentration tool, and explain what the bound does and does not prove.

Expectation, Variance, Covariance, and Momentsexpectation-variance-covariance-momentsThe basic language for risk, noise, and estimator behavior.Evidence: Compute and interpret the first two moments for a model error.Sub-Gaussian and Sub-Exponential Tailssubgaussian-random-variablesThe tail classes behind modern finite-sample bounds.Evidence: Classify a random variable by tail behavior and state the consequence.Concentration Inequalitiesconcentration-inequalitiesThe bridge from random samples to high-probability guarantees.Evidence: Choose Hoeffding, Bernstein, or Markov and justify the assumptions.

Checkpoint 2

Linear Algebra and Optimization Geometry

The matrix and curvature tools used in model analysis.

Diagnostic question

Can you read a matrix-heavy proof and say what each object is doing?

Review deck seeds

eigenvalues-and-eigenvectorssingular-value-decompositionmatrix-calculusconvex-optimization-basics

Track shapes, spectra, gradients, and condition numbers well enough to debug optimization arguments.

Eigenvalues and Eigenvectorseigenvalues-and-eigenvectorsSpectral structure explains stability, covariance, and repeated linear maps.Evidence: Explain the dominant eigenvector in a repeated-update system.Singular Value Decompositionsingular-value-decompositionSVD is the cleanest language for rank, projections, and conditioning.Evidence: Use singular values to reason about low-rank approximation.Matrix Calculusmatrix-calculusBackprop and optimization proofs require shape-correct derivatives.Evidence: Derive a gradient and state the shape before simplifying.Convex Optimization Basicsconvex-optimization-basicsConvexity is the reference case for convergence and failure-mode reasoning.Evidence: State a convergence condition and identify the violated assumption.

Checkpoint 3

Learning Theory Core

ERM, capacity, uniform convergence, and learnability.

Diagnostic question

Can you connect a finite class bound to VC dimension and Rademacher complexity?

Review deck seeds

empirical-risk-minimizationvc-dimensionrademacher-complexitypac-learning-frameworkuniform-convergence

Explain why training loss can transfer to test performance under explicit assumptions.

Empirical Risk Minimizationempirical-risk-minimizationThe main abstraction behind supervised learning objectives.Evidence: Separate empirical risk, true risk, and approximation error.VC Dimensionvc-dimensionA precise way to talk about hypothesis-class capacity.Evidence: Give a shattering example and a non-shattering counterexample.Rademacher Complexityrademacher-complexityData-dependent capacity control for modern generalization bounds.Evidence: Explain what the random signs measure.PAC Learning Frameworkpac-learning-frameworkThe canonical language for sample complexity and learnability.Evidence: State epsilon, delta, and sample complexity in one sentence.Uniform Convergenceuniform-convergenceThe proof bridge from concentration to generalization.Evidence: Identify where the union bound or complexity term enters.

Checkpoint 4

Estimation and Uncertainty

Statistical estimation, calibration, and model confidence.

Diagnostic question

Can you distinguish likelihood, posterior, uncertainty, and calibration error?

Review deck seeds

maximum-likelihood-estimationbayesian-estimationbias-variance-tradeoffcalibration-and-uncertainty

Judge whether a prediction is accurate, calibrated, and supported by the right estimator.

Maximum Likelihood Estimationmaximum-likelihood-estimationMLE connects losses, probabilistic models, and asymptotic behavior.Evidence: Write the likelihood and the equivalent loss for a simple model.Bayesian Estimationbayesian-estimationPriors and posteriors explain uncertainty beyond point estimates.Evidence: Explain how a prior changes the posterior in a toy model.Bias-Variance Tradeoffbias-variance-tradeoffA durable lens for model complexity and generalization.Evidence: State the decomposition and what each term means.Calibration and Uncertaintycalibration-and-uncertaintyResearch systems need honest confidence, not only high accuracy.Evidence: Interpret a reliability diagram and name a calibration failure.

Checkpoint 5

Deep Learning Mechanics

Backprop, losses, normalization, and regularization.

Diagnostic question

Can you derive a two-layer MLP update and explain why training fails?

Review deck seeds

feedforward-networks-and-backpropagationcross-entropy-loss-deep-divebatch-normalizationregularization-theory

Derive and debug a neural network training loop instead of treating it as a black box.

Feedforward Networks and Backpropagationfeedforward-networks-and-backpropagationThe minimal neural-network computation graph.Evidence: Derive the backward pass for a two-layer MLP.Cross-Entropy Losscross-entropy-loss-deep-diveThe standard classification objective and likelihood connection.Evidence: Explain what the gradient does to the correct-class logit.Batch Normalizationbatch-normalizationA practical training tool with subtle theory and common myths.Evidence: Reject the internal-covariate-shift explanation when it is too weak.Regularization Theoryregularization-theoryExplicit controls for complexity, stability, and overfitting.Evidence: Compare L2, dropout, and early stopping in one failure case.

Checkpoint 6

Transformers and Representations

Attention, residual streams, representation learning, and kernels.

Diagnostic question

Can you explain attention scaling, residual streams, and the NTK limit without mixing levels?

Review deck seeds

attention-mechanism-theorytransformer-architecturerepresentation-learning-theoryneural-tangent-kernel

Explain transformer components and connect representation learning to theory.

Attention Mechanism Theoryattention-mechanism-theoryThe core operation behind transformer sequence modeling.Evidence: Explain the dot-product scaling and softmax behavior.Transformer Architecturetransformer-architectureThe block-level structure behind modern sequence models.Evidence: Draw the residual stream and name what each sublayer changes.Representation Learning Theoryrepresentation-learning-theoryThe language for features, invariances, and transfer.Evidence: State what property a representation preserves or discards.Neural Tangent Kernelneural-tangent-kernelThe infinite-width reference point for lazy training.Evidence: Explain what NTK theory captures and what feature learning it misses.

Checkpoint 7

Training, Scaling, and Generalization

SGD, Adam, scaling laws, double descent, and implicit bias.

Diagnostic question

Can you tell whether a training claim is an optimization claim, a scaling claim, or a generalization claim?

Review deck seeds

stochastic-gradient-descent-convergenceadam-optimizerscaling-lawsdouble-descentimplicit-bias-and-modern-generalization

Reason about modern training behavior through optimization and empirical scaling evidence.

Stochastic Gradient Descent Convergencestochastic-gradient-descent-convergenceThe baseline for noisy optimization analysis.Evidence: Name the variance and step-size assumptions in a convergence claim.Adam Optimizeradam-optimizerAdaptive optimization is useful but not automatically safer than SGD.Evidence: Compare Adam and SGD on a condition-number or scale issue.Scaling Lawsscaling-lawsThe empirical law behind compute/data/model tradeoffs.Evidence: Explain how compute-optimal training differs from parameter scaling.Double Descentdouble-descentThe modern failure case for simple bias-variance intuition.Evidence: Explain why interpolation need not imply poor test error.Implicit Bias and Modern Generalizationimplicit-bias-and-modern-generalizationOptimization choices select solutions, not just losses.Evidence: Give one implicit-bias claim and its scope.

Checkpoint 8

Frontier Reading Artifact

Interpretability, alignment, and research-note discipline.

Diagnostic question

Can you read a frontier page and say what is proven, what is empirical, and what remains open?

Review deck seeds

mechanistic-interpretabilityinduction-headssparse-autoencodersrlhf-and-alignment

Write a technical reading note that separates assumptions, evidence, mechanisms, and open questions.

Mechanistic Interpretabilitymechanistic-interpretabilityA research area where evidence, mechanism, and speculation must stay separate.Evidence: Identify a circuit claim and the evidence behind it.Induction Headsinduction-headsA concrete circuit story for in-context pattern completion.Evidence: Explain the previous-token and induction-head composition.Sparse Autoencoderssparse-autoencodersA current method for probing superposition and feature structure.Evidence: State what an SAE feature is and how it can mislead.RLHF and Alignmentrlhf-and-alignmentA place where objective design, preference data, and failure modes meet.Evidence: Compare reward modeling, preference optimization, and reward hacking.

Labs

These are the interactive checks that keep the track from becoming reading-only. They should become evidence rows once signed-in lab tracking is wired.

High-Dimensional Probability LabConnect concentration, dimension, and failure probability through a controllable simulation.Random Matrix / Spectral Geometry LabSee eigenvalue spread, spectral norm behavior, and why matrix geometry matters for ML.

Research readiness dossier

A short evidence packet showing diagnostic results, repaired weak concepts, review history, two lab traces, and one written theorem-to-model explanation.

completed diagnostic with weak-concept repairs
daily-review streak with due-card completion
one proof sketch or derivation from the learning-theory core
one lab trace from probability or scaling behavior
one written frontier reading note with assumptions and failure modes