Research Track
You create new knowledge, not just apply it.
- Reader — Summarise and critique any paper in your area
- Reproducer — Reimplement a published result and match the numbers
- Contributor — A defensible novel increment — workshop or short paper
- Lead — Drive a coherent multi-paper research agenda
- Principal / Faculty — Define directions, advise, set the field
- Branches from
- Data Scientist (Stage 2)
- Timeline
- 2+ years, pursued alongside or after Stage 2
What this track actually is
The research track is not a job title. It is a mode of operation — a disposition toward open questions, rigorous evaluation, and the slow accumulation of genuine understanding. It branches from Stage 2 because that is the first point where you have enough mathematical and methodological foundation to read the primary literature seriously.
Unlike the engineering stages, where progress is measured by systems shipped, research progress is measured by understanding produced. The artefacts look different: papers, proofs, reproductions, ablation tables, and eventually novel results. The time horizon is different too — a PhD is typically four to six years, and a single publishable result may take six months of failed attempts.
The track runs in parallel with engineering, not after it. Many productive ML researchers are also capable engineers; many ML engineers develop genuine research taste. You do not have to choose. You do, however, have to be honest about the difference between building things and understanding things — and invest in both deliberately.
§7Section 3
Core research skills
These are the craft skills of research — the things every productive researcher does, regardless of subfield. The depth tags below indicate the minimum level required to contribute work to the research community.
- The 3-pass paper reading methodProduction
Reading papers front-to-back once is the wrong strategy — it is slow, you miss the contribution, and you retain almost nothing. Keshav's 3-pass method (skim title/abstract/conclusion → read carefully skip proofs → study proofs with paper and pen) is the professional standard. Apply it to every paper from day one.
- Literature survey and synthesisExpert
A research question already answered is not a research question. The literature survey is the first act of every serious project — finding the canonical papers, tracing their citations, identifying the gap your work addresses. This skill compounds: the more papers you have read, the faster you recognise novelty.
- Reproducibility and honest benchmarkingExpert
ML has a reproducibility crisis. Reported numbers often depend on undisclosed hyperparameter sweeps, cherry-picked seeds, or mismatched baselines. Matching a paper's numbers exactly (same data split, same seed, same metric) is surprisingly hard — and the discipline of doing it exposes exactly where results are fragile.
- Experimental design and ablationsExpert
A result without ablations is a demonstration, not a finding. Ablations answer 'which component actually caused the gain?' — they isolate variables the same way a controlled experiment does. A well-designed ablation table is often more valuable than the headline result.
- Statistical rigour in evaluationExpert
Reporting a single mean across three seeds is not rigour. The standard is: report mean and standard deviation across ≥5 seeds, compare against the strongest relevant baseline, and use a significance test if the margins are tight. Without this, you cannot distinguish signal from luck.
- Scientific writing and the LaTeX toolchainProduction
The clearest ideas in the worst prose get rejected. Academic writing has specific conventions — abstract structure, related work framing, contribution list, limitations section — that you learn by reading many papers and writing under review. LaTeX, BibTeX, and Overleaf are the required toolchain; learn them before you need them.
- The peer review processCompetent
Understanding how your work will be reviewed changes how you write it. Reviews are typically double-blind; reviewers allocate 1–3 hours to a paper; a paper that does not clearly state its contribution in the first two pages will not recover. Reviewing other papers yourself — as a sub-reviewer — is the fastest way to learn this.
- Conference and journal venue landscapeCompetent
Not all venues are equal. In ML: NeurIPS, ICML, ICLR are the top conferences; JMLR and TMLR are the top journals (TMLR is open, rolling). Subfield venues: ACL/EMNLP (NLP), CVPR/ECCV (vision), AAAI, UAI. Workshop papers are a legitimate entry point. Choosing the right venue for a contribution matters — over-targeting wastes time, under-targeting limits visibility.
- Research ethics and the sociology of scienceProduction
Fabrication and p-hacking end careers and set back entire fields. Practical ethics: never tune on the test set, always run ablations against the right baseline, attribute correctly, disclose negative results in papers. The harder questions — bias in datasets, dual-use implications, who benefits — are increasingly expected in published work.
- Building a research identityCompetent
Research is a social enterprise. An arXiv preprint, a GitHub repo that others cite, a presence at the right workshops — these compound over years. Start early: put code on GitHub, put preprints on arXiv, engage with the community. An advisor matters enormously — choose for research agenda fit and mentorship quality, not prestige alone.
§7Section 4
Mathematics: the non-negotiable gate
The site is deliberately blunt about this: math depth is the gate between applying ML and understanding it. The /mathematics curriculum covers the full ladder. Below is what this track specifically requires, beyond what Stages 0–4 already demand.
Minimum: solid through the core trio
You need linear algebra through SVD and eigendecompositions; calculus and matrix calculus through Jacobians and the chain rule in vector form; probability through MLE, MAP, Bayesian inference, and conditional distributions. Without these, you cannot read the proofs in PRML or follow a theory talk at a top conference.
Research-grade: the advanced tier
These are the topics in Tier 3 of the math curriculum. Build them in parallel with your research work — you will need each one as it becomes relevant to your subfield:
- Real analysis essentials: Sequences, continuity, convergence — the foundation of everything that says 'this converges' or 'this is smooth.'
- Measure-theoretic probability: Needed for theory papers involving expectation over function classes, ergodic arguments, or concentration inequalities.
- Information theory: Entropy, KL divergence, mutual information — ubiquitous in deep learning theory, VAEs, RL, and compression.
- Statistical learning theory: PAC learning, VC dimension, Rademacher complexity, generalization bounds — the mathematical language for asking 'why does this generalise?'
- Advanced optimization: Duality, non-convex landscapes, saddle points, second-order methods — needed for understanding why gradient descent works, when it does not, and what the alternatives are.
- Probabilistic graphical models: Bayesian networks, Markov random fields, variational inference — the language of structured uncertainty; PRML and Murphy are the references.
§7Section 5
Tools and scientific craft
Research has its own tool stack. These are the ones that matter regardless of subfield.
The mandatory writing toolchain. LaTeX for typesetting; BibTeX/natbib for citations; Overleaf for collaboration. Learn it before you need to submit a paper — the learning curve at deadline time is brutal.
The three-tool literature stack. arXiv for preprints (most important ML work appears here first); Google Scholar for citations and 'cited by'; Semantic Scholar for influence graphs and related-work discovery.
The fastest way to check SOTA and find reproducible implementations. Benchmark tables with linked code — use it before starting any baselines work.
Git for code; DVC (Data Version Control) or similar for tracking experimental configurations, datasets, and model checkpoints. A reproducible experiment requires versioning both code and data.
Jupyter for exploration and visualisation; Python scripts for reproducible experiments (notebooks are not reproducible — scripts with fixed seeds are). The workflow: explore in a notebook, crystallise into a script, run sweeps from the script.
A reference manager for your personal paper library. Start using one immediately — the cost of not having an organised library compounds painfully once you are trying to write a related-work section.
§7Section 6
The project: reproduce → extend → write
The canonical research project arc has three acts. Each act is a legitimate endpoint — not everyone who starts research reaches the third.
Act 1 — Reproduce
Reader → Reproducer4–8 weeksChoose a paper in your area with publicly available code and data. Your goal: match the reported numbers exactly — same evaluation metric, same test split, same model configuration. Do not stop at 'close.' If your numbers differ, find out why.
Reproduce without using the authors' code — implement from the paper description only. Then compare your implementation against theirs and document every discrepancy.
Act 2 — Extend
Reproducer → Contributor8–16 weeksTake your working reproduction and introduce one well-motivated modification: a different architecture, a better baseline, a new evaluation dimension, an application to a new domain. Run a full ablation table. Write up the extension with a proper related-work section.
Submit the extension to an appropriate workshop (NeurIPS/ICML/ICLR workshops, domain-specific workshops). A rejection with reviewer feedback is highly valuable — treat it as free expert mentorship.
Act 3 — Write and submit
Contributor → Lead8–16 weeksIf Act 2 produced a defensible novel result, write it up to full-paper standard using the venue's template. Get feedback from at least two people outside your immediate team before submission. Submit to the appropriate venue.
Build a coherent agenda: a set of 2–3 related papers that together address a larger question. This is what transitions a contributor into a lead — a multi-paper arc, not isolated results.
§7Section 7
Resources
Research draws on resources from every prior stage — the books below are the research-grade step up from ISLR and d2l. The key papers are the ones you must have read before claiming familiarity with the relevant area. The tools and practice resources are how you stay current. MML then ESL then PRML is the canonical reading sequence before tackling Murphy; Goodfellow is the reference for deep learning theory.
Books
Deisenroth, Faisal & Ong · 2020
The single best bridge between undergraduate mathematics and ML — covers linear algebra, calculus, and probability in the context of ML applications.
Use this if: You need to strengthen your mathematical foundations alongside ISLR or before tackling deep learning.
Reviewed 2026-06-11
Hastie, Tibshirani & Friedman · 2009
The rigorous big brother of ISLR — dense, mathematically demanding, and necessary if you want to understand WHY the algorithms work at a deep level.
Use this if: You have finished ISLR and want to go deeper into the mathematical foundations, or you are preparing for research.
Reviewed 2026-06-11
Christopher Bishop · 2006
The Bayesian-flavoured ML canon — rigorous, comprehensive, and now free from the author; the probabilistic graphical model chapters are the best treatment available.
Use this if: You are moving into research and want the mathematical depth that ISLR does not provide — particularly Bayesian methods.
Reviewed 2026-06-11
Stephen Boyd & Lieven Vandenberghe · 2004
The optimisation reference — comprehensive, mathematically rigorous, and free; most practitioners need chapters 1–5; researchers need the rest.
Use this if: You want to understand why gradient descent works, when it does not, and what the theory of convex optimisation actually says.
Reviewed 2026-06-11
Goodfellow, Bengio & Courville · 2016
The theory canon of deep learning — rigorous, dense, and essential for understanding WHY neural networks work; not a first read, but an important eventual reference.
Use this if: You have practical DL experience and want to understand the theoretical underpinnings — backprop, regularization, optimization — at depth.
Reviewed 2026-06-11
Kevin Murphy · 2022
The modern research reference — encyclopaedic, up-to-date through 2022, and freely available; Vol. 1 covers foundations, Vol. 2 covers advanced methods.
Use this if: You are doing research and need a comprehensive, current reference — Bishop is the classic, Murphy is the modern standard.
Reviewed 2026-06-11
Papers
Vaswani et al. · 2017
The paper that introduced transformers — you need to have read this; the architecture is now everywhere and the original is still the clearest description.
Use this if: You want to understand how transformers work from the original source, before reading any second-hand explanation.
Reviewed 2026-06-11
Chen & Guestrin · 2016
The paper behind the algorithm that still wins tabular Kaggle competitions — surprisingly readable and explains the regularised objective clearly.
Use this if: You use XGBoost and want to understand the algorithm, not just the library — or you are preparing for technical interviews.
Reviewed 2026-06-11
Devlin et al. · 2018
The paper that established masked language modelling as the pretraining paradigm — still the canonical read for understanding BERT-family models.
Use this if: You are working with encoder-based models or want to understand the pretraining/fine-tuning paradigm from source.
Reviewed 2026-06-11
Brown et al. · 2020
The GPT-3 paper that demonstrated in-context learning at scale — important for understanding why LLMs behave the way they do.
Use this if: You want to understand the origins of prompt engineering and in-context learning from the research that defined them.
Reviewed 2026-06-11
Wei et al. · 2022
The paper that formalised chain-of-thought prompting — short, clear, and essential for anyone doing reasoning-heavy LLM work.
Use this if: You are designing prompts for complex reasoning tasks and want the research basis for why step-by-step thinking helps.
Reviewed 2026-06-11
Lewis et al. · 2020
The original RAG paper — explains the retrieval-generation architecture that underpins most production LLM knowledge systems today.
Use this if: You are building a RAG system and want to understand the original architecture and its trade-offs.
Reviewed 2026-06-11
Hu et al. · 2021
The paper that made fine-tuning accessible — introduces the low-rank decomposition trick that is now the standard PEFT method.
Use this if: You are fine-tuning LLMs and want to understand why LoRA works and when it is appropriate.
Reviewed 2026-06-11
Ouyang et al. · 2022
The InstructGPT / RLHF paper — explains how models are aligned with human preferences; essential for understanding why modern LLMs behave so differently from raw language models.
Use this if: You want to understand RLHF and instruction tuning from the original research.
Reviewed 2026-06-11
Docs & Practice
S. Keshav · 2007
Three pages that will save you hundreds of hours — the 3-pass method is the correct way to read research papers and almost nobody teaches it.
Use this if: You are starting to read ML papers and feel like you are drowning — read this first, once, and apply it every time.
Reviewed 2026-06-11
Lilian Weng
Exceptional deep-dives on agents, LLMs, and hallucination — among the best technical writing in ML; dense but worth every word.
Use this if: You want rigorous, well-sourced overviews of a specific LLM topic (agents, RLHF, attention, hallucination) beyond what a course covers.
Reviewed 2026-06-11
Meta AI
The best place to find state-of-the-art results with reproducible code — benchmark tables, leaderboards, and linked implementations in one place.
Use this if: You want to find the current SOTA on a benchmark, or you want a paper's official implementation rather than a community reimplementation.
Reviewed 2026-06-11
Distill team (archive)
An archive of exemplary ML explanation — interactive, visually precise, and a high bar for what clear technical communication looks like; no longer actively publishing, but everything there is worth reading.
Use this if: You want the clearest possible explanation of a specific concept (attention, circuits, feature visualisation) that Distill covered.
Reviewed 2026-06-11
Cornell University
The preprint firehose — every important ML paper appears here before any journal; the skill is learning to filter signal from noise.
Use this if: You are tracking the field and want to stay current — but build the filtering skill first, or you will drown.
Reviewed 2026-06-11
§7Section 8
How you know you're done — exit criteria per rung
Unlike the engineering stages, there is no single “done.” There are rungs — and each rung has concrete, testable criteria.
Reader
- You can read a paper in your area and accurately summarise its contribution, methodology, and limitations — without looking at someone else's summary.
- You can identify what is novel versus what is incremental in a paper.
- You can apply the 3-pass method reliably: first pass ≤ 10 minutes, full read ≤ 2 hours.
Pick a paper from a top venue published in the last 6 months. Without reading any blog post or summary: what is the core claim? What is the key experiment? What would have to be true for the claim to fail?
Reproducer
- You have matched the headline numbers of at least one paper within expected variance (typically ±1–2% for classification metrics).
- You have documented every decision needed to reproduce the result: data split, random seed, hyperparameter choices, evaluation protocol.
- You have identified at least one thing the paper does not tell you that turned out to matter.
Hand your reproduction code and README to someone unfamiliar with the paper. Can they reproduce your numbers without asking you any questions?
Contributor
- You have produced at least one result that is novel — not just better numbers on a known benchmark, but a genuinely new finding or a well-motivated new setting.
- You have run a full ablation table that isolates the contribution of each design choice.
- You have a written draft at workshop-paper length (4–6 pages) that someone outside your lab has read and critiqued.
If a reviewer asked 'why does your modification help?' — can you answer with evidence from your ablations, not intuition?
Lead
- You can articulate a research agenda — a set of 2–3 connected questions where answering one opens the next.
- You have mentored at least one more junior researcher through Act 1 or Act 2.
- You have a publication record at recognisable venues.
In a 5-minute chalk talk: what is the open problem you are working on, why is it important, and what is your plan for the next 12 months?
Principal / Faculty
- Others cite your work. Others build on it.
- You can attract talented people to your agenda.
- You have shaped the framing of a problem in your area — not just answered questions, but changed which questions are asked.
Could you write a compelling 2-page research statement that a hiring committee at a strong research institution would take seriously?
§7Section 9
From here
The research track has no defined terminal stage — Principal/Faculty is a career endpoint, not a curriculum one. From the Contributor rung onward, your development is driven by the field itself: follow the conferences, read arXiv regularly, and build community. The map below connects back to the full journey.