Krys Newman/AI Frontier Search

Self-study · Machine learning

AI Frontier

Notebooks from a self-study path through neural networks and machine learning, built from scratch, in order.

Notebooks

01

01 · From Derivatives to a Backprop Graph

Building up to backpropagation from the definition of a derivative: first estimating slopes numerically, then wrapping scalars in a Value object that records how it was computed, and finally walking that graph backwards by hand to get every gradient.

A function to differentiateWhat the function looks likeThe derivative, numericallyDerivatives with more than one inputThe Value object+4 more

Reference

→

Reference notebooks

Public calculus and linear algebra notebooks worth keeping open alongside these: diagram-heavy, notation-first, and happy to use numpy, matplotlib and autodiff rather than rebuild them. Six are committed to the repo and runnable locally; the rest are one fetch away.

Linear algebraCalculusBoth, in one place

Each notebook builds its tools by hand before reaching for a library. The curriculum they follow is tracked separately at Backprop to Frontier.