Implementing (parts of) TensorFlow (almost) from Scratch

A Walkthrough of Symbolic Differentiation

This literate programming
exercise will construct a simple 2-layer feed-forward neural network to compute
the exclusive or, using symbolic
differentiation to
compute the gradients automatically. In total, about 500 lines of code,
including comments. The only functional dependency is numpy. I highly recommend
reading Chris Olah's Calculus on Computational Graphs:
Backpropagation for more
background on what this code is doing.