# Universal ResNet: The One-Neuron Approximator

*"In theory, theory and practice are the same. In practice, they are not."*

I read a very interesting paper titled *ResNet with one-neuron hidden layers is
a Universal Approximator* by Lin and Jegelka [1].
The paper describes a simplified Residual Network as a universal approximator,
giving some theoretical backing to the wildly successful ResNet architecture.
In this post, I'm going to talk about this paper and a few of the related
universal approximation theorems for neural networks.
Instead of going through all the theoretical stuff, I'm simply going introduce
some theorems and play around with some toy datasets to see if we can get close
to the theoretical limits.

(You might also want to checkout my previous post where I played around with ResNets: Residual Networks)