A friend of mine went to a talk on Friday at McGill on neural networks and couldn’t stop talking about it afterwards, so I figured I’d share with you guys. Geoffrey Hinton showcased his algorithm for handwriting recognition using neural networks. That in itself is nothing to write home about, these algorithms have been around for years, but his method is quite innovative. Basically it involves stacking several neural networks one behind the other so that the higher level neural networks learn from the lower level ones and detect higher and higher level features. Now the crazy thing is that you can look at the results and easily interpret (at least for the first few levels) exactly what the features are that the neural network is looking at to make it’s decision.

Training the network is also apparently very easy in this scheme, based on the back-propagation algorithm. There is also some interesting concepts in there about bayesian statistics, a Gaussian error function, Ising models and free energy minimization.

Here’s the really cool thing: there’s a Flash demo right here. To showcase the recognition, click on one of the digits in the bottom half of the left hand side, and click play. In the middle left of the right-hand side region, you have the decision network which is composed of ten neurons, and the highlighted one is the one that the network thinks it recognizes. Sometimes for badly formed input, it will go back and forth between two decisions that equally make sense.

Now here’s the kicker: running the network backwards will produce a ‘typical digit’. Click on one of the digits in the top left and click play. You will see a digit form itself in the bottom right window. Is that not extremely cool? If you’re interested in the physics and math and computer science behind it, take a look at the slides and the paper.