Acoustics to Deep Learning

Supporting Code

I'm starting a new series today on Acoustics to Deep Learning. Rather than simply discuss tools (Deep Learning, SVM...), I've decided to present a bunch of interesting techniques in the context of audio and sound. 

View fullsize

 While we’re covering a lot of ground here, in some cases in substantial depth, this series is not meant to be exhaustive or even thorough. I’m going to talk about what I think is interesting and compelling with as much depth and clarity I can squeeze into a YouTube video. This is mostly because I believe that with the number of resources available, being thorough is a waste of our time. Finally, we will cover some serious computational techniques here, but only in the context of examples. I think it’s important to remember that as cool as tools and techniques are, they are only, at most, a means of accomplishing something. The techniques shown here were not developed in a vacuum, but in the rich and complex world of application, and I believe that presenting tools and techniques in the absence of the appropriate context does you a disservice, and can even hinder learning – the why is just as important as the how.

Neural Networks Demystified, Part 7: Overfitting, Testing, and Regularization

We've built and trained our neural network, but before we celebrate, we must be sure that our model is representative of the real world. We'll look at ways to diagnose and fix overfitting.

Supporting Code

Nate Silver's Book

Caltech Machine Learning Course

And the lecture shown

A big thank you to everyone who watched and commented, I really enjoyed the process and I'm excited to learn more and make better videos in 2015. Next video release will be on Friday, February 27. 

Neural Networks Demystified, Part 6

After all that work it's finally time to train our Neural Network. We'll use the BFGS numerical optimization algorithm and have a look at the results.  

Supporting Code 

Yann Lecun's Efficient BackProp Paper 

More on BFGS

Neural Networks Demystified, Part 5: Numerical Gradient Checking

When building complex systems like neural networks, checking portions of your work can save hours of headache. Here we'll check our gradient computations. 

Supporting code

Link to excellent Stanford tutorial 

Neural Networks Demystified, Part 4: Backpropagation

Backpropagation as simple as possible, but no simpler. Perhaps the most misunderstood part of neural networks, Backpropagation of errors is the key step that allows ANNs to learn. In this video, I give the derivation and thought processes behind backpropagation using high school level calculus. 

Supporting Code and Equations