The following content from edX course Machine Learning taught by Andrew Ng.
Introduction to Neural Networks
Welcome to week 4! This week, we are covering neural networks. Neural networks is a model inspired by how the brain works. It was very widely used in 80s and early 90s; popularity diminished in late 90s (computationally expensive).
Recent resurgence: State-of-the-art technique for many applications. It is widely used today in many applications: when your phone interprets and understand your voice commands, it is likely that a neural network is helping to understand your speech; when you cash a check, the machines that automatically read the digits also use neural networks.
Why we need a new algorithm?
Feature “explosion” for logistic regression. Neural network will hopefully help solving this issue.
The “one learning algorithm” hypothesis
How it can solve complex non-linear problem
If network has units in layer j, units in layer j+1, then will be of dimension
This is very important to know or memorize later on for doing vectorized implementation. Or see pop quiz below.
Non-linear classification example
Multiple output units: One-vs-all