Deep learning was conceptualized by Geoffrey Hinton in the 1980s. He is widely considered to be the founding father of the field of deep learning. Hinton has worked at Google since March 2013 when his company, DNNresearch Inc., was acquired by the company.
Hinton's main contribution to the field of deep learning was to compare machine learning techniques to the human brain.
More specifically, he created the concept of a 'neural network' - which is a deep learning algorithm structured similar to the organization of neurons in the brain. Hinton took this approach because the human brain is arguably the most powerful computational engine known today.
The structure that Hinton created was called an artificial neural network (or artificial neural net for short). Here's a brief description of how they function:
Artificial neural networks are composed of layers of node
Each node is designed to behave similarly to a neuron in the brain
The first layer of a neural net is called the input layer, followed by hidden layers, then finally the output layer
Each node in the neural net performs some sort of calculation, which is passed on to other nodes deeper in the neural net
Here is a simplified visualization to demonstrate how this works:
Neural nets represented an immense stride forward in the field of deep learning.
However, it took decades for machine learning (and especially deep learning) to gain prominence.
We'll explore why in the next section.
Why Deep Learning Did Not Immediately Work
If deep learning was originally conceived decades ago, why is it just beginning to gain momentum today?
It's because any mature deep learning model requires an abundance of two resources:
At the time of deep learning's conceptual birth, researchers did not have access to enough of either data or computing power to build and train meaningful deep learning models. This has changed over time, which has led to deep learning's prominence today.
In this tutorial, you had your first introduction to the field of deep learning.
Here's a brief summary of what we discussed:
The history of deep learning
Geoffrey Hinton's role in the early days of deep learning as a field
The two factors that have historically limited progress in deep learning: data and computing power