Before going to study about the deep learning algorithms, let’s understand what deep learning is all about. We will start with the definition of machine learning first, machine learning is described as a kind of artificial intelligence where computers learn to perform something without having programmed to do it. There are various methods in which machine learning programs can be trained and deep learning is one of them. From this post, you will learn what deep learning is all about and gain knowledge on various deep learning algorithms.  

What is Deep Learning?

Deep learning is a part of the family of machine learning methods based on learning data representations. It is a technique that trains computers to do what occurs naturally to humans: learning through examples. It is the major technology behind autonomous cars, allowing them to identify a stop sign, or to distinguish a lamppost from a pedestrian. It is also the key to voice control in various consumer devices such as hands-free speakers, TVs, tablets, and phones.

In Deep Learning technology, a computer model learns to execute classification tasks directly from text, sound, or images. Deep learning models can attain state-of-the-art accuracy and, at times, even surpassing human-level performance. Models are trained with the help of neural network architectures and a huge set of labeled data that consist of many layers.

Importance of Deep learning

Accuracy is what separates deep learning from other technologies. Deep learning attains recognition accuracy at a very high level than ever seen before. This enables consumer electronics to  meet the expectations of the user, and it is most crucial for applications where safety is a major concern, like driver-less cars. The recent innovations in deep learning have enhanced to the point where in few tasks such as classifying objects in images, deep learning outperforms humans.

Deep Learning Process

Most of the deep learning methods make use of neural network architectures. Because of this, deep learning models are usually referred to as deep neural networks. The term “deep” generally points out to the number of hidden layers in a neural network. The deep networks can have around 150 hidden layers while traditional neural networks contain only 2 to 3 hidden layers. The models of deep learning are trained with the help of neural network architectures and huge sets of labeled data that learn features from the data directly without requiring manual feature extraction.

neural networks

Neural networks are organized in layers and consist of a set of interconnected nodes. Networks can contain tens or hundreds of hidden layers.

One of the most popular kinds of deep neural networks are CNNs (Convolutional Neural Networks) . A Convolutional Neural Network convolves input data with learned features and utilizes 2D convolutional layers, thus making this architecture highly suited to processing 2D data, like images.

Convolutional Neural Networks eliminate the need for feature extraction manually. So, you don’t need to detect features utilized to classify images. The convolutional neural network works by directly extracting features from images. The relevant features are learned but not pre-trained while the network trains on a collection of images. Thus, this kind of automated feature extraction enables deep learning models to be highly accurate for computer vision tasks like object classification.

image-classification

Credit: Verhaert

Standard Image Classification Approach to Deep Learning

Deep Learning Algorithms

Deep learning algorithms automatically learn feature representations (usually from unlabeled data), thus avoiding a large amount of time-consuming engineering. These algorithms are dependent on developing huge artificial neural networks that were inspired loosely by brain (cortical) computations. Below are the types of deep learning algorithms:

  • Deep Belief Network(DBN)

A Deep Belief Network is a class of deep neural network or a generative graphical model comprising numerous layers of latent variables(i.e. Hidden units) with connections in-between the layers but not units within every layer. They can be viewed as a composition of unsupervised, simple networks i.e. Sigmoid Belief Networks + Restricted Boltzmann Machines.

The major advantage of DBNs its ability of “learning features,” which is attained by a ‘layer-by-layer’ learning strategies whereas the higher level features are learnt from preceding layers. Often, they are utilized to initialize deep discriminative neural networks(a method known as generative pre-training). They are generally used for unsupervised pre-training. In this training, the network initially trains a generative model, and utilizes it to initialize a discriminative model.

  • Generative Adversarial Network(GAN)

Generative Adversarial Networks(GANs) are a class of Artificial Intelligence algorithms used in deep learning. They are implemented by a system of 2 neural network models that contest with each other in a zero-sum game framework. The 2 neural network models are known as the Generator and the Discriminator models. The generator model generates samples by taking noise as input.

The discriminator receives samples from both the training data and the generator and has to be able to differentiate between the two sources. A continuous game is played by these 2 networks where the generator is learning to yield more realistic samples while the discriminator is learning to get better at differentiating real data and generated data. These two networks are simultaneously trained and the competition will steer the generated samples to be indistinguishable from the real data.   

GANs can generate photographs that look authentic to the observers. Take, for example, a GAN can be utilized to create a cat’s synthetic photograph that deceives the discriminator into accepting it as a real photograph.

  • Backpropagation Algorithm

Backpropagation is an algorithm that is used in artificial neural networks for calculating each neuron’s error contribution after processing a batch of data (in image recognition process). The gradient descent optimization generally uses it for adjusting the weight of neurons by calculating the loss function’s gradient.

The backpropagation algorithm has repeatedly been rediscovered and is equivalent to automatic differentiation in the reverse accumulation mode. For each input value, backpropagation needs a desired and known output. So, it can be stated as a supervised learning method. It can be used with any optimizer based on gradients like truncated Newton or L-BFGS. Backpropagation is generally used for training deep neural networks (neural networks containing more than one hidden layer).

  • Softmax Regression Algorithm

Generalizing logistic regression to the classification problems where we want to handle multiple classes is termed as Softmax Regression (also known as Multi-class Logistic Regression, Maximum Entropy Classifier Regression, or Multinomial Logistic Regression). The labels are binary in logistic regression. It is used for problems like MNIST digit classification where the goal is to differentiate between ten distinct numerical digits. Softmax regression enables us to handle y(i){1,…, K} and K represents the number of classes.

  • Convolutional Neural Network (CNN) Algorithm

The convolutional neural network algorithm is a multilayer perceptron that is the particular design for identifying the 2-D image formation. It has an input layer, output layer, and numerous hidden layers. The hidden layers contain normalization layers, fully connected layers, pooling layers, and convolutional layers.

The convolutional layers apply the convolution operation to the input, and they pass the result to the next layer. The individual neuron’s response is emulated by the convolution to visual stimuli. The convolutional networks incorporate global or local pooling layers which combine the neuron clusters’ outputs at one layer into a single neuron in the next layer.

Fully connected layers are used for connecting neurons of different layers. The benefit of CNNs is that they are quite easy to train and consist of fewer parameters when compared to fully connected networks with the same number of hidden units. Explicit feature extraction can be avoided in CNNs.

Conclusion

Deep learning architectures like recurrent neural networks, deep belief networks, and deep neural networks have been applied to fields such as natural language processing, machine translation, speech recognition, computer vision, and so on. These architectures have generated results in such a way that they were superior to human experts in some cases.

Deep learning is taking machine learning very much closer to its original goal; artificial intelligence. Deep learning algorithms learn high-level features from data and this is a major step ahead of conventional machine learning. To get an insight on deep learning algorithms, there are various sources available on the internet such as e-books, websites, and so on. They provide the information related to deep learning methodology and the latest advancements in this field.