What is Deep Learning and Neural Nets?

Neural Nets deep learning

HPC Basics Series

Deep learning is a term used to describe a family of methods in machine learning that focus on the creation of a classification or entrance requirement to satisfy the comprehend component of the “sense, comprehend, and act” definition of artificial intelligence put forth by researchers at Accenture. Let’s start off with a given, the goal of an Artificial Intelligence is to first create an n-dimensional category then apply some logic to it. So, we have something that at a certain level of abstraction looks like this…

Category(x) = {x|x satisfies some model}

foreach j in list{
if list{j} = Category{x}{
Do something

Do something else


Let’s take a heterogeneous, but an intuitively simple example for exploring deep learning and classification with a deep neural network, shoes. Let’s pretend we want to create a model that can recognize shoes. Well, first we must define what is a shoe…

Neural Nets

As we can see, shoes come in many different shapes, sizes and are specialized by purpose. If we attempted to make a single category out of these representative examples our model would show specificity only as far as something that is roughly foot-shaped, and even then, it would be a rather vague approximation. So, how on earth do we develop a classification system for something as universal and diverse as a shoe?

One possible answer is to build out Category(x) for our shoes in the form of a deep neural network. A deep neural network is a hierarchical graph that represents different descriptive aspects of a shoe. It can also be thought of as a functional ontology with the addition of weights along the edges. The descriptive aspects are all things that can be computed or observed.

A concrete example: Let’s take this toy network to describe a shoe in the most basic of terms…

Upper →Sole→(Heel or No Heel)

As you can see, all the pictures shown above listed conform to this general categorization and would be categorized as shoes because all of the examples would evaluate to ‘true’ when tested for having an “Upper”, having a “Sole”, and having either a “Heel or No Heel” and for every time our test evaluates to true we could increment a counter or score. In all cases all our shoes would receive a score of 3 (Upper + Sole + Heel or No Heel = 3).

Now, what if we wanted to create a subcategory of shoe from this called “Slipper”, where “Slippers” were defined as shoes without heels. That’s pretty easy, all we’d have to do is set true for “No Heel” to be equal to something greater than 1 and we could then discriminate between “shoes” and “slippers” because shoes would score 3 and slippers would have a score greater than 3.

In a nutshell, this is how deep neural networks operate, they are a multilayered set of ‘true’ or ‘false’ criteria that have values associated with their evaluation. The difficulty comes in determining the weighting for each of the true or false evaluations within the network. This is where training comes in.

Neural Networks are trained by establishing weights on the evaluation edges. To train our network to identify Shoes and Slippers, we’d need a set of some arbitrary size that was correctly labeled as “Shoes” and “Slippers”, this set would then be run against the network and the network would adjust its weighting of each of the edges (evaluation to true) until it could accurately identify and sort Shoes from Slippers. Once this training has been successful, it then needs to be tested on another known set and the accuracy evaluated.

Deep learning and neural nets is all part of a day’s work for our clients at Nimbix. Our purpose-built environment, with the fastest computational accelerators, delivers the computer power necessary to complete these types of workloads fast – delivering faster time to results.