Understanding neural networks
Every data analyst or data scientist should know how a neural network works and where a neural network can provide a solution. It’s equally important to know the limitations of this technology. We’ll tell you all about neural networks on this page.
Introduction to neural networks
There’s a big sports complex, with fields and halls, next to a suburb in a big city. One of the sports practiced here is football (or soccer, if you like). At a given time, a group of kids are on their way to the sports complex, seemingly to play football. However, none of the kids have a sign on their bicycle that says what sport they’re going to play. The question is whether we can use some properties to determine whether these kids are going to play football or not.
To play football or not to play football
A kid carrying a new football is probably going to play football, but there’s no way to be absolutely certain. For all we know, they bought the ball as a gift. Maybe they found the ball and are going to show it to their parents. The point to be made here is that possession of a football is not enough information to make a decision.
A boy in a track suit wearing football shoes isn’t a guarantee either. The same goes for a group of children with backpacks going to the sports complex together. There’s no way to guarantee whether or not a kid is going to play football until they walk onto the pitch.
None of the properties are deterministic
In practice it’s often necessary to know beforehand how many kids are going to play football. Because none of their properties are deterministic, and the same goes for any given combination of properties, we can’t use the decision tree method. We need a different method, one that can work with combinations of properties, which will give us the answers we seek. For situations like this, we can use a neural network.
Neural networks imitate the brain
A neural network, both in its structure and function, is an imitation of our own brains. To describe how neural networks function, first we’ll illuminate the problem of the kids playing football (or not) a bit more.
The problem solved by a neural network
To answer the question of who is going to play football, first we have to determine the possible factors that can influence the answer. None of these factors are completely deterministic by themselves, as we illustrated above. The possession of a football is no guarantee that the child in question will actually play football. At the same time, a child going to the sports complex with no equipment can definitely be a football player. In order to determine who’s going to do what beforehand, we need to do careful research. In order to do this research, we’ll establish some facts about the group of children who are on their way to the sports complex.
Inspecting the data
Inspecting the data of the first twelve kids leads to the following statements:
1. Having a ball doesn’t guarantee playing football.
2. Going in a group with a ball guarantees playing football.
3. The combination of clothes and football shoes results in football.
4. Having a bicycle contains no predictive value.
Based on this data, it seems possible to predict which of these 12 children if going to play football, and which aren’t. The question that follows is: how reliable is a prediction based on this data?
Comparing the prediction to reality
For the following four children we’ll compare the prediction with their actual behavior. Using this new data, it turns out that having shoes and clothing is not deterministic, nor is going with a group with a ball.
The outcome is less obvious
The outcome of the predictions (predictive analytics) proves to be less obvious than expected, based on the first data set. To improve the prediction, we have several options:
1. Using more factors to determine the right answer. The question is which factors to add.
2. Analyzing the behavior of more subjects. There’s a good chance that this will make it harder to make accurate predictions. In the above example, adding four more children to the analysis disproved the earlier criteria thought to be accurate.
3. Accepting that the prediction won’t always be correct. In the given situation this approach is usable, but the prediction has to be a best guess.
4. Basing the prediction on a weighted average of all factors. Taking this approach means using all the available data to arrive at an answer. The difficulty here is assigning the proper weight to each factor.
In practice, only options 3 and 4 provide a solution. A combination of these two results in the best prediction. The neural network is capable of using these principles to determine the best prediction.
The neural network solution
Computers excel at processing clearly defined data according to set protocols. One of the strengths of the human brain is its capacity of working with vague, incomplete data according to often poorly-defined methods. The reason for this difference is the way in which a neuron, the processing unit of the brains, processes the data.
Neurons fire off signals
A neuron collects signals from different sources, similar to a data warehouse. Which sources these are depends on earlier information for each separate neuron. If the sum of all incoming signals exceeds a certain threshold, the neuron will fire off a signal on its exit. The strength of this signal is based on the magnitude of incoming signals. The greater the sum of incoming signals, the greater the outgoing signal. By dividing information across various neurons, and by processing the outgoing signals in different neurons, the brain can process information (oftentimes poorly-defined, incomplete information).
Computers process information
How brains process information is very different from the way in which computers do. By using a neural network, a computer can simulate the functions of neurons. This means that in a neural network, the various data has to be given the proper weight to determine connections, just like neurons do in brains.
In practice there are two ways to do this:
- Predetermined: In this method, the weights for each signal is predetermined. This means that the neural network is only applicable for clearly-defined tasks.
- Adaptive: This method generates weight by combining combinations of input signals and (known) output signals.
The answer is always correct
By altering the weights of the underlying connections, the neural network makes sure that the answer is always correct. This “learning” process of the neural network is continued until the changes in weighting fall below a pre-determined threshold. Then, the network knows the situation. From that moment the network can be used for new, unknown situations. You can approach the learning process of the neural network in two ways:
- Supervised: Uses both input and output signals to calibrate the neural network properly. This method is often applied offline. The network is only applied to its intended environment once it works as it should.
- Unsupervised: The network is completely self-learning. This method of processing can be seen as training on the job.
The circumstances determine the strategy used when applying neural networks.
The three layers of a neural network
To illustrate the structure of a neural network’s learning process, we’ll take a simple neural network as an example. Just like in our brains, this network consists of three layers:
2. Hidden layer (processing layer).
3. Output layer.
This example is about an adaptive, supervised network. At first, the weighting of the various connecting are equal. The threshold above which a neuron activates is also equal. By “feeding” the neural network a large amount of situations with known results, the weighting and the thresholds are altered automatically until the outcome is correct every time. As soon as the weighting and thresholds fall below a certain pre-determined value, the neural network is ready to be used.
The adaptive network from the example can grow further by giving it a refresher course every now and then. Feeding it a large amount of situations with known outcomes once again, like combinations where the neural network’s accuracy leaves something to be desired, can increase its accuracy.
The limitations of neural networks
Like every other method of analyzing Big Data, a neural network is not without its limitations. Although a neural network can often work effectively with vague and incomplete information, the outcome can be incomplete as well. The certainty of a decision tree is lacking in neural networks. Not every data set is suitable for a neural network either.
Searching for patterns in photos
The trick is choosing applications that are suited to a neural network, like pattern detection. Neural networks can be used to great effect in facial recognition and detecting certain patterns in photos or film. Neural networks can also effectively detect uncommon diseases. Why? Because in many cases the information is incomplete and the symptoms that are tested for often overlap and are rarely complete.
Pros and cons of neural networks
All in all, neural networks have the following advantages:
- Processing vague, incomplete data.
- Settings of a neural network can be adapted to varying circumstances and demands.
- Effective at recognizing patterns (in images).
In exchange for the following cons:
- The outcome of a neural network contains some uncertainty that isn’t always desirable.
- Before a neural network can be used, it has to undergo a “learning” phase (both online and offline).
- The quality of the outcome depends greatly on the quality of the data used in the learning phase.
- The user has little influence on the function of the network (weighting and thresholds).