“Noise”

The word gives associations, like bad or loud music, or perhaps the sound of rush hour traffic.  Whatever you think of when you hear the word, it is likely something you would do very well without. It is non-relevant or uninformative signals reaching your senses. You may think that noise is no good.

As a statistician I deal with noise on a daily basis, but another kind of noise. As I wrote in an earlier post, statistics is all about separating signal from noise in observable random variables. Noise is the non-explainable part of the observed variation, whereas the signal, the explainable part, represents information. We may say that noise is the opposite of information.

Just think of temperature, a variable phenomenon. Some of this variation is explainable, and the weather forecast uses this information to predict the temperature for tomorrow. However, there is always some variation that can not be explained. That’s statistical noise!

So, there is a certain randomness in noise, and randomness is maybe something you wouldn’t think of when we talk about brains and thoughts. You probably think that you are in control of your own mind?

Think again.

No, your brain is noisy! In fact, it produces its own random noise signals!

Just to make it clear right away. The noise is good for you in many ways. For instance, it can make you more alert and react faster, it can help you make decisions when you really don’t know what to do, and…it helps you to be creative, if you allow it…

Lean back a moment and think of a beautiful scenery, like the green hills of the Lake District in England or an alpine region of the Himalayas. The landscape is not flat, but you may take a hike and walk from one valley to another. Of course, it takes more effort in the Himalayas than in the Lake District to cross from one valley to another.

Your brain network is like a landscape like this. Your thoughts, potential decisions and memories are like valleys in an energy landscape. Pathways of strongly connected neurons form low-energy valleys in an energy landscape (Hopfield, 1982). Mountains or hills form barriers between your separate memories or thoughts, helping you to keep your thought straight.

Neurons pass electrical signals to one-another, and the signaling across the brain network may me mathematically described by a system of differential equations expressing the signal flow. In mathematical terminology the low-energy states of memories are attractors and the hillsides form the basins of attraction. Just as falling rain will flow down the hillsides to the bottom of the valley, neuronal signals entering the basin of attraction tend towards the attractor state.

This is what happens as you recall a memory. An association or a seemingly random input signal may create signals within a basin of attraction, and as the signals flow towards the attractor, the memory is recalled and potentially strengthened through so-called recurrent connections between the neurons representing the memory.

So, what about the noise, where does it enter this picture?

As described in the excellent book “The noisy brain” by Rolls and Deco (2010), by which I have been inspired to write this post, the neurons in your brain are constantly sending spontaneous signals even when they are “resting”. They refer to this as the spontaneous state, and for a set of resting neurons the neural firings may be approximated by a Poisson distribution, and the average firing rate is about 10 spikes per second (may vary between brain regions). Hence, neurons are sending noisy signals even when they are “off work”!

For neurons in an activated and stable attractor state the firing rate is higher, for instance, 100 spikes per second.

The figure below show two Poisson distributions with different mean firing rates, one for the spontaneous low-rate state and another for high-activity attractor state.

 

 

 

Typically, a neuron will generate an electrical pulse (an action potential) to be forwarded to its output connections, if there within a limited time interval is sufficient excitatory signals from input connections from other neurons. This is known as the integrate-and-fire-model (Lapicque  1907). This is a kind of all-or-nothing response. If the sum of input signals is too low, nothing happens, but as soon as the sum crosses a threshold, the electrical pulse is generated.

The signal, which is passed on, may for instance, trigger a response in your motor cortex making you physically react to sudden changes in your surroundings. Maybe, you need to make a sudden maneuver to avoid hitting an animal jumping in front of your car! In that moment the spontaneous noise from the resting neurons may have made you more alert since they tend to bring the sum of signals closer to the firing threshold. Hence, your reaction time was lowered, and you became more alert due to this inherent noise in your brain.

When you let your thoughts wander, you move around in the landscape, one valley to another, like a hiker. One thought serves as an association to another, and the signals move between the attractor states. Occasionally, though, random firing of neurons may distract your thought, bringing you from one valley to another. However, this is more likely to happen if you are hiking in the Lake District, where only low hills separate the valleys, than in the alpine Himalayas with tall mountain ridges.

This serves as a perfect analogy for what Barbara Oakley (Oakley, 2014) referred to as diffuse mode and focused mode in thinking. In a focused mode the basins of attraction are deep and you are not so easily distracted, because you are really focused on something. On the other hand, in the diffuse mode random and spontaneous firings may easily bring you from one basin to another, and day-dreaming is a typical diffuse mode example.

Creative persons, like the inventor Thomas A Edison, and many outstanding scientists have deliberately used the diffuse mode to enhance their creativity, because creativity is the ability to make new combinations of old ideas. Mental de-focusing, by taking a rest on a couch, a shower, or a walk in the park takes you from the Himalayas to the Lake District, and the random firing in your brain, the noise, is more likely to give you random associations, and new ideas may arise. Like I wrote in my previous post “Google pedagogics” there are evidence that also the cerebellum is highly involved in this creative process, but exactly how is not completely understood yet. It is however interesting that the Purkinje cells of the cerebellum show very high spontaneous firing rates, indicating a high noise level.

In statistics the signal-to-noise ratio is a well known concept. The familiar ANOVA (Analysis of Variance) is a statistical approach, which uses this principle to identify information in the observed variation of a random variable. As is known from ANOVA theory, information is easy to find if the signal-to-noise ratio is high (gives large F-test statistic), but difficult to find if the signal is weak compared to the noise (small F). The signal-to-noise ratio can directly be illustrated by the depth of the valleys. Deep valleys and high mountain ridges parallels high signal-to-noise ratio, and vice versa.

So, we have now seen that a “noisy brain” is not too bad to have! A third example of this is also described by Rolls and Deco, namely the way spontaneous firing may help you to take a decision when you’re stuck between two options. The decision-networks of your brain can represent possible decisions as attractors as well, and like a ball balancing on the ridge between two equally deep basins, you may feel “dead-locked” not knowing what to choose. Often the decision is made on the basis of random external input through the senses, but also random firings in the spontaneous state from neighbor neurons may cause the ball to roll down into one of the decision valleys. Hence, random noise may also help you make probabilistic decisions! This may also explain how we sample our attention points from the posterior point distribution, as I wrote about in the post “Your random attention”.

These are only some of the positive effects of internal noise in our brains. There are of course also other effects of noise, some of which are less positive. The concepts of noise and the signal-to-noise ratio may help explain decreased short time memory concentration and long-time memory formation with aging, the sometimes irrational behavior of stressed persons (the “shadow side” of our personality, as Carl Jung wrote about), mental disorders like Schizophrenia, or simply the lack of focus characterizing people with ADHD.

I will end this post with an open question which I will return to and discuss in a later post.

If our brains are non-deterministic, noisy and decisions may come as a result of random firings of neurons…

…do we have free will?

 

References

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8), 2554-2558.

Lapicque, L. (1907). Recherches quantitatives sur l’excitation électrique des nerfs traitée comme une polarisation. J. Physiol. Pathol. Gen, 9(1), 620-635.

Oakley, B. (2014). A Mind for Numbers: How to Excel at Math and Science (even If You Flunked Algebra). Penguin.

Rolls, E. T., & Deco, G. (2010). The noisy brain. Stochastic dynamics as a principle of brain function.(Oxford Univ. Press, UK, 2010).