Scientist have always been curious to why we feel tired and must sleep after a long day. New theories in neuroscience may explain why, and interesting enough, what happens during sleep bears resemblance to statistical methods like shrinkage estimation and variable selection. The theories also make it clear why sleeping is so important for effective learning and long term memory.

When we are awake and conscious we are constantly processing input information, and we are in a state of learning much of the time. New thoughts and memories are formed by so-called long term potentation (LTP), a process were the strengths of the connections (synapses) between the neurons in the brain are increased. Some connections are also weakened by an opposite process called long term depression (LTD), if the connections turn out to be less important.

Due to random spiking of neurons “errors” may also be induced by LTP of non-important synapses and LTD of important synapses. (The random spiking of neurons is an important property og the brain network which I will return to in a later post.)

A memory or a skill that we learn is stored as a so-called “engram”, a strengthened pathway between neurons in the neuronal network which is formed by LTP and LTD. You may think of the engrams as valleys in an energy landscape with mountains formed by LTD and the valleys by LTP. Recalling a memory activates signals along the pathway like a ball rolling through the valleys of the energy landscape.

The formation of memories may mathematically be described in terms of recurrent associative networks (e.g. Rolls and Deco, 2010). According to these models learning may be described by the Hebb rule (Hebb, 1949) where input signals from numerous neurons to a single neuron are multiplied by their respective synaptic weights and summed. If the sum of weighted inputs is sufficiently large, the receiver neuron will pass on a signal. If a signal is created, the input synapses are strengthened through LTP. This is known as “the integrate and fire model”.

Let’s leave the neural technicalities there for now, and go on to what this has to do with sleeping, statistics and learning.

Well, according to a relatively new theory by Tononi and Cirelli (2006) the reason why we feel tired is because the conscious learning period (daytime) results in a net increase in synaptic strengths in our brain. This has an exhausting effect because stronger synapses means more energy consumption.

Our brain accounts for approximately 20% of our energy demand during the day, but it is not constant. The energy demand is at its lowest right after sleep, and may increase by up to 18% during the awake period. This makes us tired, and new learning potential by further LTP is reduced.

So, what happens during the night which turns the energy level down? According to Tononi and Cirelli the explanation may be a overall and uniform LTD (weakening) of all synaptic strengths in the brain. In statistical terms we call this scaling. By dividing all synaptic weights by some constant, the average synaptic weight is normalized to a base level before we wake up ready for new inputs and learning.

Exactly how this scaling is carried out in the brain during sleep is still under investigation, but a theory is that the slow wave activity observed during the NREM phase of sleeping induces a massive LTD proportional to the synaptic strengths. This means that strong synapses remain strong and weak synapses remain weak. The weakest connections may also be removed altogether. Hence, long term memories and rehearsed skills are not removed, but rather reinforced during sleep by the removal of irrelevant connections.

The way irrelevant connections are reduced and potentially removed during sleep is very similar to what is known as shrinkage methods in statistical inference. The well-known Ridge estimator in multiple regression, which facilitates a kind of scaling of regression coefficients, is an obvious example. Shrinking estimates towards zero often reduces variance and improves prediction. We may say that the shrinkage improves the signal-to-noise ratio, which appears to have the same effect as taking a nap for our brain connections.

Shrinkage by scaling is not the only option. So-called soft-shrinkage which is part of methods like LASSO (Tibshirani, 1996) and ST-PLS (Sæbø et al., 2008) is another. An extra benefit of this approach is the possibility to remove the influence of some variables altogether by forcing their effects to be zero, thereby introducing variable selection. Scaling, on the other hand, will not remove variables completely.

The mechanisms for overall LTD during sleep is currently unknown, but a soft-shrinkage property could explain complete removal of irrelevant connections during sleep. It will be interesting to follow the research in the future at this point.

This theory supports the findings that rest improves long term memory, and it gives a clear message to us all that getting enough sleep is important for learning. Not only does the increased energy demand during the awake period reduce the learning potential, but sleep also consolidates the already stored engrams.

Tononi and Cirelli states that sleep in fact may be the price to pay for our high conscious level and learning potential through brain plasticity.

However, their theory is not the only theory explaining why sleep consolidates memory. Others suggest that the dreaming phase during REM sleep also has a function. Replaying thoughts and experiences during dreaming may increase the synaptic strengths of the engrams and improve long term memory. Further, the rather bizarre sequences of associations that may occur during sleep may have a creative side effect. Many have experienced that their dreams give creative input to their daily activity, and the great inventor Edison even used this actively in his work! The story also tells that the russian scientist Mendelejev, who worked hard putting together the table of the elements, got the final pieces to fit together during sleep.

Sleeping is therefore very important for learning and creativity, and even a short nap during the day can have an invigorating effect on learning.
Maybe a “power nap” after lectures should be compulsory for all students?

 

References

Tononi, G. and Cirelli, C. (2006). Sleep function and synaptic homeostasis. Sleep Medicine Reviews, 10, 49-62.

Hebb, D.O. (1949).The organization of behaviour: A neuropsychological theory, Wiley, New York.

Rolls, E.T. and Deco, G. (2010). The noisy brain, Oxford University Press, Oxford.
Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. J. R. Statist. Soc B, 58 (1), 267-288.

Sæbø, S. et al. (2008). ST-PLS: a multi-directional nearest shrunken centroid type classi er via PLS, J. Chemometrics, 22 (1), 54-62.