For some quick definitions, general intelligence (g) is how smart we are, a single factor underlying our general cognitive ability. Working memory is our ‘mental workspace’ that stores and processes task-relevant information. It is the interface between the current focus of attention and long-term memories. interface between the current focus of attention and long-term memories. Executive control is our ability to manage our attention and goal focus. Meta-analyses systematically assess all peer-reviewed, published studies meeting relevant criteria for a particular type of training. A meta-analysis, compared to a single peer reviewed paper enables us to draw much stronger conclusions about the effectiveness of brain training interventions.

For a quick primer on experimental design and terms such as ‘active vs passive control’ refer to ‘Some useful definitions’ at the end of this review.

Definitions of Intelligence

Most of the studies that address the issue of whether working memory training improves intelligence use a standard dual n-back working memory training task, and use matrices fluid intelligence tests to measure general intelligence (g). We should first acknowledge that (1) the dual n-back is just one working memory training task, and (2) that matrices (e.g Raven’s) tests are only one type of IQ test.

It is standard for short-term and/or working memory tests to be incorporated in full scale-IQ tests, as one important measure of IQ.

In the well-known WAIS-IV full scale IQ test, matrices tests measure what is labelled the ‘Perceptual Organization Index’. The Working Memory Index is a distinct category in the measurement of general IQ.

In an influential theoretical paper in the field of psychometric IQ testing using factor-analytic approaches, Kevin S. McGrew states:

“the Cattell–Horn Gf–Gc and Carroll Three-Stratum models have emerged as the consensus psychometric-based models for understanding the structure of human intelligence. Although the two models differ in a number of ways, the strong correspondence between the two models has resulted in the increased use of a broad umbrella term for a synthesis of the two models (Cattell–Horn–Carroll theory of cognitive abilities—CHC theory).” McGrew, 2009

This model is shown here:

Adapted from Kevin McGrew. Gwm has also been called ‘Gsm’ in representations of the model.

A well known ‘full scale’ IQ test is the WJ IV, and its subtests map onto the broad IQ factors of CHC theory as follows:

It’s clear that the construct of IQ embraces more than what is measured by matrices tests, and that it includes working memory.

So if working memory training improves working memory, then it improves IQ. We’ll see there is strong evidence that dual n-back training improves working memory. There’s also evidence (although less conclusive) that dual n-back training improves fluid intelligence (Gf) measured by matrices tests.

In what follows we’ll look in more detail at the following:

1. Meta-analyses of broad cognitive abilities studies

2. Meta-analyses of working memory studies

3. Meta-analyses of matrices (Gf) studies

4. Specific critiques of the Gf studies (e.g. Bayesian)

5. Fronto-parietal network neuroplasticity studies

1. Meta-analyses of broad cognitive abilities

The most recently published meta-analysis of the working memory training literature by Schwaighofer and colleagues’ (2015) summarizes the training gains in the following table. The asterisks indicate statistical significance. Long-term means 5-12 months after training.

To convert these effect sizes (both short term and long term) to standardized scores such as points in an IQ test, multiply them by 15 – one standard deviation. So for example, the visuospatial working memory gain of 0.63 is equivalent to 0.63 x 15 = 9.5 points on a standardized scale.

In this meta-analysis  Schwaighofer and colleagues found that

• Age was not relevant to such gains: they occurred across the full age spectrum.
• Training duration most likely makes a difference: the more you training, the greater the gains.

And they suggest that

• More complex activities during working memory brain training – with multiple exercises – may result in more practical advantages after training (what is called ‘wide transfer’).

The average age for this meta-study was relatively young. What about for older adults?

A meta-analysis by Karbach and Verhaeghen (Nov 2014) examined the effects of working memory and executive control training (49 studies) in both younger adults and 60+ year olds for a number of general cognitive abilities including attention control, IQ (fluid intelligence), episodic memory (memory for personal experiences), short term and working memory, and processing speed. This study found the following effect sizes with no differences in training-based gains between younger and older adults:

Note that a 0.4 effect size is equivalent to 6.5 points on a standardized scale. These effect sizes are considerable, and include a large IQ (fluid intelligence gain) of ~5.5 IQ points.

For older adults, similar results were found in the meta-analyses of Hindin & Zelinksi, 2012, and Karr et al., 2014. The net gain in overall cognitive ability after (on average) 9 hours of working memory and executive function training is similar in size to the effect of (on average) about five months of  regular 45-minute sessions aerobic training (review).

2. Meta-analyses of working memory (Gwm) studies

This meta-review of verbal and visuo-spatial short term memory and verbal and visuo-spatial working memory (Schwaighofer et al., 2015) shows there are both short term (within a few days of training) and long-term (6-12 month follow up) training effects, and these effects are considerable (see table below).

So for example, the visuospatial working memory gain of 0.63 is equivalent to 0.63 x 15 = 9.5 points. Long term visuo-spatial working memory gains from training due to neuroplasticity are 6.5 points.

Melby-Lervag and Hulme’s 2013 meta-analysis, also concluded that working memory training resulted in visuo-spatial and verbal working memory gains.

Effect sizes reported by individual studies for visuo-spatial working memory, with short-term gains on the left and long term (up to 1 year) on the right, is shown in this plot.

Effect sizes reported by individual studies for verbal working memory, with short-term gains on the left and long term (up to 1 year) on the right, is shown in this plot.

It’s clear from these plots that working memory training improves working memory. Even at 6-12 month’s after training there is a persisting working memory gain, assessed with standard working memory tests. This demonstrates long-term neuroplasticity change,  and this is consistent with the brain science literature (below).

The controversy surrounding studies looking at the effects of working memory training (e.g. the dual n-back) on Gf measured by matrices tests, fails to address these studies looking at the effects of working memory training on working memory capacity/efficiency – another equally valid measure of IQ. Even the fluid intelligence (Gf) gain skeptics Melby-Lervag and Hulme in their 2013 meta-analysis concluded that working memory training resulted in visuo-spatial and verbal working memory gains.

3. Meta-analyses of fluid intelligence (Gf) studies

The 2015 Au et al. meta-analysis found a significant IQ increase in fluid intelligence from working memory training. The authors conclude:

“We urge that future studies move beyond attempts to answer the simple question of whether or not there is transfer [from training to increases in IQ] and, instead, seek to explore the nature and extent of how these improved test scores may reflect true improvements in intelligence that can translate into practical, real-world settings.” Jacky Au and colleagues, University of California, April 2015 

Here is the plot of effect sizes reported in relevant studies reviewed in this meta-analysis:

It is the case that many of these studies are underpowered, but the Au et al (2015) estimates are still the best available. (5)

Au and colleagues’ conclusion that working memory training results in IQ gains finds support in the meta-analysis by Karbach and Verhaeghen (Nov 2014) who estimated a  5.5 point IQ increase. And more recently, it is supported by the latest comprehensive 2015 meta-review by Schwaighofer and his colleagues. They found there were significant increases in IQ measured by both non-verbal and verbal ability after training, regardless of control group. In this study they found that these IQ gains did not last without further training when measured between 6 and 12 months later. But in the short term there was a real IQ gain from training, and we can assume these gains could be maintained or improved with continued training.

The effect sizes reported by all studies looking at working memory training’s effect on non-verbal (Gf) IQ are shown in this plot from Schwaighofer and colleagues, 2015:

The effect sizes reported by all studies looking at working memory training’s effect on verbal IQ are shown in this plot:

We can reasonably conclude from these meta-analyses that Gf gains from training are real – for both younger and older adults.

• Estimates of the overall training benefit for both non-verbal (Gf) and verbal IQ range between 2.0 – 5.5 standardized points (1,2,3 )
• An estimate of the overall training benefit for IQ when we only look at experiments in which the comparison/control group is ‘passive’ and does no computer activity is ~7 IQ points (1)

4. Specific Critiques of the Gf Studies

Melby-Lervag and Hulme critique

The conclusion of this meta-analysis has been challenged by Melby-Lervåg and Hulme in a reanalysis of the data. While they do not doubt that there is a Gf gain after brain training of around 7-8 IQ standardized points, they argue that this gain is essentially a placebo effect since when the comparison (‘control’) groups are active (i.e. do other computer tasks such as a simple attention exercises), the effect size is greatly reduced.

But this Oct 2015 response to Melby-Lervag and Hulme counters their arguments and supports Au et al.’s original conclusion that the working memory training does indeed improve IQ:

We demonstrate that there is in fact no evidence that the type of control group per se moderates the effects of working memory training on measures of fluid intelligence and reaffirm the original conclusions in Au et al., which are robust to multiple methods of calculating effect size, including the one proposed by Melby-Lervåg and Hulme. (Au et al., Oct, 2015)

Au et al (2015)  point out that

“…the present direction of effects actually suggests that passive control groups could end up outperforming active control groups which runs opposite to the direction suggested by the idea that Hawthorne or expectancy effects drive improvements in both active control and treatment groups.”

Dougherty and colleagues (2015) Bayesian critique

“We find that studies using a noncontact (passive) control group strongly favor the alternative hypothesis that training leads to transfer but that studies using active-control groups show modest evidence in favor of the null. We discuss these findings in the context of placebo effects.”

Since the ‘susceptibility to placebo’ is a plausible alternative explanation to Dougherty and colleagues’ it needs to be directly addressed in future studies. What is needed before placebo criticisms can be regarded as a serious challenge is direct evidence for placebo effects in the form of experiments where expectations are systematically varied, or adding a third group to the controlled trial set-up, which takes an existing intervention that is known to work – if both that group and the group given the effective intervention fail to beat the placebo, researchers know that their trial design is flawed.

5. Fronto-parietal network neuroplasticity studies

Studies that look a cortical network and synaptic neuroplasticity effects from WM and CC training are consistent with wide IQ transfer interpretations of the behavioral meta-analyses

There are now many neuroimaging studies showing consistent fronto-parietal network (FPN) neuroplasticity effects from working memory (e.g. dual n-back) brain training (e.g. Thompson et al., 2016; Metzler-Baddeley et al. 2016; Kundu et al., 2015). There is extensive evidence for the recruitment of the fronto-parietal network (FPN) – relative to other cortical networks – in tasks requiring fluid intelligence (IQ) (e.g. Preusse et al., 2011).

FPN  neuroplasticity effects are clearly not ‘placebo’ effects, and they help explain the consistent WM training gains that are seen in meta-analyses, as well as the consistent emotion-regulation effects that are observed.

Barbey et al. (2012) investigated the neural substrates of the general factor of intelligence (g) and executive function in 182 patients with focal brain damage using MRI brain imaging. They concluded:

Impaired performance on these measures in the WAIS-IV and Delis-Kaplan Executive Function test were associated with damage to a distributed network of left lateralized brain areas, including regions of frontal and parietal cortex and white matter association tracts, which bind these areas into a coordinated system. The observed findings support an integrative framework for understanding the architecture of general intelligence and executive function, supporting their reliance upon a shared fronto-parietal network for the integration and control of cognitive representations.

Kundu and colleagues (2015) have recently proposed that the transfer of WM training to other cognitive abilities is supported by changes in connectivity in frontoparietal and parieto-occipital networks – active in  both the trained and transfer tasks.  The frontoparietal network is part of the Cognitive Control Network (9) involved in attention control and goal maintenance.

Consistent with this, in a recent MIT, Harvard and Stanford neuroimaging study (Jan, 2016), Thompson and colleagues, found:

[Dual n-back] training differentially affected activations in two large-scale frontoparietal networks thought to underlie working memory: the executive control network and the dorsal attention network. …Load-dependent functional connectivity both within and between these two networks increased following training, and the magnitudes of increased connectivity were positively correlated with improvements in task performance. These results provide insight into the adaptive neural systems that underlie large gains in working memory capacity through training.

And in their recent paper Task complexity and location specific changes of cortical thickness in executive and salience networks after working memory training, Metzler-Baddeley and colleagues (2016) found that working memory training resulted in increases of cortical thickness in right parieto-frontal cortex. (They also found that training led to a reduction of thickness in the right insula and that these changes were related to changes in working memory span.)

Interference Control

There is an important overlap in brain circuitry between interference control, working memory capacity and IQ (Gf). Brain imaging studies reveal that neural mechanisms of interference control underlie the relationship between fluid intelligence and working memory span.

Greenwood and Parasuraman (2015) hypothesize that training-related increases in control of attention in the frontoparietal control network and circuits underlying interference control underlie ‘far transfer’ of cognitive training to untrained abilities, notably to fluid intelligence.

Interference control allows us to suppress distractions. There is compelling evidence that distraction suppression (evident in behavior, neuronal firing, scalp electroencephalography, and hemodynamic change) is important for protecting target processing during perception and holding information in working memory. Consistent with this evidence, forms of cognitive training that increase the ability to ignore distractions (e.g., working memory training and perceptual training) not only affect the frontoparietal control network but also affect transfer to fluid intelligence.

Working memory training games that incorporate systematic interference control and distraction shielding may be expected to enhance IQ gains.

Summary

This brief review of the 2014-2015 scientific meta-analysis literature supports the conclusion that working memory and executive control training  increases general cognitive performance – whether IQ (verbal or non-verbal ability), short-term memory or working memory. This training benefit is mediated in part by neuroplastic changes in the frontoparietal network.

Based on the available evidence, we can conclude that there are not sufficient grounds to discredit the claim that working memory training is an effective and efficient strategy for improving IQ.

.

References

Au, J., Buschkuehl, M., Duncan, G. J., & Jaeggi, S. M. (2015). There is no convincing evidence that working memory training is NOT effective: A reply to Melby-Lervåg and Hulme.  Psychonomic Bulletin & Review. Oct, 2015. Abstract

Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: a meta-analysis. Psychonomic Bulletin & Review, 22(2), 366-377. Abstract

Barbey, A. K., Colom, R., Paul, E. J., Grafman, J. (2013). Architecture of fluid intelligence and working memory revealed by lesion mapping. Brain Structure and Function, 219, 2. 485-94. Article.

Barbey, A. K., Colom, R., Solomon, J., Krueger, F., Forbes, C., & Grafman, J. (2012). An integrative architecture for general intelligence and executive function revealed by lesion mapping. Brain, 135(4), 1154–1164. http://doi.org/10.1093/brain/aws021

Bogg, T., & Lasecki, L. (2014). Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence. Frontiers in Psychology, 5, 1589. Abstract.

Greenwood, P. M., & Parasuraman, R. (2015). The Mechanisms of Far Transfer From Cognitive Training: Review and Hypothesis.Neuropsychology. [Ahead of print].

Hindin S.B., Zelinski E.M. Extended Practice and Aerobic Exercise Interventions Benefit Untrained Cognitive Outcomes in Older Adults: A Meta-Analysis. Journal of the American Geriatrics Society.2012;60(1):136–141. [Article]

Karbach, J., & Verhaeghen, P. (2014). Making working memory work: A meta-analysis of executive-control and working memory training in older adults. Psychological Science, 25, 2027–2037. Abstract.

Karr J.E., Areshenkoff C.N, Rast P, Garcia-Barrera M.A.  (2014). An Empirical Comparison of the Therapeutic Benefits of Physical Exercise and Cognitive Training on the Executive Functions of Older Adults: A Meta-Analysis of Controlled Trials. Neuropsychology, 28(6):829-45. [Article]

Jaeggi, S.M., Buschkuehl, M., Jonides, J., & Perrig, W.J. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences of the United States of America, 105(19), 6829-6833. Abstract / Article

Metzler-Baddeley, C., Caeyenberghs, K., Foley, S., & Jones, D. K. (n.d.). Task complexity and location specific changes of cortical thickness in executive and salience networks after working memory training.NeuroImage. http://doi.org/10.1016/j.neuroimage.2016.01.007

McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1–10. http://doi.org/10.1016/j.intell.2008.08.004

Melby-Lervag, M., & Hulme, C. (2013). Is working-memory training effective? A meta-analytic review. Developmental Psychology, 49, 270– 291. Abstract.

Preusse, F., Elke, van der M., Deshpande, G., Krueger, F., & Wartenburger, I. (2011). Fluid Intelligence Allows Flexible Recruitment of the Parieto-Frontal Network in Analogical Reasoning. Frontiers in Human Neuroscience, 5. http://doi.org/Abstract

Schwaighofer, M., Fischer, F., Buhner, M. (2015) Does Working Memory Training Transfer? A Meta-Analysis Including Training Conditions as Moderators. Educational Psychologist 50, 2. Abstract, Article.

Some useful definitions

For those of you without training in experimental design, here are some useful definitions that will equip you to understood some of the more technical content of this review, and help you evaluate it for yourself.

Experiments / Randomized Control Trials involve randomly assigning participants in the study to receive one of a number of cognitive interventions. One of these interventions is the computerized cognitive training (brain training) program. One of these interventions is the standard of comparison or control. The control may be an active control (e.g. playing a simple game or doing cross-words for the same duration as the brain training), or a passive control where there is no intervention at all.

Peer-reviewed journal articles.  These are published articles of randomized control trials (studies) on brain training that have been submitted to the scrutiny of experts in the same field, and judged to acceptable for publication.

Meta-analyses systematically assess all peer-reviewed studies meeting adequate standards of experimental design and relevance criteria for a particular type of brain training. A meta-analysis uses a statistical approach to combine the results from multiple trials to improve estimates of the size of the effect and resolve uncertainty when reports disagree – for example when one study concludes there is an effect and another study does not. It can also correct for publication bias – the tendency to only publish reports when there is a positive result. A meta-analysis, compared to a single peer-reviewed journal article, enables us to draw much stronger conclusions about the effectiveness of brain training interventions.

If there is statistical significance in a brain training study, it means that the difference in tested outcomes such as average IQ score between training group and the control group is very unlikely (p < 0.05) to have occurred by chance. If the study is well-designed, this gives us confidence that the difference in IQ scores between the brain training and placebo group is due to the training itself, and not some fluke.

The effect size is a measure of the magnitude of the outcome difference between the two groups, which can be measured in standardized scores. Effect size is typically measured in  ‘standard deviation’ units (g). When  SD = 1.0, this is equivalent to 15 points in a standardized IQ test. If SD = 0.5 this would be 7.5 points. And so on. As a reference, antidepressant drugs typically have an effect size (compared to placebo) of 0.3 – 0.5 – i.e. 4.5 – 7.5 points.

Working Memory and Executive Control Training Apps

My own working memory and executive functioning brain training software implements a combination of working memory training (e.g. n-back) and executive function training. (e.g. interference control and task-switching for cognitive flexibility). This cognitive training IQ app can be found here.  I am happy to provide this software free of charge for any research or education related projects.

An implementation of the classic dual n-back training app – for desktop and mobile – can be found here.

The company CogMed also provides working memory training tools with a scientific basis.