Meta brain imaging studies with multilevel kernel density analysis

There is so much empirical data on brain imaging studies where subjects perform a specific task that it is crucial to perform meta analyses on the data to see how reliable the results are. Wager et al. (2008) set out to review attempts to do just this, multilevel kernel density analysis (MKDA) approach, which recreates a map of significant regions from each study in order to analyze the consistency (whether the same voxels appear in other studies) and the specificity (whether only those voxels appear in other studies for that task) of each. One of the major challenges in brain imaging (and multivariate statistics in general) is to keep the familywise error rate down to 0.05 as many different regions are analyzed.

In addition to considering these issues in-depth, they report on some pretty interesting data. By defining a peak 10 mm outside of the “consensus region” in a given meta-analysis as a non-replication, they can determine a rough false positive rate for that region. For working memory storage (26 studies, N=305), this rate is 40%; for executive working memory (60 studies, N=664), the rate is 20%; for emotion (163 studies, N=2010), the rate is 11%; for long term memory (166 studies, N=1877), the rate is 10%. Their data reveals two interesting trends:

1) Studies with larger sample sizes have more statistical power and are thus less likely to fall victim to a false positive error, and

2) The false positive rate in general (especially for working memory storage) is large enough to make this an area of concern.

Reference

Wager TD, Lindquist MA, Nichols TE, Kober H, Van Snellenberg VX. 2009 Evaluating the consistency and specificity of neuroimaging data using meta-analysis. Neuroimage 45:S210-S221.  doi:10.1016/j.neuroimage.2008.10.061.