.png){kind=icon}
Practical activity
A range of simple practical activities related to accuracy and precision are available from the website:
Further example
One issue where accuracy and precision are very relevant is meta-analysis. Often if you look in the literature you will find a range of published studies that all explore the same question. For example, you could easily find literally dozens of studies addressing the question of whether medication or cognitive interventions are most effective in the treatment of mild depression. Not all these studies will reach the same quantitative conclusion. This variation may be down to differences in sample populations, methodology, in the nature of the interventions, statistical power, and/or statistical analysis involved. A systematic review is a formal process by which the body of independent studies are considered together in order to produce a consensus position on the current state of knowledge on the basis of this body of research. A meta-analysis is a specific type of systematic review that uses statistical methods to try and reach a quantitative consensus conclusion by combining data from the available studies.
The link to precision is clear. Meta-analysis is attractive because it is an attempt to average across the introduced noise and produce an analysis with greater statistical power than any of the independent studies on their own. Hence, the motivation for doing a meta-analysis can be thought of as improving precision. However, the big challenge to doing meta-analyses is avoiding bias.
You can imagine that if the outcome of a particular study affects whether it is published or not then the range of different published works you can use in your meta-analysis will be biased. It is easy to imagine how such publication biases might occur. For example, if a drug-company sponsored two different studies exploring the effectiveness of their particular drug in the treatment of depression, and one found a beneficial effect and the other did not; you could easily imagine why the drug company might put more effort into ensuring that the former study was published. You can also imagine how if you are not very careful in how you search for published studies that your means of selecting studies for use in your meta-analysis might introduce biases - two well-known ones are language bias (studies published in English are more likely to end up in meta-analyses) and citation biases(studies that are widely cited in the scientific literature are more likely to be used in a meta-analysis).
People use funnel plots as a way to explore whether there is bias in studies used in their meta-analysis, and precision is key to these plots. We give an example plot in Figure S11.1. The funnel plot is a scatter-plot, with a point plotted for each study: on the x-axis we would have effect size (in our case this might be recovery rate from mild depression of patients on medication minus the recovery rate of those subject to purely cognitive intervention). The y-axis should be some measure of the precision (or power) of the study (this might be number of participants in the study, for example). We would expect that high-precision studies have effect sizes that are clustered close together, but as precision drops so there will be a greater spread of reported values. However, that spread should be symmetrical (low power is just as likely to cause a reduction in reported effect size as an increase). Hence the graph gets its name from the expected shape of the spread of points; looking like a symmetrical inverted funnel tapering as we move up the y-axis to higher precision studies. Any strong deviation from symmetry is taken as evidence of potential for bias in the meta-analysis, but it is also worth remembering that asymmetry can also be induced if all the studies follow a similar but flawed experimental design that introduces a bias in each individual study that becomes more apparent as power decreases, or if the low-powered experiments are more likely to also have methodological flaws. As well as visual inspection of the funnel plot there are statistical methods available (e.g. Egger’s test) to give an objective means on deciding whether there is evidence of asymmetry.
Figure S11.1 An example funnel plot. The x-axis is a measure of the effect size, whilst the y-axis is an estimate of the power of the study. The points represent individual studies.