*Geek Box: Average Treatment Effect
One of the main reasons why RCTs are considered more reliable than other research designs is that the size of the effect (if any) is presumed to be more accurate. When it comes to effect sizes, what intervention trials look at is the average treatment effect (ATE). This is the difference between the average (mean) effect in the intervention group vs. the average effect in the control group. This can be an issue in trials looking at the effects of a supplemental nutrient, as often the control group has adequate intake of that nutrient anyway from diet, and the difference in the ATE between the intervention and control group is not large and is statistically insignificant.
To calculate the ATE in both arms of a trial, the general standard approach is to compare the difference between the baseline value (for whatever is being measured) and the end value. Let’s say, for example, that a group start a trial in the intervention group with a blood glucose reading of 6.5mmol/L and finish with a mean level of 4.5mmol/L: a mean difference of 2.0mmol/L. Now, one limitation of the ATE is that we look at the mean, and cannot see individual effects. Some people may have larger or smaller responses.
The other factor to bear in mind is that the ATE for the same exposure or intervention may differ from trial to trial, for example if the baseline values are higher or lower, if the achieved level is higher or lower, if the duration of the trial differs, etc. This means, from a meta-analysis perspective, to compare these trials could bias the outcome and over or underestimate the effects of an intervention. In the present meta-analysis, rather than take the ATE of each included study, the used the actual level of blood pressure after the intervention, and the difference between attained levels.
Let’s come back to our example above, and let’s say in that study the control group increased from 6.5 to 7.0mmol/L. In this example, the mean post-intervention difference between intervention and control would be 2.5mmol/L. This means the effect size isn’t the difference between baseline and end of study, which is influenced by the factors mentioned above, but the actual difference between intervention and control values after the intervention.