*Geek Box: Average Treatment Effects
One very misunderstood aspect of interpreting studies, in particular intervention studies, is that a lack of effect means that the intervention worked for no one. This is an easy trap to fall into, because what RCTs compare between the intervention and control group is the mean, or average, treatment effect. This is the average of the effect from all the participants in the intervention group vs. the average effect from all participants in the control.
This is often referred to as the ‘ATE’ [average treatment effect]. The difficulty, particularly for nutrition interventions, is that the results and conclusions are based off the ATE, which often results in an over-simplistic extrapolation of “there is no effect of X intervention on Y outcome”. But look closer at the data, what can you see? You may often see that in some participants [figure below], there was an effect. For example, in the present study, look at Figure 1(a), and look at the paired line graph for fibre; clearly a number of participants had quite a substantial decrease in liver fat. Many didn’t, and some increased; so overall the mean, the average, is ’no effect’. But it is important to bear in mind that it worked for someone.
Now, to figure out ‘why’ it worked for some, not others, generally requires further, post-hoc analysis by the investigators. This is something that should be done, but in practice often isn’t. The take home point, particularly for you reading nutrition studies, is to always bear in mind that it is the ATE being reported, and compared, but sometimes it can pay to dig a little further and see if the ATE is masking an effect in a subgroup of participants.