*Geek Box: Standardised Mean Difference Measure of Effect
The standardised mean difference [SMD] of effect size is also known as Cohen’s d [which we have come across in a previous Deepdive]. It compares two means: one from a treatment group and one from a control group or comparison group, and calculates an effect size by subtracting the effect in the placebo/control group from the effect in the intervention group, and dividing it by the pooled standard deviation of the groups.
The difference between the SMD and a p-values is that the p-value simply tells you whether there is an effect that is statistically significant, while the SMD tells you the size of the effect, i.e., how much did the magnitude of effect in the treatment group differ to the magnitude of effect in the control group. Similar to the p-value being set at <0.05 for significant being arbitrary, interpreting the SMD results have been subject to similar classifications, and the general thresholds of 0.2, 0.5, and 0.8 are considered small, medium, and large, respectively. However, there is no accepted definition, and researchers may choose their own thresholds for effect size.
The SMD may [and should] be accompanied with 95% confidence intervals, where the SMD value is the point estimate and the 95% CI provide the range for the potential effect size. This allows for comparison in effect sizes between studies. The effect size interpretation for the SMD is just a general guideline and – similar to the way in which a statistically insignificant finding does not mean ‘there is no effect’ – the relevance of a ‘small’ or ‘large’ effect size will depend on the exposure and specific context being studies. A Cohen’s d of 0.4, for example, may be small-medium, but may represents a clinically meaningful difference.