Evaluating effect modification using propensity score matching and disease risk score methods: an empirical study

Study type
Protocol
Date of Approval
Study reference ID
18_174
Lay Summary

The safety and benefit of medications and surgical interventions are best evaluated using clinical trials which randomize subjects to different interventions groups and evaluate outcomes. In the absence of such trials, routinely collected data can be used to provide evidence of relative benefit and safety of clinical interventions.

However, since subjects in real-life clinical practice are not randomized but receive the interventions based up on their clinical profile, those receiving different treatments differ in their clinical characteristics. Differences in outcomes between the groups who received one intervention versus the group who did not receive the intervention (or who received a different intervention) may be explained by differences in baseline risk factors and not just the intervention. Hence, one has to account for differences in baseline characteristics to be able to know the effect of the intervention.

Furthermore, the benefit and safety of interventions may not be the same for every patient; some group of patients (e.g. males) might get more benefit compared to others (e.g. females). To evaluate the presence of such subgroup effects, one has to look in to the effect of the interventions with in each subgroup using statistical techniques known as propensity score and disease risk score methods.

In this study, we plan to use routinely collected data from CPRD to evaluate statistical methods to evaluate subgroups effect of medications or surgical interventions while accounting for differences in baseline characteristics.

Technical Summary

Propensity score (PS) and disease risk score (also called prognostic score, DRS) methods are popular methods to account for confounding in observational studies. In evaluating effect modification in studies of drug effects or surgical interventions, treatment effect is estimated within subgroups of the effect modifier after propensity score matching (PSM). While, PSM improves balance on covariates included in the PS model, such balance cannot be assured within strata of a covariate, for example a potential effect modifier, unless further matching is done using that specific covariate. A recent literature review showed that many studies do not account for the fact that creating subgroups using a covariate, an effect modifier, will break the covariate balance created on the full PS matched set. However, the impact of the imbalance on the bias of the estimated treatment effect is not studied. In addition, alternative methods such as matching using the PS and the effect modifier or combining DRS methods and PS to evaluate effect modification has not been investigated using empirical data.

Health Outcomes to be Measured

Type 2 diabetes
- Cancer
- Mortality
- Cardiovascular disease
- Acute liver injury
- Post-operative complications
- Fractures

Collaborators

Liam Smeeth - Chief Investigator - London School of Hygiene & Tropical Medicine ( LSHTM )
Sanni Ali - Corresponding Applicant - London School of Hygiene & Tropical Medicine ( LSHTM )
Daniel Prieto-Alhambra - Collaborator - University of Oxford
Elizabeth Williamson - Collaborator - London School of Hygiene & Tropical Medicine ( LSHTM )
Ian Douglas - Collaborator - London School of Hygiene & Tropical Medicine ( LSHTM )