Methodological challenges in the use of propensity scores in distributed networks

Study type
Protocol
Date of Approval
Study reference ID
22_001822
Lay Summary

Propensity scores (PS) are a statistical technique adopted in drug safety and effectiveness research to minimize potential confounding, a bias that distorts the estimated treatment effects of the drugs under investigation. The PS is a summary score that includes many patient characteristics (demographic, clinical, history of medication use, etc.), the use of which makes the treatment groups more comparable. Although this approach is commonly used, there remain crucial aspects surrounding their use that require further scrutiny, including how much history should be considered when estimating PS. A related problem is that drug safety networks that implement PS methods in data from multiple jurisdictions have a coordinating centre sending protocols to data partners, with results returned to the coordinating centre for analysis but little attention paid to differences between estimates. To examine these issues, we will conduct a methodological study assessing the consequences of limited database history in an example of comparing the clinical outcomes of two medication classes commonly used to treat diabetes (sulfonylureas vs. metformin). Additionally, within a network based on CPRD regions, we will develop tools for visually showing differences in results by jurisdiction (or data partner), and we will identify benefits and drawbacks of methods that reduce differences in characteristics between data partners. Outcomes studied will include heart attacks, strokes, dying of cardiovascular causes, and dying from any cause. Our goal is to provide guidelines to researchers on the best ways to implement PS methods when dealing with limited database history or multiple data partners.

Technical Summary

Testing in real-world data is a critical step in identifying limitations of analytic approaches in pharmacoepidemiology. In this study, we will examine 1) the impact of changing the baseline period of covariates measurement on PS estimation, and 2) how analyses of distributed data networks should approach heterogeneity in confounding structures, outcomes, and effect estimates. Cohort studies in pharmacoepidemiology must typically select the length of the baseline period to measure covariates, as databases provide varying and truncated historical information. Using CPRD data linked to Hospital Episode Statistics and Office for National Statistics data, we will conduct a methodological study using the example of sulfonylureas vs. metformin as first line treatment for type 2 diabetes and the risks of myocardial infarction, ischemic stroke, cardiovascular death, and all-cause mortality. We will evaluate the empirical distribution of the PS adopting progressively restricted lookback time windows, as well as standardized mean differences to quantify covariates imbalance after matching. This study will inform researchers about the importance of properly selecting the length of the baseline period used to compute the PS, which should be balanced with the resulting size of the cohort. Moreover, an increasing number of PS studies take place in distributed networks with multiple data partners. Heterogeneity between these partners can be difficult to describe, and little work has been done identifying how target populations may help clarify effect estimates. Given the differences between regions within CPRD, treating each region as a separate data partner can help explore many methodological questions. After creating a pseudo-distributed data network, we will create tools to examine differences in confounding structures, outcome rates, and effect estimates. We will also assess whether using sampling weights to standardize effect estimates from each region to explicit target populations facilitates comparison of estimates within each region to one another.

Health Outcomes to be Measured

● Myocardial infarction
● Ischemic stroke
● Cardiovascular death
● All-cause mortality

Collaborators

Samy Suissa - Chief Investigator - Sir Mortimer B Davis Jewish General Hospital
Robert Platt - Corresponding Applicant - McGill University
Enrico Ripamonti - Collaborator - University of Brescia
Kristian Filion - Collaborator - McGill University
Michael Webster-Clark - Collaborator - McGill University
pauline reynier - Collaborator - Sir Mortimer B Davis Jewish General Hospital
Qi Zhang - Collaborator - Sir Mortimer B Davis Jewish General Hospital
Rubiya Akter - Collaborator - McGill University

Former Collaborators

Enrico Ripamonti - Collaborator - University Milano-Bicocca

Linkages

HES Admitted Patient Care;ONS Death Registration Data;Practice Level Index of Multiple Deprivation