Design for analysis
The most common registry analysis failure: data was collected without a prespecified analysis plan. The result is either a fishing expedition for significant findings (p-hacking) or a dataset that cannot answer the questions it was supposed to address.
Pre-specify your primary analyses before collecting data. This means defining:
- Primary outcome(s)
- Primary exposure(s) or comparison groups
- Covariates to adjust for
- Statistical methods
- Missing data handling approach
Your SAB should review and approve the analysis plan.
Common registry analyses
Disease characterization
- Prevalence / incidence estimation, How common is the disease? How many new cases per year?
- Symptom frequency and distribution, What proportion of participants have each clinical feature?
- Demographic analysis, Age at onset, sex distribution, time to diagnosis
- Genotype phenotype analysis, Which genetic variants are associated with which clinical features?
Natural history
- Disease progression modeling, How do symptoms change over time?
- Survival analysis (time-to-event), Time to disease milestones, time to treatment initiation
- Longitudinal mixed models, Track individual trajectories and population level trends
Treatment patterns and outcomes
- Treatment use, Which treatments are participants receiving, and for how long?
- Comparative effectiveness, Do participants on treatment A have better outcomes than participants on treatment B? (Requires careful confounding adjustment)
Statistical considerations for rare diseases
Small sample sizes
Most rare disease registries operate with small N. Implications:
- Pre-specify analyses to avoid overfitting
- Use Bayesian methods where appropriate (allow incorporation of prior knowledge)
- Be cautious about subgroup analyses, report effect sizes and confidence intervals, not just p-values
- Power analyses should be conducted before data collection, not after
Missing data
Missing data is ubiquitous in observational registries. Approaches:
- Complete case analysis, Easiest; valid only when data is missing completely at random (MCAR), which is rarely true
- Multiple imputation, Appropriate for data missing at random (MAR); widely used
- Sensitivity analysis, Test how your conclusions change under different missing data assumptions
Confounding
Registry data is observational, participants are not randomized to treatments or exposures. Confounding is a major threat to causal inference. Methods:
- Propensity score matching or weighting, Balance treatment groups on observed covariates
- Instrumental variable analysis, For when confounding by indication is severe
- Target trial emulation, Explicitly design the analysis as if it were a randomized trial
Tools and platforms
| Tool | Use |
|---|---|
| R (CRAN) | General statistical analysis; extensive rare disease packages |
| Python (pandas, statsmodels) | Data manipulation and analysis |
| SAS | Industry/regulatory standard; expensive |
| ATLAS (OHDSI) | Cohort analysis on OMOP data |
| REDCap built-in reports | Basic frequency tables and exports |
| Tableau / Power BI | Data visualization and dashboards |