Build Status codecov.io CRAN_Status_Badge CRAN_Status_Badge

EmpiricalCalibration is part of HADES.


This R package contains routines for performing empirical calibration of observational study estimates. By using a set of negative control hypotheses we can estimate the empirical null distribution of a particular observational study setup. This empirical null distribution can be used to compute a calibrated p-value, which reflects the probability of observing an estimated effect size when the null hypothesis is true taking both random and systematic error into account, as described in the paper Interpreting observational studies: why empirical calibration is needed to correct p-values.

Also supported is empirical calibration of confidence intervals, based on the results for a set of negative and positive controls, as described in the paper Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data.


Screenshots and examples

Calibration effect plot

data(sccs) #Load one of the included data sets
negatives <- sccs[sccs$groundTruth == 0,] #Select the negative controls
null <- fitNull(logRr = negatives$logRr, seLogRr = negatives$seLogRr) #Fit the null distribution
positive <- sccs[sccs$groundTruth == 1,]  #Select the positive control

#Create the plot above:
plotCalibrationEffect(logRrNegatives = negatives$logRr,
                      seLogRrNegatives = negatives$seLogRr,
                      logRrPositives = positive$logRr,
                      seLogRrPositives = positive$seLogRr,
                      null = null)

#Compute the calibrated p-value:
calibrateP(null = null, logRr = positive$logRr, seLogRr = positive$seLogRr) #Compute calibrated p-value
[1] 0.8390598


This is a pure R package.

System requirements

Requires R (version 3.1.0 or newer).


In R, use the following commands to install the latest stable version from CRAN:


To install the latest development version directly from GitHub, use:

install_github("ohdsi/EmpiricalCalibration", ref = "develop")

User Documentation

Documentation can be found on the package website.

PDF versions of the documentation is also available:



Read here how you can contribute to this package.


EmpiricalCalibration is licensed under Apache License 2.0


This package has been developed in RStudio.

Development status

This package is ready for use.