Coverage probability

Compute and plot coverage of CI for different confidence level. Useful for fake data check.

compute_coverage(
  post_samples,
  truth,
  CI = seq(0, 1, 0.05),
  type = c("eti", "hdi")
)

plot_coverage(
  post_samples,
  truth,
  CI = seq(0, 1, 0.05),
  type = c("eti", "hdi")
)

Arguments

post_samples

Matrix of posterior samples. Rows represent a sample and columns represent variables.

truth

Vector of true parameter values (should be the same length as the number of columns in post_samples).

CI

Vector of confidence levels.

type

Type of confidence intervals: either "eti" (equal-tailed intervals) or "hdi" (highest density intervals).

Value

compute_coverage returns a Dataframe containing coverage (and 95% uncertainty interval for the coverage) for different confidence level (nominal coverage). plot_coverage returns a ggplot of the coverage as the function of the nominal coverage with 95% uncertainty interval.

Examples

N <- 100
N_post <- 1e3
truth <- rep(0, N)
post_samples <- sapply(rnorm(N, 0, 1), function(x) {rnorm(N_post, x, 1)})

compute_coverage(post_samples, truth)
#> # A tibble: 21 × 4
#>    Nominal Coverage   Lower  Upper
#>      <dbl>    <dbl>   <dbl>  <dbl>
#>  1    0        0    0       0     
#>  2    0.05     0.02 0.00243 0.0704
#>  3    0.1      0.08 0.0352  0.152 
#>  4    0.15     0.09 0.0420  0.164 
#>  5    0.2      0.13 0.0711  0.212 
#>  6    0.25     0.19 0.118   0.281 
#>  7    0.3      0.22 0.143   0.314 
#>  8    0.35     0.28 0.195   0.379 
#>  9    0.4      0.33 0.239   0.431 
#> 10    0.45     0.41 0.313   0.513 
#> # ℹ 11 more rows
plot_coverage(post_samples, truth)