Coverage probability

Compute and plot coverage of CI for different confidence level. Useful for fake data check.

compute_coverage(
  post_samples,
  truth,
  CI = seq(0, 1, 0.05),
  type = c("eti", "hdi")
)

plot_coverage(
  post_samples,
  truth,
  CI = seq(0, 1, 0.05),
  type = c("eti", "hdi")
)

Arguments

post_samples

Matrix of posterior samples. Rows represent a sample and columns represent variables.

truth

Vector of true parameter values (should be the same length as the number of columns in post_samples).

CI

Vector of confidence levels.

type

Type of confidence intervals: either "eti" (equal-tailed intervals) or "hdi" (highest density intervals).

Value

compute_coverage returns a Dataframe containing coverage (and 95% uncertainty interval for the coverage) for different confidence level (nominal coverage). plot_coverage returns a ggplot of the coverage as the function of the nominal coverage with 95% uncertainty interval.

Examples

N <- 100
N_post <- 1e3
truth <- rep(0, N)
post_samples <- sapply(rnorm(N, 0, 1), function(x) {rnorm(N_post, x, 1)})

compute_coverage(post_samples, truth)
#> # A tibble: 21 × 4
#>    Nominal Coverage  Lower Upper
#>      <dbl>    <dbl>  <dbl> <dbl>
#>  1    0        0    0      0    
#>  2    0.05     0.07 0.0286 0.139
#>  3    0.1      0.12 0.0636 0.200
#>  4    0.15     0.17 0.102  0.258
#>  5    0.2      0.26 0.177  0.357
#>  6    0.25     0.31 0.221  0.410
#>  7    0.3      0.33 0.239  0.431
#>  8    0.35     0.34 0.248  0.442
#>  9    0.4      0.38 0.285  0.483
#> 10    0.45     0.45 0.350  0.553
#> # ℹ 11 more rows
plot_coverage(post_samples, truth)