Skip to contents

Run Formal Benchmarking

run_benchmarking.Rd

Wrapper function to run formal benchmarking on a set of pre-specified covariates. Returns a data.frame containing the benchmarked parameter values, the estimated bias, MRCS, and minimum k_sigma and k_rho values for a killer confounder.

Usage

run_benchmarking(
  estimate,
  RV,
  formula = NULL,
  weights = NULL,
  pop_svy = NULL,
  sample_svy = NULL,
  Y = NULL,
  weighting_vars = NULL,
  benchmark_vars = "all",
  data = NULL,
  treatment = NULL,
  outcome = NULL,
  selection = NULL,
  population_targets = NULL,
  weighting_method = "ebal",
  weight_max = Inf,
  sigma2 = NULL,
  estimand = "ATT"
)

Arguments

estimate

Weighted estimate

RV

Robustness Value

formula

Raking formula for survey estimand

weights

A vector, containing the estimated survey weights

pop_svy

Survey object, containing the population the survey sample is being re-weighted to

sample_svy

Survey object, containing the survey sample being re-weighted

Y

outcome of interest (used for survey object)

weighting_vars

Vector of variables to use in the weights estimation for ATT or PATE

benchmark_vars

Vector of variables to benchmark parameters for. If benchmark_vars = 'all', benchmarking will be run across all variables included in the weights. If not set to all, benchmarking will be conducted across the covariates included in the vector.

data

A data.frame containing the observed covariates included in the weights; must include variables specified in weighting_vars

treatment

Denotes which variable is the treatment variable

outcome

Denotes which variable is the outcome variable

selection

Denotes which variable is the selection variable

population_targets

Population targets for the raking formula in surveys (optional, if not provided, will be generated from pop_svy)

weighting_method

Weighting method. Supports weighting methods from the package WeightIt.

weight_max

Maximum weight to trim at. Default set to Inf.

sigma2

If estimand = "PATE", sigma2 must specify the bound on treatment effect heterogeneity. For the other two estimands, the function will automatically calculate the sample variance across the control units, or the survey sample.

estimand

Specifies estimand; possible parameters include "ATT", "PATE", or "Survey"

Value

data.frame containing the benchmarked parameter values, the estimated bias, MRCS, and minimum k_sigma and k_rho values for a killer confounder for the set of pre-specified covariates.

Examples

# For the external validity setting: 
data(jtpa_women)
site_name <- "NE"
df_site <- jtpa_women[which(jtpa_women$site == site_name), ]
df_else <- jtpa_women[which(jtpa_women$site != site_name), ]

# Estimate unweighted estimator:
model_dim <- estimatr::lm_robust(Y ~ T, data = df_site)
PATE <- coef(lm(Y ~ T, data = df_else))[2]
DiM <- coef(model_dim)[2]
# Generate weights using observed covariates:
df_all <- jtpa_women
df_all$S <- ifelse(jtpa_women$site == "NE", 1, 0)
model_ps <- WeightIt::weightit(
  (1 - S) ~ . - site - T - Y, 
  data = df_all, method = "ebal", estimand = "ATT"
)
weights <- model_ps$weights[df_all$S == 1]
# Estimate IPW model:
model_ipw <- estimatr::lm_robust(Y ~ T, data = df_site, weights = weights)
ipw <- coef(model_ipw)[2]
# Estimate bound for var(tau):
vartau <- var(df_site$Y[df_site$T == 1]) - var(df_site$Y[df_site$T == 0])
RV <- robustness_value(estimate = ipw, b_star = 0, sigma2 = vartau, weights = weights)

# Select weighting variables:
weighting_vars <- names(df_all)[which(!names(df_all) %in% c("site", "S", "Y", "T"))]

# Run benchmarking:
df_benchmark <- run_benchmarking(
  weighting_vars = weighting_vars,
  data = df_all[, -1],
  treatment = "T", outcome = "Y", selection = "S",
  estimate = ipw,
  RV = RV, sigma2 = vartau,
  estimand = "PATE"
)

print(df_benchmark)
#>   variable R2_benchmark rho_benchmark    bias    MRCS k_sigma_min k_rho_min
#> 1 prevearn         0.04         -0.22 -115.00  -11.80        9.99     -2.92
#> 2      age         0.06         -0.09  -55.45  -24.47        6.91     -7.37
#> 3  married         0.11          0.00   -2.52 -539.33        3.82   -224.19
#> 4   hrwage         0.05          0.02   13.51  100.40        8.32     27.40
#> 5    black         0.20         -0.03  -33.11  -40.97        2.03    -24.72
#> 6 hispanic         0.14         -0.06  -62.63  -21.66        3.01    -10.30
#> 7  hsorged         0.12          0.10   95.06   14.27        3.51      6.22
#> 8 yrs_educ         0.00         -0.07   -5.23 -259.49      408.90     -9.85