+ Welcome!

Hi, my name is Melody!

mail_outline Email: melodyyhuang [asperand] berkeley [dot] edu

Twitter: @melodyyhuang

I'm a Statistics Ph.D. candidate at University of California, Berkeley, working with Prof. Erin Hartman. My work broadly focuses on causal inference and the intersection of statistics and social science.

Outside of statistics, I spend my time thinking about coffee beans, how it's not fair that women can't safely run at night, and optimal board game strategies.

Recent News

  • [May 2022] I will be at ACIC, presenting on my work in sensitivity analysis. Come say hi!
  • [Apr. 2022] I will presenting on sensitivity analysis for survey weighting at MPSA 2022!
  • [Feb. 2022] My paper on sensitivity analysis for external validity is now on ArXiv (link).
  • [Jan. 2022] My work on improving covariate balancing approaches (joint work with Brian Vegetabile, Lane Burgette, Claude Setodji, and Beth Ann Griffin) will be appearing in Epidemiology!
  • [Nov. 2021] Our paper on improving the generalization of experimental results (joint work with Naoki Egami, Erin Hartman, and Luke Miratrix), is now on AriXiv (link).
  • [Oct. 2021] I am presenting my work on sensitivity analysis for generalizing experimental results at WSDS 2021!

+ Papers

Sensitivity Analysis in the Generalization of Experimental Results

Randomized control trials (RCT’s) allow researchers to estimate causal effects in an experimental sample with minimal identifying assumptions. However, to generalize a causal effect from an RCT to a target population, researchers must adjust for a set of treatment effect moderators. In practice, it is impossible to know whether the set of moderators has been properly accounted for. In the following paper, I propose a three parameter sensitivity analysis for the generalization of experimental results using weighted estimators, with several advantages over existing methods. First, the framework does not require any parametric assumption on either the selection or treatment effect heterogeneity mechanism. Second, I show that the sensitivity parameters are guaranteed to be bounded and propose (1) a diagnostic for researchers to determine how robust a point estimate is to killer confounders, and (2) an adjusted calibration approach for researchers to accurately benchmark the parameters using existing data. Finally, I demonstrate that the proposed framework can be easily extended to the class of doubly robust, augmented weighted estimators. The sensitivity analysis framework is applied to a set of Jobs Training Program experiments.

Leveraging Observational Outcomes to Improve the Generalization of Experimental Results
with Naoki Egami, Erin Hartman, and Luke Miratrix

Randomized control trials are often considered the gold standard in causal inference due to their high internal validity. Despite its importance, generalizing experimental results to a target population is challenging in social and biomedical sciences. Recent papers clarify the assumptions necessary for generalization and develop various weighting estimators for the population average treatment effect (PATE). However, in practice, many of these methods result in large variance and little statistical power, thereby limiting the value of the PATE inference. In this article, we propose post-residualized weighting, in which information about the outcome measured in the observational population data is used to improve the efficiency of many existing popular methods without making additional assumptions. We empirically demonstrate the efficiency gains through simulations and apply our proposed method to a set of jobs training program experiments.

Higher Moments for Optimal Balance Weighting in Causal Estimation
with Brian Vegetabile, Lane Burgette, Claude Setodji, and Beth Ann Griffin

We expand upon a simulation study that compared three promising methods for estimating weights for assessing the average treatment effect on the treated for binary treatments: generalized boosted models, covariate-balancing propensity scores, and entropy balance. The original study showed that generalized boosted models can outperform covariate-balancing propensity scores, and entropy balance when there are likely to be non-linear associations in both the treatment assignment and outcome models and when the other two models are fine-tuned to obtain balance only on first-order moments. We explore the potential benefit of using higher-order moments in the balancing conditions for covariate-balancing propensity scores and entropy balance. Our findings showcase that these two models should, by default, include higher order moments and focusing only on first moments can result in substantial bias in estimated treatment effect estimates from both models that could be avoided using higher moments.

Design and Analysis of Experiments with Non-Compliance under Principal Ignorability
with Erin Hartman

Even in the best-designed experiment, noncompliance with treatment assignment can complicate analysis. Under one-way noncompliance, researchers typically rely on an instrumental variables approach, under an exclusion restriction assumption, to identify the complier average causal effect (CACE). This approach suffers from high variance, particularly when the experiment has a low compliance rate. The following paper suggests blocking designs that can help overcome precision losses in the face of high rates of noncompliance in experiments when a placebo-controlled design is infeasible. We also introduce the principal ignorability assumption and a class of principal score weighted estimators, which are largely absent from the experimental political science literature. We then introduce the ''block principal ignorability'' assumption which, when combined with a blocking design, suggests a simple difference-in-means estimator for estimating the CACE. We show that blocking can improve precision of both IV and principal score weighting approaches, and further show that our simple, design-based solution has superior performance to both principal score weighting and instrumental variables under blocking. Finally, in a re-evaluation of the Gerber, Green, and Nickerson (2003) study, we find that blocked, principal ignorability approaches to estimation of the CACE, including our blocked difference-in-means and principal score weighting estimators, result in confidence intervals roughly half the size of traditional instrumental variable approaches.

+ Talks

Sensitivity Analysis for Generalizing Experimental Results

  • Berkeley Causal Inference Seminar (March 2021)
  • Women in Statistics and Data Science (October 2021)
  • Joint Statistical Meetings (August 2021)
  • RAND Statistics Group, Invited Brown Bag (July 2021)

Sensitivity Analysis for Survey Weighting

  • 79th Annual Midwest Political Science Conference (April 2021)
  • 2021 APSA Annual Meeting (October 2021)

Leveraging Population Outcomes to Improve the Generalization of Experimental Results

  • 79th Annual Midwest Political Science Conference - (Invited Session) (April 2021)
  • 38th Annual Meeting of the Society for Political Methodology (July 2021)
  • Harvard Experimental Political Science Conference (April 2021)
  • Harvard University, Miratrix C.A.R.E.S. Lab (December 2020)
  • 37th Annual Meeting of the Society for Political Methodology (July 2020) (Poster)
    Awarded Society of Political Methodology Best Graduate Student Poster in Methods
  • RAND Statistics Group, Invited Brown Bag (July 2019)

Balancing Higher Moments Matter for Causal Estimation

  • RAND Center for Causal Inference (September 2020)

Designs and Considerations in Experiments with Low Compliance Rates

  • 36th Annual Meeting of the Society for Political Methodology (July 2019) (Poster)
  • Google Ph.D. Research Conference (July 2019) (Poster)