Object Documentation

Sensemakr.sensemakr

Object comprised of necessary parameters and statistics for sensitivity analysis

Base.summary(sense_obj::sensemakr, digits::Int64 = 3)

Print a summary of the sensitivity results for sense_obj, including robustness value, extreme confounding scenario, and benchmark bounding.

Arguments:

• sense_obj: object to be summarized.
• digits (default: 3): number of digits to round numbers to.

Sensemakr.plot(sense_obj::sensemakr; kwargs...)

Provide the contour and extreme scenario sensitivity plots of the sensitivity analysis results obtained with the function sensemakr

They are basically dispatchers to the core plot functions ovbcontourplot and ovbextremeplot

Arguments:

• sense_obj: object with the statistics to be plotted.
• plot_type (default: "contour"): Either "extreme" or "contour".
• sensitivity_of (default: "estimate"): Either "estimate" or "t-value"

ovb_minimal_reporting(sense_obj::sensemakr, digits::Int64 = 3, res_display::Bool = true)

This function returns the HTML code for a table summarizing the sensemakr object.

Arguments:

• digits (default: 3): number of digits to round numbers to.
• res_display (default: true): whether to display the table.

ovb_contour_plot(estimate::Float64, se::Float64, dof::Int64, kwargs...)

Contour plots of omitted variable bias for sensitivity analysis.

The horizontal axis of the plot shows hypothetical values of the partial R2 of the unobserved confounder(s) with the treatment. The vertical axis shows hypothetical values of the partial R2 of the unobserved confounder(s) with the outcome. The contour levels represent the adjusted estimates (or t-values) of the treatment effect.

The reference points are the bounds on the partial R2 of the unobserved confounder if it were k times “as strong” as the observed covariates used for benchmarking (see arguments kd and ky). The dotted red line show the chosen critical threshold (for instance, zero): confounders with such strength (or stronger) are sufficient to invalidate the research conclusions. All results are exact for single confounders and conservative for multiple/nonlinear confounders.

See Cinelli and Hazlett (2020) for details.

Arguments:

• estimate: estimated parameter of the variable of interest
• se: standard error of the estimated parameter of the variable of interest
• dof degrees of freedom of the estimated regression model
• r2dz_x (default: nothing): number or array containing hypothetical partial R2 of unobserved confounder Z with treatment D, given covariates X.
• r2yz_dx (default: nothing): number or array containing hypothetical partial R2 of unobserved confounder Z with outcome Y, given covariates X and treatment D.
• sensitivity_of (default: "estimate"): either "estimate" or "t-value"
• kd (default: 1): a float or vector of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.
• ky (default: nothing): a float or vector of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.
• benchmark_covariates (default: nothing): a string or vector of strings with the names of the variables to use for benchmarking.
• bound_label (default: nothing): label of the bound variable.
• reduce (default: true): whether to reduce (true) or increase (false) the estimate due to putative confounding.
• estimate_threshold (default: 0): threshold line to emphasize when contours correspond to estimate.
• t_threshold (default: 2): threshold line to emphasize when contours correspond to t-value.
• lim (default: nothing): x axis maximum.
• lim_y (default: nothing): y axis maximum.
• col_contour (default: "black"): color of contour lines.
• col_thr_line (default: "red"): color of the threshold contour line.
• label_text (default: true): whether to include label text.
• label_bump_x (default: nothing): x axis position of label above 0.
• label_bump_y (default: nothing): y axis position of label above 0.
• plot_margin_fraction (default: 0.05): margin fraction added to the top of lim and lim_y.
• round_dig (default: 3): rounding digit of the display numbers.
• n_levels (default: nothing): maximum number of countours in the plot.

ovb_contour_plot(model::StatsModels.TableRegressionModel, treatment::String, kwargs...)

Uses model and treatment to obtain the estimate, standard error, and degrees of freedom; it then dispatches them to the relevant method.

ovb_contour_plot(sense_obj::sensemakr, kwargs...)

Extracts the estimate, standard error, and degrees of freedom from sense_obj; it then dispatches them to the relevant method.

add_bound_to_contour(r2dz_x::Union{Real, Array{<:Real}}, r2yz_dx::Union{Real, Array{<:Real}; kwargs...}

Add bound label to the contour plot of omitted variable bias for sensitivity analysis.

The reference points are the bounds on the partial R2 of the unobserved confounder if it were k times "as strong" as the observed covariate used for benchmarking (see arguments kd and ky).

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out, as implied by z being kd-times as strong as the benchmark_covariates.
• r2yz_dx partial $R^2$ of a putative unobserved confounder "z" with the outcome variable "y", with observed covariates "x" and the treatment variable "d" partialed out, as implied by z being ky-times as strong as the benchmark_covariates.
• kd (default: 1): a float or vector of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.
• ky (default: nothing): same as kd except measured in terms of strength of association with the outcome variable.
• bound_value (default: nothing): the value of the reference point.
• bound_label (default: nothing): a string that label the reference point.
• sensitivity_of (default: nothing): either “estimate” or “t-value”.
• label_text (default: true): whether to include the label text.
• label_bump_x (default: nothing): x axis position of label above 0.
• label_bump_y (default: nothing): y axis position of label above 0.
• n_levels (default: nothing): maximum number of countours in the plot.

add_bound_to_contour(bound::DataFrame; kwargs...)

Extracts r2dzx, r2yzdx, and bound_label vectors from bound and dispatches them to the relevant method.

ovb_extreme_plot(estimate::Float64, se::Float64, dof::Int64; kwargs...)

Extreme scenario plots of omitted variable bias for sensitivity analysis.

The horizontal axis shows the partial R2 of the unobserved confounder(s) with the treatment. The vertical axis shows the adjusted treatment effect estimate. The partial R2 of the confounder with the outcome is represented by different curves for each scenario, as given by the parameter r2yz_dx. The red marks on horizontal axis are bounds on the partial R2 of the unobserved confounder kd times as strong as the covariates used for benchmarking. The dotted red line represent the threshold for the effect estimate deemed to be problematic (for instance, zero).

See Cinelli and Hazlett (2020) for details.

Arguments:

• estimate: estimated parameter of the variable of interest
• se: standard error of the estimated parameter of the variable of interest
• dof degrees of freedom of the estimated regression model
• sensitivity_of (default: "estimate"): either "estimate" or "t-value". This function is currently only defined for "estimate"
• benchmark_covariates (default: nothing): a string or vector of strings with the names of the variables to use for benchmarking.
• kd (default: 1): a float or vector of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.
• ky (default: nothing): a float or vector of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.
• r2dz_x (default: nothing): a float or vector of floats with the partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out, as implied by z being kd-times as strong as the benchmark_covariates.
• r2yz_dx (default: [1.0, 0.75, 0.5]): a float or vector of floats with the partial $R^2$ of a putative unobserved confounder "z" with the outcome variable "y", with observed covariates "x" and the treatment variable "d" partialed out, as implied by z being ky-times as strong as the benchmark_covariates.
• reduce (default: true): whether to reduce (true) or increase (false) the estimate due to putative confounding.
• threshold (default: 0): threshold line to emphasize when contours correspond to estimate.
• lim (default: nothing): x axis maximum.
• lim_y (default: nothing): y axis maximum.
• xlab (default: nothing): x axis label text.
• ylab (default: nothing): y axis label text.

ovb_extreme_plot(sense_obj::sensemakr; kwargs...)

Extracts the estimate, standard error, and degrees of freedom from sense_obj; it then dispatches them to the relevant method.

ovb_bounds(model::StatsModels.TableRegressionModel, treatment::String; kwargs...)

Returns a DataFrame which provides bounds on the strength of unobserved confounders using observed covariates, as in Cinelli and Hazlett (2020).

Arguments

• model: object for the restricted regression you have provided.
• treatment: name of the treatment variable or variable of interest.
• benchmark_covariates: names of variables to use for benchmark bounding.
• kd (default: 1): multiple of the strength of association between a benchmark variable and the treatment.
• ky: same as kd except measured in terms of strength of association with the outcome variable.
• alpha (default: 0.05): significance level for the robustness value $RV_{qa}$ to render the estimate not significant.
• h0 (default: 0): null hypothesis effect size.
• reduce (default: true): whether to reduce (true) or increase (false) the estimate due to putative computing.
• bound (default: "partial_r2"): type of bound to perform; as of now, only partial $R^2$ bounding is allowed.
• adjusted_estimates (default: true): whether to compute bias-adjusted estimates, standard errors, and t-statistics.

ovb_partial_r2_bound(; kwargs...)

Returns a DataFrame with the bounds on the strength of the unobserved confounder.

Adjusted estimates, standard errors and t-values (among other quantities) need to be computed manually by the user using those bounds with the functions adjustedestimate, adjustedse and adjusted_t.

Arguments

• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• r2dxj_x: float with the partial R2 of covariate Xj with the treatment D (after partialling out the effect of the remaining covariates X, excluding Xj).
• r2yxj_dx: float with the partial R2 of covariate Xj with the outcome Y (after partialling out the effect of the remaining covariates X, excluding Xj).
• benchmark_covariates: a string or vector of strings with names of the variables to use for benchmark bounding.
• kd (default: 1): a float or list of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.
• ky (default: nothing): same as kd except measured in terms of strength of association with the outcome variable.

robustness_value(; model::Union{Nothing, StatsModels.TableRegressionModel} = nothing, covariates::Union{Nothing, String, Vector{String}} = nothing, t_statistic::Union{Real, Nothing} = nothing, dof::Union{Nothing, Int64} = nothing, kwargs...)

Compute the robustness value of a regression coefficient.

The robustness value describes the minimum strength of association (parameterized in terms of partial R2) that omitted variables would need to have both with the treatment and with the outcome to change the estimated coefficient by a certain amount (for instance, to bring it down to zero).

For instance, a robustness value of 1% means that an unobserved confounder that explain 1% of the residual variance of the outcome and 1% of the residual variance of the treatment is strong enough to explain away the estimated effect. Whereas a robustness value of 90% means that any unobserved confounder that explain less than 90% of the residual variance of both the outcome and the treatment assignment cannot fully account for the observed effect. You may also compute robustness value taking into account sampling uncertainty. See details in Cinelli and Hazlett (2020).

Arguments:

• model: object for the restricted regression you have provided.
• covariates: names of the variables to use for benchmark bounding.
• t_statistic: the t_statistic for the restricted model regression.
• dof: degrees of freedom of the restricted regression
• q (default: 1): a float with the percent to reduce the point estimate by for the robustness value RV_q.
• alpha (default = 1.0): a float with the significance level for the robustness value RV_qa to render the estimate not significant.

partial_r2(; model::Union{Nothing, StatsModels.TableRegressionModel} = nothing, covariates::Union{Nothing, String, Vector{String}} = nothing, t_statistic::Union{Real, Nothing} = nothing, dof::Union{Nothing, Int64} = nothing)

Compute the partial R2 for a linear regression model.

The partial R2 describes how much of the residual variance of the outcome (after partialing out the other covariates) a covariate explains.

The partial R2 can be used as an extreme-scenario sensitivity analysis to omitted variables. Considering an unobserved confounder that explains 100% of the residual variance of the outcome, the partial R2 describes how strongly associated with the treatment this unobserved confounder would need to be in order to explain away the estimated effect.

For details see Cinelli and Hazlett (2020).

Arguments:

• model: object for the restricted regression you have provided.
• covariates: names of the variables to use for benchmark bounding.
• t_statistic: the t_statistic for the restricted model regression.
• dof: degrees of freedom of the restricted regression

partial_f2(; model::Union{Nothing, StatsModels.TableRegressionModel} = nothing, covariates::Union{Nothing, String, Vector{String}} = nothing, t_statistic::Union{Real, Nothing} = nothing, dof::Union{Nothing, Int64} = nothing)

Compute the partial (Cohen's) f2 for a linear regression model.

The partial (Cohen's) f2 is a common measure of effect size (a transformation of the partial R2) that can also be used directly for sensitivity analysis using a bias factor table. For details see Cinelli and Hazlett (2020).

Arguments

• model: object for the restricted regression you have provided.
• covariates: names of the variables to use for benchmark bounding.
• t_statistic: the t_statistic for the restricted model regression.
• dof: degrees of freedom of the restricted regression

Partial R2 of groups of covariates in a linear regression model.

This function computes the partial R2 of a group of covariates in a linear regression model. Multivariate version of the partial_r2 function; see that for more details.

Arguments

• model: object for the restricted regression you have provided.
• covariates: names of the variables to use for benchmark bounding.
• t_statistic: t_statistic for the restricted model regression.
• p: number of parameters.
• dof: degrees of freedom of the restricted regression.

adjusted_estimate(r2dz_x::Union{Float64, Vector{Float64}}, r2yz_dx::Union{Float64, Vector{Float64}}; kwargs...)

Compute the bias-adjusted coefficient estimate.

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out.
• r2yz_dx: partial $R^2$ of a putative unobserved confounder "z" with the outcome "y", with observed covariates "x" and treatment "d" partialed out.
• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• estimate: float with the unadjusted estimate of the coefficient for the independent variable of interest.
• se: float with the unadjusted standard error of the regression.
• dof: an int with the degrees of freedom of the regression.
• reduce (default: true): whether to reduce (true) or increase (false) the estimate due to putative confounding.

adjusted_se(r2dz_x::Union{Float64, Vector{Float64}}, r2yz_dx::Union{Float64, Vector{Float64}}; kwargs...)

Compute the bias-adjusted regression standard error.

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out.
• r2yz_dx: partial $R^2$ of a putative unobserved confounder "z" with the outcome "y", with observed covariates "x" and treatment "d" partialed out.
• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• se: float with the unadjusted standard error of the regression.
• dof: an int with the degrees of freedom of the regression.

adjusted_t(r2dz_x::Union{Float64, Vector{Float64}}, r2yz_dx::Union{Float64, Vector{Float64}}; kwargs...)

Compute bias-adjusted t-statistic, (adjustedestimate - h0) / adjustedse.

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out.
• r2yz_dx: partial $R^2$ of a putative unobserved confounder "z" with the outcome "y", with observed covariates "x" and treatment "d" partialed out.
• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• estimate: float with the unadjusted estimate of the coefficient for the independent variable of interest.
• se: float with the unadjusted standard error of the regression.
• dof: an int with the degrees of freedom of the regression.
• reduce (default: true): whether to reduce (true) or increase (false) the estimate due to putative confounding.
• h0: (default: 0): the test value for null hypothesis.

adjusted_partial_r2(r2dz_x::Union{Float64, Vector{Float64}}, r2yz_dx::Union{Float64, Vector{Float64}}; kwargs...)

Compute the bias-adjusted partial R2, based on adjusted_t

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out.
• r2yz_dx: partial $R^2$ of a putative unobserved confounder "z" with the outcome "y", with observed covariates "x" and treatment "d" partialed out.
• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• estimate: float with the unadjusted estimate of the coefficient for the independent variable of interest.
• se: float with the unadjusted standard error of the regression.
• dof: an int with the degrees of freedom of the regression.
• reduce (default: true): whether to reduce (true) or increase (false) the estimate due to putative confounding.
• h0: (default: 0): the test value for null hypothesis.

bias(r2dz_x::Union{Float64, Vector{Float64}}, r2yz_dx::Union{Float64, Vector{Float64}}; kwargs...)

Compute the omitted variable bias for the partial R2 parameterization.

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out.
• r2yz_dx: partial $R^2$ of a putative unobserved confounder "z" with the outcome "y", with observed covariates "x" and treatment "d" partialed out.
• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• se: float with the unadjusted standard error of the regression.
• dof: an int with the degrees of freedom of the regression.

relative_bias(r2dz_x::Union{Float64, Vector{Float64}}, r2yz_dx::Union{Float64, Vector{Float64}}; kwargs...)

Compute the relative bias for the partial R2 parameterization.

Arguments

• r2dz_x: partial $R^2$ of a putative unobserved confounder "z" with the treatment variable "d", with observed covariates "x" partialed out.
• r2yz_dx: partial $R^2$ of a putative unobserved confounder "z" with the outcome "y", with observed covariates "x" and treatment "d" partialed out.
• model: StatsModels.TableRegressionModel object for the restricted regression you have provided.
• treatment: string with the name of the treatment variable of interest.
• estimate: float with the unadjusted estimate of the coefficient for the independent variable of interest.
• se: float with the unadjusted standard error of the regression.
• dof: an int with the degrees of freedom of the regression.

Provide the example data of darfur based on a survey among Darfurian refugees in eastern Chad.

The data is on attitudes of Darfurian refugees in eastern Chad. The main “treatment” variable is directlyharmed, which indicates that the individual was physically injured during attacks on villages in Darfur, largely between 2003 and 2004. The main outcome of interest is peacefactor, a measure of pro-peace attitudes.

Key covariates include herderdar (whether they were a herder in Darfur), farmerdar (whether they were a farmer in Darfur), age, female (indicator for female), and past_voted (whether they report having voted in an earlier election, prior to the conflict).

A data frame with 1276 rows and 14 columns.

wouldvote

If elections were held in Darfur in the future, would you vote? (0/1)

peacefactor

A measure of pro-peace attitudes, from a factor analysis of several questions. Rescaled such that 0 is minimally pro-peace and 1 is maximally pro-peace.

peace_formerenemies

Would you be willing to make peace with your former enemies? (0/1)

peace_jjindiv

Would you be willing to make peace with Janjweed individuals who carried out violence? (0/1)

peace_jjtribes

Would you be willing to make peace with the tribes that were part of the Janjaweed? (0/1)

gossoldierexecute

Should Government of Sudan soldiers who perpetrated attacks on civilians be executed? (0/1)

directlyharmed

A binary variable indicating whether the respondent was personally physically injured during attacks on villages in Darfur largely between 2003-2004. 529 respondents report being personally injured, while 747 do not report being injured.

age

Age of respondent in whole integer years. Ages in the data range from 18 to 100.

farmer_dar

The respondent was a farmer in Darfur (0/1). 1,051 respondents were farmers, 225 were not.

herder_dar

The respondent was a herder in Darfur (0/1). 190 respondents were farmers, 1,086 were not.

pastvoted

The respondent reported having voted in a previous election before the conflict (0/1). 821 respondents reported having voted in a previous election, 455 reported not having voted in a previous election.

hhsize_darfur

Household size while in Darfur.

village

Factor variable indicating village of respondent. 486 unique villages are accounted for in the data.

female

The respondent identifies as female (0/1). 582 respondents are female-identified, 694 are not.

Cinelli, C. and Hazlett, C. (2020), “Making Sense of Sensitivity: Extending Omitted Variable Bias.” Journal of the Royal Statistical Society, Series B (Statistical Methodology).

Hazlett, Chad. (2019) “Angry or Weary? How Violence Impacts Attitudes toward Peace among Darfurian Refugees.” Journal of Conflict Resolution: 0022002719879217.