A study often reports several related comparisons. Whether their
p-values are adjusted, and over which set of hypotheses, is part of the
analysis and is decided in infercalcR. infereportR then shows the stored
raw or adjusted values and keeps the definition of the family attached
to every table, text and plot label. This vignette uses the
PlantGrowth and ToothGrowth data sets that
ship with R.
PlantGrowth has three groups, which give three pairwise
comparisons. Each one is a Welch two-sample t-test computed by
infercalcR. The finished rows are then combined into a single result
with stable row identifiers and explicit contrast labels, so that the
family can be adjusted as a whole.
weight <- PlantGrowth$weight
group <- PlantGrowth$group
spec <- method_spec("t_means_two_independent_welch")
pairs <- data.frame(
row_id = c("trt1-ctrl", "trt2-ctrl", "trt2-trt1"),
comparison = c("trt1", "trt2", "trt2"),
reference = c("ctrl", "ctrl", "trt1")
)
rows <- do.call(rbind, lapply(seq_len(nrow(pairs)), function(i) {
fit <- run_inference(spec,
x = weight[group == pairs$comparison[i]],
y = weight[group == pairs$reference[i]])
result_data(fit)
}))
rows$row_id <- pairs$row_id
rows$variable_id <- "weight"
rows$contrast <- paste(pairs$comparison, "minus", pairs$reference)
unadjusted <- new_inference_result(rows, list(
method_id = "t_means_two_independent_welch",
result_id = "plantgrowth-pairwise",
population = "PlantGrowth: 30 plants, 10 per group"
))The Holm adjustment is a calculation, so it also happens in infercalcR. It is applied once to the complete declared family:
The p argument selects which stored p-value is
displayed. Nothing is recalculated; the choice only decides which column
feeds p_display.
cols <- c("row_id", "contrast", "estimate_display", "p.value", "p.adjusted", "p_display")
report_table(family, p = "raw")[, cols]
#> row_id contrast estimate_display p.value p.adjusted p_display
#> 1 trt1-ctrl trt1 minus ctrl -0.371 0.250382509 0.25038251 p = 0.250
#> 2 trt2-ctrl trt2 minus ctrl 0.494 0.047899256 0.09579851 p = 0.048
#> 3 trt2-trt1 trt2 minus trt1 0.865 0.009298405 0.02789521 p = 0.009
report_table(family, p = "adjusted")[, cols]
#> row_id contrast estimate_display p.value p.adjusted p_display
#> 1 trt1-ctrl trt1 minus ctrl -0.371 0.250382509 0.25038251 p = 0.250
#> 2 trt2-ctrl trt2 minus ctrl 0.494 0.047899256 0.09579851 p = 0.096
#> 3 trt2-trt1 trt2 minus trt1 0.865 0.009298405 0.02789521 p = 0.028Asking for adjusted values that were never computed is an error rather than a silent fallback to raw p-values:
rows picks rows for display. Selecting happens after the
adjustment, and the metadata still describes the full family, including
its size and members:
one <- report_table(family, rows = "trt2-trt1", p = "adjusted")
one[, cols]
#> row_id contrast estimate_display p.value p.adjusted p_display
#> 1 trt2-trt1 trt2 minus trt1 0.865 0.009298405 0.02789521 p = 0.028
str(report_metadata(one)$adjustment)
#> List of 1
#> $ :List of 5
#> ..$ family_id : chr "all-pairwise"
#> ..$ rows : chr [1:3] "trt1-ctrl" "trt2-ctrl" "trt2-trt1"
#> ..$ size : int 3
#> ..$ method : chr "holm"
#> ..$ interval_adjustment: chr "unchanged"Full text names the adjustment method next to the displayed p-value:
cat(report_text(family, p = "adjusted"), sep = "\n")
#> t_means_two_independent_welch; mean difference (x minus reference or y); trt1 minus ctrl: estimate -0.371; 95% interval [-1.03, 0.288] (Welch t; identity; pointwise); statistic -1.19; p = 0.250 (adjusted); recorded adjustment: holm; n = 20; excluded = 0.
#> t_means_two_independent_welch; mean difference (x minus reference or y); trt2 minus ctrl: estimate 0.494; 95% interval [0.00513, 0.983] (Welch t; identity; pointwise); statistic 2.13; p = 0.096 (adjusted); recorded adjustment: holm; n = 20; excluded = 0.
#> t_means_two_independent_welch; mean difference (x minus reference or y); trt2 minus trt1: estimate 0.865; 95% interval [0.249, 1.48] (Welch t; identity; pointwise); statistic 3.01; p = 0.028 (adjusted); recorded adjustment: holm; n = 20; excluded = 0.The remaining sections need the optional ggplot2 package; their chunks run only when it is installed.
Each bracket is described by one row of a positions data frame.
Coordinates are in data units. group1 and
group2 name the endpoints, and groups gives
the order of the groups on the discrete axis.
library(ggplot2)
positions <- data.frame(
row_id = pairs$row_id,
group1 = pairs$reference,
group2 = pairs$comparison,
y_position = c(6.5, 7.3, 6.9)
)
base <- ggplot(PlantGrowth, aes(group, weight)) +
geom_boxplot(width = 0.5) +
labs(x = NULL, y = "Dried weight")
base + stat_inference(family, positions, groups = levels(group),
p = "adjusted", tip_length = 0.1)Changing the order of the axis moves the brackets but never reverses
a contrast. Below, trt2 is drawn first, and each label
still shows the stored difference with its original sign, for example
trt2 minus trt1:
reordered <- PlantGrowth
reordered$group <- factor(reordered$group, levels = c("trt2", "trt1", "ctrl"))
ggplot(reordered, aes(group, weight)) +
geom_boxplot(width = 0.5) +
stat_inference(family, positions, groups = levels(reordered$group),
label = "estimate_ci", tip_length = 0.1) +
labs(x = NULL, y = "Dried weight")An omnibus test has no pair of endpoints, so it is annotated with
text only. Its position gives x and y_position
instead of group1 and group2:
omnibus <- run_inference(method_spec("anova_one_way_welch"),
x = weight, group = group, result_id = "welch-anova")
report_table(omnibus)[, c("component", "estimand", "statistic_display", "p_display")]
#> component estimand
#> 1 omnibus One-way analysis of means (not assuming equal variances)
#> statistic_display p_display
#> 1 5.18 p = 0.017pairwise_plot <- base + stat_inference(family, positions, groups = levels(group),
p = "adjusted", tip_length = 0.1)
text_position <- data.frame(row_id = result_data(omnibus)$row_id,
x = 2, y_position = 7.7)
annotate_inference(pairwise_plot, omnibus, text_position,
template = "Welch ANOVA: {p}")annotate_inference() adds the same layers as
stat_inference() but first checks that the existing plot
uses Cartesian coordinates or coord_flip(). Polar and other
nonlinear coordinates are rejected.
Positions are always given in the original data units. With
scale_y_log10(), a y_position of 100 is drawn
at 2 on the log scale, transformed exactly once. For horizontal plots,
adjust the text justification and leave room for the labels:
pairwise_flipped <- base +
stat_inference(family, positions, groups = levels(group), p = "adjusted",
tip_length = 0.1, text_hjust = -0.1, text_vjust = 0.5) +
scale_y_continuous(expand = expansion(mult = c(0.05, 0.25))) +
coord_flip()
pairwise_flippedResults that belong to subgroups carry their subgroup key as a
column, and the result metadata declares it in facet_vars.
Here the two supplements in ToothGrowth are compared
separately at each dose:
tg <- ToothGrowth
doses <- sort(unique(tg$dose))
by_dose <- do.call(rbind, lapply(doses, function(d) {
fit <- run_inference(spec,
x = tg$len[tg$dose == d & tg$supp == "OJ"],
y = tg$len[tg$dose == d & tg$supp == "VC"])
out <- result_data(fit)
out$dose <- d
out
}))
by_dose$row_id <- paste0("OJ-VC-dose-", doses)
by_dose$contrast <- "OJ minus VC"
dose_results <- new_inference_result(by_dose, list(
method_id = "t_means_two_independent_welch",
result_id = "toothgrowth-by-dose",
population = "ToothGrowth: 10 animals per supplement and dose",
facet_vars = "dose"
))
report_table(dose_results)[, c("row_id", "dose", "estimate_display", "p_display")]
#> row_id dose estimate_display p_display
#> 1 OJ-VC-dose-0.5 0.5 5.25 p = 0.006
#> 2 OJ-VC-dose-1 1.0 5.93 p = 0.001
#> 3 OJ-VC-dose-2 2.0 -0.08 p = 0.964These three p-values are not adjusted. The positions repeat the facet key, and it must match the stored rows exactly; a missing or conflicting key is an error instead of a label copied into every panel:
facet_positions <- data.frame(
row_id = by_dose$row_id, dose = doses,
group1 = "OJ", group2 = "VC",
y_position = c(24, 30, 36)
)
ggplot(tg, aes(supp, len)) +
geom_boxplot(width = 0.5) +
facet_wrap(~dose) +
stat_inference(dose_results, facet_positions, groups = levels(tg$supp),
tip_length = 1) +
labs(x = "Supplement", y = "Odontoblast length")Annotations are placed only where the positions say. Automatic
placement, automatic dodging and free facet scales are not inferred. For
dodged groups, give numeric xmin and xmax
instead of group1 and group2.
This package is part of an inference suite (infercalcR, inferguidR and infereportR) that is deeply inspired by DescrTab2 [1], developed at the Institute of Medical Biometry (IMBI), Heidelberg University, by Jan Meis, Lukas Baumann, Maximilian Pilz and Lukas Sauer. Its test-choice tree, its validation statement and its practice of computing each test with an established implementation are the model for how the suite chooses, computes and reports statistical tests. DescrTab2 was archived on CRAN on 18 October 2025; its source code remains available on GitHub. The methods-consulting pages of the University of Zurich [2], with their decision tree, decision assistant, worked examples and explanations, inspired the guidance in the suite. Power and sample-size planning and the distribution calculator follow G*Power 3.1 [3, 4]. We thank the authors of these works for making them openly available. The package README cites all software the package uses.