--- title: "Comparison families, adjusted p-values and plot annotations" author: "Raban Heller" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Comparison families, adjusted p-values and plot annotations} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} resource_files: - figures/logo.svg - figures/logo-dark.svg --- ```{r setup, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") has_ggplot2 <- requireNamespace("ggplot2", quietly = TRUE) ``` A study often reports several related comparisons. Whether their p-values are adjusted, and over which set of hypotheses, is part of the analysis and is decided in infercalcR. infereportR then shows the stored raw or adjusted values and keeps the definition of the family attached to every table, text and plot label. This vignette uses the `PlantGrowth` and `ToothGrowth` data sets that ship with R. ```{r packages} library(infercalcR) library(infereportR) ``` ## One result for a family of comparisons `PlantGrowth` has three groups, which give three pairwise comparisons. Each one is a Welch two-sample t-test computed by infercalcR. The finished rows are then combined into a single result with stable row identifiers and explicit contrast labels, so that the family can be adjusted as a whole. ```{r family} weight <- PlantGrowth$weight group <- PlantGrowth$group spec <- method_spec("t_means_two_independent_welch") pairs <- data.frame( row_id = c("trt1-ctrl", "trt2-ctrl", "trt2-trt1"), comparison = c("trt1", "trt2", "trt2"), reference = c("ctrl", "ctrl", "trt1") ) rows <- do.call(rbind, lapply(seq_len(nrow(pairs)), function(i) { fit <- run_inference(spec, x = weight[group == pairs$comparison[i]], y = weight[group == pairs$reference[i]]) result_data(fit) })) rows$row_id <- pairs$row_id rows$variable_id <- "weight" rows$contrast <- paste(pairs$comparison, "minus", pairs$reference) unadjusted <- new_inference_result(rows, list( method_id = "t_means_two_independent_welch", result_id = "plantgrowth-pairwise", population = "PlantGrowth: 30 plants, 10 per group" )) ``` The Holm adjustment is a calculation, so it also happens in infercalcR. It is applied once to the complete declared family: ```{r adjust} family <- adjust_inference(unadjusted, method = "holm", family_id = "all-pairwise") ``` ## Choose the stored p-value The `p` argument selects which stored p-value is displayed. Nothing is recalculated; the choice only decides which column feeds `p_display`. ```{r p-choice} cols <- c("row_id", "contrast", "estimate_display", "p.value", "p.adjusted", "p_display") report_table(family, p = "raw")[, cols] report_table(family, p = "adjusted")[, cols] ``` Asking for adjusted values that were never computed is an error rather than a silent fallback to raw p-values: ```{r p-missing} try(report_table(unadjusted, p = "adjusted")) ``` ## Select rows after adjustment `rows` picks rows for display. Selecting happens after the adjustment, and the metadata still describes the full family, including its size and members: ```{r rows} one <- report_table(family, rows = "trt2-trt1", p = "adjusted") one[, cols] str(report_metadata(one)$adjustment) ``` Full text names the adjustment method next to the displayed p-value: ```{r text} cat(report_text(family, p = "adjusted"), sep = "\n") ``` ## Stars and unavailable results The `stars` preset uses the thresholds 0.05, 0.01, 0.001 and 0.0001: ```{r stars} stars <- report_text(family, p = "adjusted", style = "stars") data.frame(row_id = names(stars), label = as.character(stars)) ``` A result that could not be computed keeps its status. It never receives stars or `ns`, and full text states the recorded reason: ```{r unavailable} incomplete <- new_inference_result( data.frame( row_id = c("a", "b"), estimand = "mean difference", estimate = c(0.8, NA), conf.low = c(0.1, NA), conf.high = c(1.5, NA), conf.level = 0.95, p.value = c(0.03, NA), status = c("ok", "undefined"), reason = c(NA, "fewer than two observations in one group") ), list(method_id = "example", result_id = "incomplete") ) cat(report_text(incomplete, style = "stars"), sep = "\n") cat(report_text(incomplete), sep = "\n") ``` ## Brackets on a plot The remaining sections need the optional ggplot2 package; their chunks run only when it is installed. Each bracket is described by one row of a positions data frame. Coordinates are in data units. `group1` and `group2` name the endpoints, and `groups` gives the order of the groups on the discrete axis. ```{r brackets, eval = has_ggplot2, fig.width = 6, fig.height = 4.5, fig.alt = "Box plots of dried plant weight for ctrl, trt1 and trt2 with three brackets, one per pairwise comparison, each labeled with its Holm-adjusted p-value."} library(ggplot2) positions <- data.frame( row_id = pairs$row_id, group1 = pairs$reference, group2 = pairs$comparison, y_position = c(6.5, 7.3, 6.9) ) base <- ggplot(PlantGrowth, aes(group, weight)) + geom_boxplot(width = 0.5) + labs(x = NULL, y = "Dried weight") base + stat_inference(family, positions, groups = levels(group), p = "adjusted", tip_length = 0.1) ``` Changing the order of the axis moves the brackets but never reverses a contrast. Below, `trt2` is drawn first, and each label still shows the stored difference with its original sign, for example `trt2` minus `trt1`: ```{r reordered, eval = has_ggplot2, fig.width = 6, fig.height = 4.5, fig.alt = "The same box plots with the group order trt2, trt1, ctrl; each bracket shows the stored difference and its interval."} reordered <- PlantGrowth reordered$group <- factor(reordered$group, levels = c("trt2", "trt1", "ctrl")) ggplot(reordered, aes(group, weight)) + geom_boxplot(width = 0.5) + stat_inference(family, positions, groups = levels(reordered$group), label = "estimate_ci", tip_length = 0.1) + labs(x = NULL, y = "Dried weight") ``` ## Text labels for omnibus tests An omnibus test has no pair of endpoints, so it is annotated with text only. Its position gives `x` and `y_position` instead of `group1` and `group2`: ```{r omnibus} omnibus <- run_inference(method_spec("anova_one_way_welch"), x = weight, group = group, result_id = "welch-anova") report_table(omnibus)[, c("component", "estimand", "statistic_display", "p_display")] ``` ```{r omnibus-plot, eval = has_ggplot2, fig.width = 6, fig.height = 4.5, fig.alt = "Box plots with the three pairwise brackets and a text label for the Welch one-way analysis of means above them."} pairwise_plot <- base + stat_inference(family, positions, groups = levels(group), p = "adjusted", tip_length = 0.1) text_position <- data.frame(row_id = result_data(omnibus)$row_id, x = 2, y_position = 7.7) annotate_inference(pairwise_plot, omnibus, text_position, template = "Welch ANOVA: {p}") ``` `annotate_inference()` adds the same layers as `stat_inference()` but first checks that the existing plot uses Cartesian coordinates or `coord_flip()`. Polar and other nonlinear coordinates are rejected. ## Flipped coordinates and transformed scales Positions are always given in the original data units. With `scale_y_log10()`, a `y_position` of 100 is drawn at 2 on the log scale, transformed exactly once. For horizontal plots, adjust the text justification and leave room for the labels: ```{r flipped, eval = has_ggplot2, fig.width = 6, fig.height = 4, fig.alt = "Horizontal box plots of dried plant weight with the three pairwise brackets and their labels to the right of each bracket."} pairwise_flipped <- base + stat_inference(family, positions, groups = levels(group), p = "adjusted", tip_length = 0.1, text_hjust = -0.1, text_vjust = 0.5) + scale_y_continuous(expand = expansion(mult = c(0.05, 0.25))) + coord_flip() pairwise_flipped ``` ## Results per facet Results that belong to subgroups carry their subgroup key as a column, and the result metadata declares it in `facet_vars`. Here the two supplements in `ToothGrowth` are compared separately at each dose: ```{r facets-data} tg <- ToothGrowth doses <- sort(unique(tg$dose)) by_dose <- do.call(rbind, lapply(doses, function(d) { fit <- run_inference(spec, x = tg$len[tg$dose == d & tg$supp == "OJ"], y = tg$len[tg$dose == d & tg$supp == "VC"]) out <- result_data(fit) out$dose <- d out })) by_dose$row_id <- paste0("OJ-VC-dose-", doses) by_dose$contrast <- "OJ minus VC" dose_results <- new_inference_result(by_dose, list( method_id = "t_means_two_independent_welch", result_id = "toothgrowth-by-dose", population = "ToothGrowth: 10 animals per supplement and dose", facet_vars = "dose" )) report_table(dose_results)[, c("row_id", "dose", "estimate_display", "p_display")] ``` These three p-values are not adjusted. The positions repeat the facet key, and it must match the stored rows exactly; a missing or conflicting key is an error instead of a label copied into every panel: ```{r facets-plot, eval = has_ggplot2, fig.width = 7, fig.height = 3.5, fig.alt = "Box plots of odontoblast length by supplement in three panels, one per dose, each with a bracket between OJ and VC labeled with its p-value."} facet_positions <- data.frame( row_id = by_dose$row_id, dose = doses, group1 = "OJ", group2 = "VC", y_position = c(24, 30, 36) ) ggplot(tg, aes(supp, len)) + geom_boxplot(width = 0.5) + facet_wrap(~dose) + stat_inference(dose_results, facet_positions, groups = levels(tg$supp), tip_length = 1) + labs(x = "Supplement", y = "Odontoblast length") ``` Annotations are placed only where the positions say. Automatic placement, automatic dodging and free facet scales are not inferred. For dodged groups, give numeric `xmin` and `xmax` instead of `group1` and `group2`. ```{r acknowledgments, child = "children/_acknowledgments.Rmd"} ```