--- title: "Analysing two Delphi rounds offline" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Analysing two Delphi rounds offline} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} resource_files: - figures/logo.svg - figures/logo-dark.svg ---

```{r setup, include=FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") ``` This guide executes an **offline analysis workflow** with synthetic ratings. It needs neither Shiny nor PostgreSQL. It does not demonstrate recruitment, data collection, or production approval. The consensus thresholds are examples; a real study must justify and specify its rules in advance. ## Validate the protocol and snapshot ```{r protocol} library(delphyr) config <- demo_protocol() validate_protocol(config)$valid protocol <- new_protocol(config) protocol$analysis$consensus$group_policy ``` The default protocol requires adequate results in every designated panel group (`all_required_groups`). The compact snapshot example instead explicitly uses `pooled`: all ten ratings come from one group. That choice is frozen in its protocol; the analysis does not silently adjust the rule after seeing results. ```{r snapshot} first <- demo_snapshot() first$protocol$analysis$consensus$group_policy first$items ``` `new_snapshot()` accepts responses, assignments, an instrument, and a validated protocol. Its content hash binds those inputs to the round number. Construct a new snapshot to change inputs; the analysis rejects content that no longer matches its recorded hash. ## Read results with their denominators ```{r analysis} analysis <- analyse_round(first) analysis$decisions analysis$results[, c("stratum", "n_assigned", "n_submitted", "n_valid", "n_agree", "n_disagree", "p_agree", "classification")] ``` The ten valid ratings contain seven values in 7–9, one in 1–3, and two in the middle range. Their unrounded proportions are 0.7 and 0.1, producing `consensus_in` under this example rule. The empty `public_contributors` group remains visible as `insufficient_data`. Pooled consensus does not demonstrate consensus within every group. ```{r required-groups} required_groups <- protocol$analysis$consensus analyse_round(first, rules = required_groups)$decisions ``` The explicitly supplied alternative rule returns `insufficient_data`. Its own rule hash is recorded in the analysis; it does not alter the source snapshot. A prespecified primary analysis should use the frozen study protocol. ```{r missingness} with_missing <- analyse_round(demo_snapshot(c(NA, 4, 6, 7, 7, 8, 8, 9, 9, 9))) with_missing$denominators with_missing$missingness with_missing$decisions ``` In this helper, `NA` becomes an explicit `not_answered` status. Nine valid ratings fall below the example minimum of ten. Assigned people, submissions, and special response categories are counted separately; missingness does not become a zero rating. Median and quartiles use valid ratings, with type 7 quantiles. ## Construct and compare a second round The demo helper creates round 1. For the second round, construct a new snapshot with `round_number = 2L`. Participant identifiers and item meaning stay the same in this example. ```{r second-round} second_demo <- demo_snapshot(c(2, 4, 7, 7, 7, 8, 8, 9, 9, 9)) response_columns <- c("panelist_id", "item_code", "item_version", "dimension_code", "answer_status", "value_integer", "value_text") assignment_columns <- c("panelist_id", "group_code", "submitted") second <- new_snapshot( responses = second_demo$data[, response_columns], assignments = unique(second_demo$data[, assignment_columns]), items = second_demo$items, protocol = second_demo$protocol, round_number = 2L ) analyse_round(second)$decisions comparison <- compare_rounds(first, second) comparison[, c("status", "n_paired", "n_lost", "n_new", "median_absolute_change", "proportion_unchanged")] ``` Ten people are paired and eight ratings remain unchanged. Descriptive change does not automatically establish stability or end a study; stable disagreement is possible. Unpaired round means must not be interpreted as individual change. When an item's meaning changes, exclude it from direct comparison. An explicit mapping can document that review decision even when version numbers match: ```{r comparability} mapping <- data.frame( item_code = "I001", dimension_code = "relevance", previous_version = 1L, current_version = 1L, comparable = FALSE, reason = "Comparison excluded after methodological review" ) compare_rounds(first, second, mapping)[, c("status", "reason", "n_paired")] ``` New item versions are not comparable by default without an appropriate explicit mapping. A mapping records a methodological decision; it does not replace review. ## Prepare a feedback draft ```{r feedback} feedback <- prepare_feedback(analysis) validate_feedback(feedback) feedback$results[, c("stratum", "n_valid", "classification", "suppressed")] ``` By default, the draft contains only the overall result. Small groups must not be reconstructed by subtracting subgroup results from the total. When group statistics are enabled, the current conservative policy suppresses every result for an item if any displayed stratum falls below the minimum: ```{r suppression} group_feedback <- prepare_feedback(analysis, policy = list( group_statistics = TRUE, minimum_display_cell_n = 5L, complementary_suppression = TRUE )) group_feedback$results[, c("stratum", "n_valid", "classification", "suppressed")] ``` This cell rule is not a general anonymity guarantee. Qualitative summaries need review and a source reference. A draft is not yet an approved, immutable participant feedback release. ## Check reproducibility and provenance ```{r reproducibility} identical(analysis, analyse_round(first)) analysis$provenance[c("software_version", "quantile_type", "snapshot_hash", "rules_hash", "result_hash")] ``` The same frozen inputs and rules produce the same result in the same software environment. Hashes detect changes; they do not replace access control or methodological review. Sharing results also requires the relevant data approval and export profile. ## Reproduce a private research export The database export uses the same `analyse_round()` logic. Its manifest includes `snapshot.json`, analysis tables, a dictionary, frozen instrument texts, and the HTML report. After extracting an authorized export, `reproduce_export()` checks its file hashes and recomputes the analysis without the application or Quarto: ```{r exported-study, eval=FALSE} # A previously authorized local export, not an example downloaded from a public URL. reproduced <- reproduce_export("/path/to/extracted-export") reproduced$provenance$result_hash ``` Report services require an authorized database connection and are separate from the executable offline example above. Reports distinguish frozen research data from editorial records observed at export time. Missing author information stays explicitly undocumented. Qualitative originals and individual free-text answers are excluded from the numeric profile. A research export is not a public-release approval. Stored protocol and instrument wording retain their authored language; the documentation does not automatically translate scientific content. ```{r session} sessionInfo() ```