---
title: "Reproducible analytical summaries"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Reproducible analytical summaries}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
resource_files:
  - figures/logo.svg
  - figures/logo-dark.svg
---

<!-- cttir-vignette-logo: start -->
<style>
.cttir-vignette-logo { float: right; width: 120px; max-width: 30%; margin: 0 0 1rem 1.5rem; }
.cttir-vignette-logo img { width: 100%; height: auto; border: 0; background: transparent; box-shadow: none; }
.cttir-vignette-logo .cttir-vignette-logo-light { display: block; }
.cttir-vignette-logo .cttir-vignette-logo-dark { display: none; }
@media (prefers-color-scheme: dark) {
  .cttir-vignette-logo .cttir-vignette-logo-light { display: none; }
  .cttir-vignette-logo .cttir-vignette-logo-dark { display: block; }
}
[data-bs-theme="light"] .cttir-vignette-logo .cttir-vignette-logo-light { display: block; }
[data-bs-theme="light"] .cttir-vignette-logo .cttir-vignette-logo-dark { display: none; }
[data-bs-theme="dark"] .cttir-vignette-logo .cttir-vignette-logo-light { display: none; }
[data-bs-theme="dark"] .cttir-vignette-logo .cttir-vignette-logo-dark { display: block; }
</style>
<div class="cttir-vignette-logo">
  <img class="cttir-vignette-logo-light" src="figures/logo.svg" alt="hyperannotatR logo" width="120" height="139" />
  <img class="cttir-vignette-logo-dark" src="figures/logo-dark.svg" alt="hyperannotatR logo" width="120" height="139" />
</div>
<!-- cttir-vignette-logo: end -->


```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(hyperannotatR)
```

This example follows persisted annotations through masks, raw pixel records,
ROI spectra, image summaries, and agreement tables. It uses the small synthetic
reference bundle in the source repository at
`tests/testthat/fixtures/analytical/`. Its NumPy generator computes the numerical
reference independently of hyperannotatR. The workflow never reads the expected JSON
to calculate its results; the regression test uses that JSON only for comparison.

The example functions are installed at `examples/reproducible-analysis.R`.
They are reusable source code, not exported package APIs. The fixture bundle is
kept in the repository tests rather than duplicated in the installed package.
To reproduce this vignette, use a source checkout with the reviewed package code
installed, or use `pkgload::load_all()` from that checkout first. Run the following
code from the repository root (the vignette build uses the equivalent relative
path from its own directory).

```{r load-example}
source(system.file("examples", "reproducible-analysis.R", package = "hyperannotatR"))
```

```{r root-path, eval=FALSE}
fixture_dir <- normalizePath("tests/testthat/fixtures/analytical")
```

```{r vignette-path, include=FALSE}
fixture_dir <- normalizePath("../tests/testthat/fixtures/analytical", mustWork = TRUE)
```

Create the input once, then reopen it for every analysis. The synthetic entry ID
is `analytical-image-001`; the two imported ROI IDs are preserved. The settings
record `at_annotation_revision(project)`, a SHA-256 digest of the annotation
content (`"sha256:..."`): rerunning on the same persisted annotations records the
same revision, and any change to the annotations records a different one. For an
adapted study, retain its actual entry IDs and analyse the persisted project
rather than reimporting annotations into a fresh project for every run.

```{r persist}
work <- tempfile("analytical-example-")
dir.create(work)
project_path <- file.path(work, "annotations.rds")
prepare_analytical_example(fixture_dir, project_path)
first <- run_analytical_example(project_path, fixture_dir, file.path(work, "first"))
second <- run_analytical_example(project_path, fixture_dir, file.path(work, "second"))
stopifnot(identical(first, second))
files <- list.files(file.path(work, "first"))
stopifnot(identical(
  unname(tools::md5sum(file.path(work, "first", files))),
  unname(tools::md5sum(file.path(work, "second", files)))
))
```

Both functions refuse to overwrite existing outputs. The analysis does not save
or mutate its input project. The ten CSV tables and `settings.json` are
deterministic for the same persisted input and software environment. Source
paths, signatures, input hashes and software versions deliberately remain in
the settings; relocating files or changing software can change provenance.
No run timestamp or runtime cache/read-generation identity enters these outputs.
The saved project's own historical timestamps remain in that input file and
therefore in its checksum. Optional execution timestamps belong in a separate
run log, not in a result table or its deterministic configuration.

## Counts and estimands

Rectangles A and B each select nine pixels on the 4 by 5 grid. They share two
pixels. The bitfield mask assigns A=1, B=2, overlap=3 and background=0.
Membership is determined at polygon pixel centres at level zero with
`touches=FALSE`, `engine="stars"` and `overlap="bitor"`.

```{r counts}
first$tables$counts
first$tables$mask
first$tables$mask_codebook
```

| Table | Row or count unit | Overlap treatment |
|---|---|---|
| `pixel_memberships` | One ROI-pixel-band record: 54 rows | Shared cells appear once per ROI |
| `unique_pixels` | One image-pixel-band record: 48 rows | Deduplicate `(x,y,band)` within this one image |
| `roi_statistics` | One ROI-band-statistic: 42 rows | Each ROI is summarized separately |
| `spectra` | One ROI-band finite mean: 6 rows | Each ROI has its own spectrum |
| `image_statistics` | One image-band summary: 3 rows | Each selected cell contributes once |
| `counts` | One image: 16 unique cells, 18 ROI memberships, 2 ROIs | Two overlap cells counted explicitly |

The `x` and `y` columns are one-based column/row indices, whereas ROI geometry
uses zero-based image coordinates. Band identity and spectral coordinates are
recorded in settings. `unit="nm"` describes the spectral coordinate, not the
intensity values. No intensity unit or physical calibration is inferred: this
cube declares pixel calibration `(1,1)` in `px`. Settings explicitly mark its
physical calibration unknown, so no physical-area result is produced.

The image summary below estimates finite-only statistics across the **unique
selected pixels within this image**. Its band-one sum is 1937, with 15 finite
contributors out of 16 selected cells, so its mean is `1937/15`. Pooling repeated
ROI records would use 17 finite contributors and sum 2193. Averaging ROI means
would weight the two ROIs equally. Those are different estimands and neither is
silently substituted for the unique-pixel image summary.

```{r image-summary}
first$tables$image_statistics
stopifnot(first$tables$image_statistics$sum[1] == 1937)
stopifnot(first$tables$image_statistics$mean[1] == 1937 / 15)
```

## Nonfinite values and spectra

`at_extract(..., stat="all", nonfinite="omit")` and
`at_extract_spectrum(..., nonfinite="omit")` choose finite-only statistics
explicitly. `n_px` and the statistic `n` still mean selected support. `n_valid`
counts finite contributors; `n_invalid` counts selected NA, NaN and infinite
values. Sample SD uses denominator `n_valid - 1`; it is undefined for a singleton.
All-invalid summaries, including sum, are NA. Empty selections have zero
contributors; the example retains an image-band row with NA statistics.

```{r spectra}
first$tables$spectra[c("label", "band", "wavelength", "unit", "value",
                      "n_px", "n_valid", "n_invalid")]
```

The raw pixel API retains nonfinite samples. There are four unique invalid
pixel-band values but five invalid ROI-pixel-band records, because the positive
infinity in the overlap belongs to both ROIs. The raw tables include
`sample_status` (`finite`, `NA`, `NaN`, `+Inf`, `-Inf`) to preserve distinctions
that CSV missing-value serialization could collapse. The four unselected
background cells are a spatial exclusion, separate from invalid selected
samples. `n_excluded_rois` counts ROIs with no selected cells; it is zero here.
The example applies no ROI or label filters.

This workflow produces numerical spectra and tables. It generates no image
preview. Display normalization for the mixed NaN/Inf RGB fixture remains outside
this qualification; no RGB or native ImageMagick success is claimed.

## Agreement with explicit alignment

The separate 3 by 4 categorical and bitfield NPY cases exercise agreement; these
are not prediction masks for the 4 by 5 cube. The independent fixture generator
declares their common row/column grids. Since arrays alone cannot prove source
registration, this example uses `alignment="assert"` and records that basis.
For study masks with durable common source/grid metadata, use
`alignment="check"` to verify it. Equal dimensions alone are not evidence of
registration, and assertion never resamples mismatched grids.

`at_read_npy()` receives explicit encoding, background and label codebooks.
Ordinary comparisons use `code_alignment="strict"`. The deliberately swapped
categorical codes use `code_alignment="by-label"`, preserving original codebooks
in settings. An absent class remains a table row with undefined Dice/IoU, and
macro means exclude such undefined rows.

```{r agreement}
first$tables$agreement
first$tables$agreement_overall
```

Categorical accuracy is 3/4, Cohen's kappa is 5/8, and A/B Dice and IoU are 3/4
and 3/5. Explicit remapping reproduces them. Bitfield A has Dice/IoU 1; B has
Dice 3/5 and IoU 3/7. Its overall accuracy of 2/3 is exact membership-set
agreement, including background; categorical kappa is undefined for bitfields.
Each overall agreement row counts all 12 cells, including background. Its
nonfinite policy is `error`.

## Traceability and scope

`settings.json` accompanies the tables with the persisted entry/ROI IDs, the
annotation revision digest, durable source descriptor and reader options, MD5 hashes of the
project and all eight data inputs (including the ENVI payload), calibration,
band coordinates/units, extraction and rasterization settings, mask codebook,
both sides' validated agreement metadata/codebooks, estimands, and count
definitions. It records the actual R and dependency versions and sf external
library versions. The ENVI backend is implemented by hyperannotatR, whose version is
recorded; separately unavailable plugin-version metadata is explicitly
`"unknown"`. MD5 checks exact-byte integrity, not authenticity. The source tree
or build artifact used for a study should also be archived with these outputs.

```{r settings}
first$settings$sampling_units
first$settings$extraction
first$settings$versions
```

This is a descriptive example for one image and two overlapping ROIs. Pixels,
bands and ROI means are not independent image replicates. Inferential uncertainty
requires the study's independent sampling unit (for example a donor, specimen or
image, as justified by its design), with nesting and overlap handled explicitly.
There is no resampling here and the recorded seed is null. Any added resampling
must declare its seed and independent unit. No model training or clinical claim
is part of this example.

The implementation retains existing extraction memory bounds: one bounding-box
tile per ROI, a `max_px=1e6` raw extraction guard, and in-memory deduplication and
tables. It is a small synthetic workflow, not a streaming analysis engine or
whole-slide resource qualification.

```{r cleanup, include=FALSE}
unlink(work, recursive = TRUE)
```