Audit, local runtime, privacy and troubleshooting

doctor(), audit() and update() do different things

p <- cttiR::project("Audit example", "methods", "Describe a tabular comparison", path = example_root)
report <- cttiR::audit(p$path)
report
#> Audit: warning (scaffold)
#> 
#> not_applicable     not_tested           pass        warning 
#>              1              3             23              4 
#> INS-002 not_tested: No verified owned runtime; offline project creation remains available.
#> INS-003 not_tested: No owned runtime state is recorded; model locality is unknown.
#> KB-003 not_tested: No source registry is configured; only the bundled or active catalog coverage is known.
#> KB-004 warning: 64 cataloged packages; 4666 of 5486 exports have static signatures and 5473 have reference topics; 16 installed versions differ from the cataloged revision. Workflow approvals remain separate.
#> RES-002 warning: Resource observations are dated; remote freshness was not checked.
#> STD-004 warning: Analysis configuration incomplete: 6 missing fields, 0 capability gaps.
#> RES-008 warning: Project dependency pins are unresolved: DescrTab2 (not_installed); colorspace (version_mismatch); grDevices (version_mismatch); knitr (version_mismatch); nlme (version_mismatch); rmarkdown (version_mismatch); stats (version_mismatch); survival (version_mismatch); tools (version_mismatch); utils (version_mismatch). Optional packages are included; no packages were installed.
head(as.data.frame(report)[, c("id", "status", "message")], 10)
#>         id     status
#> 1  INS-001       pass
#> 2  INS-002 not_tested
#> 3  INS-003 not_tested
#> 4  INS-004       pass
#> 5   KB-001       pass
#> 6   KB-002       pass
#> 7   KB-003 not_tested
#> 8   KB-004    warning
#> 9   KB-005       pass
#> 10 RES-001       pass
#>                                                                                                                                                                                                                                                 message
#> 1                                                                                                                                                          cttiR 0.1.1.9000: contract exports, bundled resources and 11 declared Imports are available.
#> 2                                                                                                                                                                                No verified owned runtime; offline project creation remains available.
#> 3                                                                                                                                                                                        No owned runtime state is recorded; model locality is unknown.
#> 4  Catalog storage: Absent; it will be created under a writable parent on first explicit write. Runtime storage: Absent; it will be created under a writable parent on first explicit write. Only metadata was inspected; no write probe was performed.
#> 5                                                                                                                                                                       Bundled resources, catalog identities, schemas and selected snapshots verified.
#> 6                                                                                                                                                                         Resource databases, catalog references and retained manifests are consistent.
#> 7                                                                                                                                                               No source registry is configured; only the bundled or active catalog coverage is known.
#> 8                                                      64 cataloged packages; 4666 of 5486 exports have static signatures and 5473 have reference topics; 16 installed versions differ from the cataloged revision. Workflow approvals remain separate.
#> 9                                                                                                                                Documentation corpora are intact; 0 exports removed in other retained revisions stay absent from the current revision.
#> 10                                                                                                                                                                      SQLite integrity, foreign keys and normalized SQLite/JSON row content verified.

The default audit reads metadata only. It never opens referenced datasets, runs study pipelines, installs dependencies, starts services or contacts a network. The integration scope opts into temporary synthetic projects and an isolated catalog fixture; live = TRUE additionally allows a bounded probe of an already running owned local model.

RES-008 compares every required and optional dependency pin with installed version metadata, including CRAN interoperability packages. In renv mode it reads the project library; a matching global installation does not substitute for a missing project package. Unresolved pins warn for a scaffold and fail an analysis_ready claim. Matching version numbers alone do not establish matching source hashes or scientific suitability. Nothing is installed by this check.

Generated interoperability helpers also enforce pins at runtime. Run them from the project root so they can read cttir-lock.json, or pass a complete reviewed pins list explicitly. They refuse a missing lock, missing package pins and installed-version mismatches, including when the class namespace is already loaded. The lock is read as JSON; no project code is sourced, and nothing is installed automatically. jsonlite reads the lock and its version is then checked against that lock before object operations proceed.

Repairs are an explicit allowlist: restore a missing managed file whose baseline is known, roll back an interrupted write whose writer has stopped, and repoint a corrupt catalog pointer to the last verified snapshot. Edited files are never overwritten.

unlink(file.path(p$path, "code", "validate_project.R"))
cttiR::audit(p$path, scope = "project")$overall_status
#> [1] "fail"
repaired <- cttiR::audit(p$path, scope = "project", repair = TRUE)
repaired$overall_status
#> [1] "warning"
file.exists(file.path(p$path, "code", "validate_project.R"))
#> [1] TRUE

audit(strict = TRUE) raises cttir_audit_failed after writing any requested reports when a required check fails or stays unverified; the condition carries the full report for CI use.

Readiness levels

Project creation always reports scaffold_ready. Higher levels come from local evidence: matching installed or renv-locked dependency versions (environment_ready), a passing synthetic demonstration produced by the unmodified stage code (demo_verified), a passing study-data run receipt (data_ready) and, finally, recorded mappings, reviewed settings, analysis approval and approved stage adapters (analysis_ready). None of these is a scientific validation of results.

Local runtime and planner

The opt-in live audit (INT-003) checks both structured output and a fixed synthetic linear-regression planning task. It requires an accepted local proposal, the expected grounded capability and current bundled workflow approval evidence. It can diagnose an unqualified model without promoting it. These probes use no project goal or study data and are not an independent usefulness benchmark. The audit budget accommodates the initial probe and at most two planner requests under the supported timeout settings.

setup() prepares an owned, cloud-disabled Ollama runtime on Linux x86_64 from a pinned publisher archive and verifies a local model with a bounded structured probe. A dry run is read-only:

plan <- cttiR::setup(dry_run = TRUE)
plan$state
#> [1] "planned"
plan$model$name
#> [1] "auto"

The default project planner is deterministic. The recorded 7B CPU results are currently development evidence only. Fresh independent qualification on the shipped artifact remains pending, so no manifest model authorizes automatic planning. Automatic model selection retains its tested resource policy, but blocks acquisition when no independently qualified model is available. With options(cttiR.planner = "local_llm") and a verified owned runtime, a model plans only if the manifest marks it qualified at its exact digest and the tested prompt, grounding policy, registry, schema, runtime binary and inference settings still match; otherwise the deterministic planner is used and the fallback reason model_not_qualified is recorded. Using an unqualified model needs the separate acknowledgement options(cttiR.planner_allow_unqualified = TRUE), and its qualification label is then stored with every decision it made. Goals that read like instructions to the model are never sent (goal_injection_suspected). Deterministic planning and ask() block hard instruction markers; ambiguous phrases such as “safety mode” retain keyword matching with a review note. Screened goals cannot add keyword-derived workflow candidates. Explicit options always win over suggestions, and replay or sync() never re-plan.

The prompt includes registered package names and aliases. A validated model proposal retains only capabilities with registered keyword evidence in the goal; omissions are recorded in bounded notes and the project decision evidence. This can omit legitimate requests expressed without those keywords. Filtering is not workflow approval, and fixed inference settings do not guarantee identical suggestions. Review proposed study decisions before using them for analysis. Requests matching more capabilities than the proposal schema permits return no capability selection and ask for a narrower request. Candidate capability IDs carry an explicit unapproved-status note; they do not become reviewed adapters.

Planner and assistance qualification also bind package function bodies, defaults, static rules, R/platform and direct/transitive Imports and Depends versions. Mutable caches are excluded. The implementation fingerprint is deliberately conservative: even an unrelated package-code change reopens qualification. These identity checks do not prove scientific validity or the integrity of a third-party package rebuilt with different contents under the same version. Long inputs are bounded again after JSON escaping, preserving valid input JSON. Planner 4 keeps the longest prefix that fits this character budget. Every reply, including a repair, must report a valid prompt token count with room for the reserved output tokens in the configured context. Missing or excessive counts cause context_overflow fallback without another request. This checks the runtime’s reported count; it is not an independent tokenizer or proof that the runtime preserved every input token. Earlier planner evaluations remain historical development evidence and do not qualify the new policy.

Planner benchmark reports separate post-grounding scores from the capability claims present before keyword filtering, and retain the removed IDs in exported rows. With diagnostic keep_raw = TRUE, parseable raw capability arrays are also scored against the case’s gold capabilities, including rejected attempts. Missing, malformed or truncated raw replies remain unscored; the report gives the number of scored attempts so absent diagnostics cannot masquerade as a zero unsupported-claim rate. The first-attempt rejection rate excludes cases where no model request was made. These additional diagnostics do not replace or relax any pre-registered threshold. Historical comparison results and representative failures are bundled in benchmarks/planner-review-history.json; the old compact reports cannot recover all pre-grounding claims.

Accepted local planning stores the model identity, digest, qualification, prompt version and request options in the specification and project lock. Replay uses these recorded values; older specifications with no recorded settings keep them unknown. cttiR.planner_threads defaults to the smaller of 16 and the measured physical core count. cttiR.planner_processor defaults to "cpu"; "auto" permits unqualified GPU use. cttiR.planner_timeout defaults to 120 seconds (allowed range 5 to 600). Changing options does not confer qualification. Historical CPU results used a Xeon w9-3475X with 16 threads; other CPUs and runtime backends have not inherited that measured result.

Data flow and privacy

• Goals, configuration files, documentation and model output are treated as untrusted data; nothing is evaluated.
• Study data are never read during creation, synchronization, audit or catalog updates. check_analysis_data() and code/run_workflow.R read data only when you supply or bind them explicitly.
• Machine-specific paths belong in the ignored .cttir/local.yml.
• The planner (when enabled) and ask() never send text off the machine; the runtime listens on loopback only and refuses cloud-backed models.
• Exported Shiny drafts can contain your text; review them before sharing.
• Standard workflowr reports use upstream MathJax scripts and a workflowr badge. Viewing their HTML can contact external servers even after local computation. Remote JavaScript runs in the report’s browser context. Before using confidential results, review and provision the report assets for that environment; offline viewing is not established by a successful local render.

Troubleshooting

Setup previews (setup(dry_run = TRUE)) report offline-known acquisition and planning blockers separately, published full download sizes, available disk space and recorded disk admission budgets. Sizes are not estimates of remaining incremental downloads, and an absent model profile has unknown size/budget. A verified owned process is reported with PID and host; file presence alone does not verify a runtime. Project summaries report the planning mode used and direct users to this preview without probing or starting an optional local runtime. Explicit model preparation remains available for diagnostics even when the model is unqualified; it does not grant planning qualification.

Installing an exact model release

setup_release() takes a locally reviewed JSON manifest and its SHA-256 from a separately trusted release record. This development package does not yet bundle a qualified release manifest. The schema binds package, catalog, policy, prompt, runtime, base model, adapter, GGUF and template identities. A manifest cannot grant qualification. Linux x86_64 is the only supported acquisition platform.

# Obtain these files and the manifest checksum from a reviewed release record.
preview <- cttiR::setup_release(
  manifest = "release.json", manifest_sha256 = trusted_manifest_sha256,
  model_file = "model.gguf", offline = TRUE
)
preview$storage
preview$download_estimate
preview$blockers

# A separate explicit action permits the previewed work.
installed <- cttiR::setup_release(
  manifest = "release.json", manifest_sha256 = trusted_manifest_sha256,
  model_file = "model.gguf", offline = TRUE,
  dry_run = FALSE, consent = TRUE
)

Preview makes no HTTP requests or writes. It reports conservative disk admission requirements, including staging space; these are estimates, not reservations. Offline import still needs the pinned runtime to be installed already. Omitting model_file permits a verified cache or, with downloads enabled, the immutable HTTPS model URL in the manifest. Model and runtime transfers require command-line curl and resume retained partial downloads. Corrupt or conflicting files remain for inspection. An occupied unmanaged daemon is not modified. Select an unused loopback port through cttiR.ollama_endpoint when an isolated runtime is needed. The Runtime view exposes the same release inputs and consent control.

The installer checks the local model digest and template and performs a bounded structured-output probe. model_ready is separate from assistance qualification, project environment readiness and analysis approval. Keep previous release manifests and artifacts for rollback by explicitly selecting their original manifest. The atomic runtime-state record identifies the activated release.

Storage and process lifetime

cttiR::runtime_storage()

# Choose exact download basenames from the inventory; nothing is auto-selected.
cleanup <- cttiR::runtime_cleanup(files = selected_download_basenames)
cttiR::runtime_cleanup(files = selected_download_basenames,
  expected = cleanup$expected, dry_run = FALSE, consent = TRUE)

shutdown <- cttiR::runtime_stop()
cttiR::runtime_stop(expected = shutdown$expected,
  dry_run = FALSE, consent = TRUE)

Storage inventory covers model downloads, runtime archives, blobs and retained manifests, excluding runtime libraries and logs from its byte total. Cleanup protects all retained release pins, model blobs and unrelated files. It rechecks selected hashes under the setup lock and records removals. Shutdown rechecks process ownership and creation time, sends a graceful signal only to that process and preserves all files. A stopping result reports that the process remained alive after five seconds; it does not claim that all resources were released. No package hook starts, stops or downloads a runtime or model.