--- title: "Audit, local runtime, privacy and troubleshooting" output: rmarkdown::html_vignette: mathjax: null vignette: > %\VignetteIndexEntry{Audit, local runtime, privacy and troubleshooting} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} resource_files: - figures/logo.svg - figures/logo-dark.svg --- ```{r setup, include=FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") example_root <- tempfile("cttir-audit-") dir.create(example_root) old_options <- options(cttiR.catalog_dir = file.path(example_root, "catalog"), cttiR.runtime_dir = file.path(example_root, "runtime")) ``` ## doctor(), audit() and update() do different things | Function | Purpose | Side effects | |---|---|---| | `doctor()` | brief read-only diagnostics | none | | `audit()` | full check registry, reports, optional allowlisted repairs | reports only when `output` is given; repairs only with `repair = TRUE` | | `update()` | refresh the knowledge/resource catalogs | new immutable snapshots and one pointer change; never installs packages | ```{r audit} p <- cttiR::project("Audit example", "methods", "Describe a tabular comparison", path = example_root) report <- cttiR::audit(p$path) report head(as.data.frame(report)[, c("id", "status", "message")], 10) ``` The default audit reads metadata only. It never opens referenced datasets, runs study pipelines, installs dependencies, starts services or contacts a network. The `integration` scope opts into temporary synthetic projects and an isolated catalog fixture; `live = TRUE` additionally allows a bounded probe of an already running owned local model. `RES-008` compares every required and optional dependency pin with installed version metadata, including CRAN interoperability packages. In renv mode it reads the project library; a matching global installation does not substitute for a missing project package. Unresolved pins warn for a scaffold and fail an `analysis_ready` claim. Matching version numbers alone do not establish matching source hashes or scientific suitability. Nothing is installed by this check. Generated interoperability helpers also enforce pins at runtime. Run them from the project root so they can read `cttir-lock.json`, or pass a complete reviewed `pins` list explicitly. They refuse a missing lock, missing package pins and installed-version mismatches, including when the class namespace is already loaded. The lock is read as JSON; no project code is sourced, and nothing is installed automatically. `jsonlite` reads the lock and its version is then checked against that lock before object operations proceed. Repairs are an explicit allowlist: restore a missing managed file whose baseline is known, roll back an interrupted write whose writer has stopped, and repoint a corrupt catalog pointer to the last verified snapshot. Edited files are never overwritten. ```{r repair} unlink(file.path(p$path, "code", "validate_project.R")) cttiR::audit(p$path, scope = "project")$overall_status repaired <- cttiR::audit(p$path, scope = "project", repair = TRUE) repaired$overall_status file.exists(file.path(p$path, "code", "validate_project.R")) ``` `audit(strict = TRUE)` raises `cttir_audit_failed` after writing any requested reports when a required check fails or stays unverified; the condition carries the full report for CI use. ## Readiness levels Project creation always reports `scaffold_ready`. Higher levels come from local evidence: matching installed or renv-locked dependency versions (`environment_ready`), a passing synthetic demonstration produced by the unmodified stage code (`demo_verified`), a passing study-data run receipt (`data_ready`) and, finally, recorded mappings, reviewed settings, analysis approval and approved stage adapters (`analysis_ready`). None of these is a scientific validation of results. ## Local runtime and planner The opt-in live audit (`INT-003`) checks both structured output and a fixed synthetic linear-regression planning task. It requires an accepted local proposal, the expected grounded capability and current bundled workflow approval evidence. It can diagnose an unqualified model without promoting it. These probes use no project goal or study data and are not an independent usefulness benchmark. The audit budget accommodates the initial probe and at most two planner requests under the supported timeout settings. `setup()` prepares an owned, cloud-disabled Ollama runtime on Linux x86_64 from a pinned publisher archive and verifies a local model with a bounded structured probe. A dry run is read-only: ```{r setup-plan} plan <- cttiR::setup(dry_run = TRUE) plan$state plan$model$name ``` The default project planner is deterministic. The recorded 7B CPU results are currently development evidence only. Fresh independent qualification on the shipped artifact remains pending, so no manifest model authorizes automatic planning. Automatic model selection retains its tested resource policy, but blocks acquisition when no independently qualified model is available. With `options(cttiR.planner = "local_llm")` and a verified owned runtime, a model plans only if the manifest marks it qualified at its exact digest and the tested prompt, grounding policy, registry, schema, runtime binary and inference settings still match; otherwise the deterministic planner is used and the fallback reason `model_not_qualified` is recorded. Using an unqualified model needs the separate acknowledgement `options(cttiR.planner_allow_unqualified = TRUE)`, and its qualification label is then stored with every decision it made. Goals that read like instructions to the model are never sent (`goal_injection_suspected`). Deterministic planning and `ask()` block hard instruction markers; ambiguous phrases such as "safety mode" retain keyword matching with a review note. Screened goals cannot add keyword-derived workflow candidates. Explicit options always win over suggestions, and replay or `sync()` never re-plan. The prompt includes registered package names and aliases. A validated model proposal retains only capabilities with registered keyword evidence in the goal; omissions are recorded in bounded notes and the project decision evidence. This can omit legitimate requests expressed without those keywords. Filtering is not workflow approval, and fixed inference settings do not guarantee identical suggestions. Review proposed study decisions before using them for analysis. Requests matching more capabilities than the proposal schema permits return no capability selection and ask for a narrower request. Candidate capability IDs carry an explicit unapproved-status note; they do not become reviewed adapters. Planner and assistance qualification also bind package function bodies, defaults, static rules, R/platform and direct/transitive Imports and Depends versions. Mutable caches are excluded. The implementation fingerprint is deliberately conservative: even an unrelated package-code change reopens qualification. These identity checks do not prove scientific validity or the integrity of a third-party package rebuilt with different contents under the same version. Long inputs are bounded again after JSON escaping, preserving valid input JSON. Planner 4 keeps the longest prefix that fits this character budget. Every reply, including a repair, must report a valid prompt token count with room for the reserved output tokens in the configured context. Missing or excessive counts cause `context_overflow` fallback without another request. This checks the runtime's reported count; it is not an independent tokenizer or proof that the runtime preserved every input token. Earlier planner evaluations remain historical development evidence and do not qualify the new policy. Planner benchmark reports separate post-grounding scores from the capability claims present before keyword filtering, and retain the removed IDs in exported rows. With diagnostic `keep_raw = TRUE`, parseable raw capability arrays are also scored against the case's gold capabilities, including rejected attempts. Missing, malformed or truncated raw replies remain unscored; the report gives the number of scored attempts so absent diagnostics cannot masquerade as a zero unsupported-claim rate. The first-attempt rejection rate excludes cases where no model request was made. These additional diagnostics do not replace or relax any pre-registered threshold. Historical comparison results and representative failures are bundled in `benchmarks/planner-review-history.json`; the old compact reports cannot recover all pre-grounding claims. Accepted local planning stores the model identity, digest, qualification, prompt version and request options in the specification and project lock. Replay uses these recorded values; older specifications with no recorded settings keep them unknown. `cttiR.planner_threads` defaults to the smaller of 16 and the measured physical core count. `cttiR.planner_processor` defaults to `"cpu"`; `"auto"` permits unqualified GPU use. `cttiR.planner_timeout` defaults to 120 seconds (allowed range 5 to 600). Changing options does not confer qualification. Historical CPU results used a Xeon w9-3475X with 16 threads; other CPUs and runtime backends have not inherited that measured result. ## Data flow and privacy * Goals, configuration files, documentation and model output are treated as untrusted data; nothing is evaluated. * Study data are never read during creation, synchronization, audit or catalog updates. `check_analysis_data()` and `code/run_workflow.R` read data only when you supply or bind them explicitly. * Machine-specific paths belong in the ignored `.cttir/local.yml`. * The planner (when enabled) and `ask()` never send text off the machine; the runtime listens on loopback only and refuses cloud-backed models. * Exported Shiny drafts can contain your text; review them before sharing. * Standard workflowr reports use upstream MathJax scripts and a workflowr badge. Viewing their HTML can contact external servers even after local computation. Remote JavaScript runs in the report's browser context. Before using confidential results, review and provision the report assets for that environment; offline viewing is not established by a successful local render. ## Troubleshooting | Symptom | Next step | |---|---| | `cttir_transaction_conflict` with an active writer | wait for the other operation; if it crashed, run `cttiR::audit(path, repair = TRUE)` | | `cttir_catalog_corrupt` | `cttiR::audit(scope = "knowledge", repair = TRUE)` or `cttiR::rollback_knowledge(, dry_run = FALSE)` | | `workflow_approval_pending` blocker | the pinned catalog lacks approvals for a stage; review with `cttiR::packages(path)` | | `environment_pending` blocker | install the pinned versions or enable `workflow$environment = "renv"` with `prepare_environment = TRUE` | | `code/run_workflow.R` stops with a list | record the listed mappings/settings/approval with `cttiR::sync()` and bind the dataset in `.cttir/local.yml` | | `setup()` returns `blocked` | read `$blockers`; offline mode never downloads, and only Linux x86_64 is supported | ```{r cleanup, include=FALSE} options(old_options) unlink(example_root, recursive = TRUE) ``` Setup previews (`setup(dry_run = TRUE)`) report offline-known acquisition and planning blockers separately, published full download sizes, available disk space and recorded disk admission budgets. Sizes are not estimates of remaining incremental downloads, and an absent model profile has unknown size/budget. A verified owned process is reported with PID and host; file presence alone does not verify a runtime. Project summaries report the planning mode used and direct users to this preview without probing or starting an optional local runtime. Explicit model preparation remains available for diagnostics even when the model is unqualified; it does not grant planning qualification. ## Installing an exact model release `setup_release()` takes a locally reviewed JSON manifest and its SHA-256 from a separately trusted release record. This development package does not yet bundle a qualified release manifest. The schema binds package, catalog, policy, prompt, runtime, base model, adapter, GGUF and template identities. A manifest cannot grant qualification. Linux x86_64 is the only supported acquisition platform. ```{r exact-release, eval=FALSE} # Obtain these files and the manifest checksum from a reviewed release record. preview <- cttiR::setup_release( manifest = "release.json", manifest_sha256 = trusted_manifest_sha256, model_file = "model.gguf", offline = TRUE ) preview$storage preview$download_estimate preview$blockers # A separate explicit action permits the previewed work. installed <- cttiR::setup_release( manifest = "release.json", manifest_sha256 = trusted_manifest_sha256, model_file = "model.gguf", offline = TRUE, dry_run = FALSE, consent = TRUE ) ``` Preview makes no HTTP requests or writes. It reports conservative disk admission requirements, including staging space; these are estimates, not reservations. Offline import still needs the pinned runtime to be installed already. Omitting `model_file` permits a verified cache or, with downloads enabled, the immutable HTTPS model URL in the manifest. Model and runtime transfers require command-line `curl` and resume retained partial downloads. Corrupt or conflicting files remain for inspection. An occupied unmanaged daemon is not modified. Select an unused loopback port through `cttiR.ollama_endpoint` when an isolated runtime is needed. The Runtime view exposes the same release inputs and consent control. The installer checks the local model digest and template and performs a bounded structured-output probe. `model_ready` is separate from assistance qualification, project environment readiness and analysis approval. Keep previous release manifests and artifacts for rollback by explicitly selecting their original manifest. The atomic runtime-state record identifies the activated release. ## Storage and process lifetime ```{r managed-storage, eval=FALSE} cttiR::runtime_storage() # Choose exact download basenames from the inventory; nothing is auto-selected. cleanup <- cttiR::runtime_cleanup(files = selected_download_basenames) cttiR::runtime_cleanup(files = selected_download_basenames, expected = cleanup$expected, dry_run = FALSE, consent = TRUE) shutdown <- cttiR::runtime_stop() cttiR::runtime_stop(expected = shutdown$expected, dry_run = FALSE, consent = TRUE) ``` Storage inventory covers model downloads, runtime archives, blobs and retained manifests, excluding runtime libraries and logs from its byte total. Cleanup protects all retained release pins, model blobs and unrelated files. It rechecks selected hashes under the setup lock and records removals. Shutdown rechecks process ownership and creation time, sends a graceful signal only to that process and preserves all files. A `stopping` result reports that the process remained alive after five seconds; it does not claim that all resources were released. No package hook starts, stops or downloads a runtime or model.