Getting started with sorenR

Self-Organizing Representations with Evolving Neighborhoods is an experimental pure-R prototype graph representation. Its scientific novelty and benefit over existing methods are not established. This example uses only synthetic data.

Fit a representation

Rows are observations and columns are numeric features. Preprocessing is learned from the training matrix and stays frozen for later transforms and updates.

library(sorenR)
t <- seq(0, 4 * pi, length.out = 80)
x <- cbind(time = t, sine = sin(t), cosine = cos(t))
control <- soren_control(max_prototypes = 16L, initial_prototypes = 8L,
                         memory_size = 40L, seed = 42L)
model <- soren_fit(x, n_components = 2L, control = control)
model
#> Experimental SOREN model: 8 prototypes, 2 dimensions, update 0

Transform new observations

The frozen transform does not retrain the model or consume random numbers. Feature names and their order must match training.

t_next <- seq(4 * pi, 4.5 * pi, length.out = 12)
x_next <- cbind(time = t_next, sine = sin(t_next), cosine = cos(t_next))
mapped <- soren_transform(model, x_next, diagnostics = TRUE)
head(mapped$coordinates)
#>           SOREN1      SOREN2
#> [1,] -0.25808456 -0.13826910
#> [2,] -0.23308638 -0.12565644
#> [3,] -0.20302512 -0.10997095
#> [4,] -0.16856645 -0.09146966
#> [5,] -0.13099199 -0.07067270
#> [6,] -0.09213571 -0.04833475
head(mapped$support)
#>    residual support_ratio outside_reference
#> 1 0.5298778     0.5151344             FALSE
#> 2 0.7247796     0.7046133             FALSE
#> 3 0.9166496     0.8911446             FALSE
#> 4 1.1042006     1.0734772              TRUE
#> 5 1.2863376     1.2505464              TRUE
#> 6 1.4620596     1.4213791              TRUE

Support ratios are uncalibrated coverage diagnostics, not confidence probabilities. The output lies inside the convex hull of prototype coordinates; far-away inputs are not reliable extrapolations.

Audit independent neighborhoods

Training stress and independent neighbor retrieval measure different properties. Supply query and reference matrices separately; the audit uses frozen preprocessing and the current map without fitting or retaining either bank. These deterministic synthetic observations illustrate the API, rather than a scientific validation.

t_query <- seq(0.15, 3.85 * pi, length.out = 9)
t_reference <- seq(0.07, 3.93 * pi, length.out = 31)
query <- cbind(time = t_query, sine = sin(t_query), cosine = cos(t_query))
reference <- cbind(time = t_reference, sine = sin(t_reference), cosine = cos(t_reference))
audit <- soren_neighborhood_audit(model, query, reference, k = 5L, chunk_size = 8L)
audit$summary
#> $mean_recall_at_k
#> [1] 0.5777778
#> 
#> $n_query
#> [1] 9
#> 
#> $n_reference
#> [1] 31
#> 
#> $k
#> [1] 5
#> 
#> $n_excluded
#> [1] 0
#> 
#> $input_tied_query_count
#> [1] 0
#> 
#> $embedding_tied_query_count
#> [1] 0
#> 
#> $input_tied_query_fraction
#> [1] 0
#> 
#> $embedding_tied_query_fraction
#> [1] 0
head(audit$per_query)
#>   query_index recall_at_k eligible_reference_count excluded_reference
#> 1           1         0.6                       31                  0
#> 2           2         0.6                       31                  0
#> 3           3         0.4                       31                  0
#> 4           4         0.4                       31                  0
#> 5           5         0.6                       31                  0
#> 6           6         0.6                       31                  0
#>   input_boundary_size embedding_boundary_size input_boundary_weight
#> 1                   1                       1                     1
#> 2                   1                       1                     1
#> 3                   1                       1                     1
#> 4                   1                       1                     1
#> 5                   1                       1                     1
#> 6                   1                       1                     1
#>   embedding_boundary_weight
#> 1                         1
#> 2                         1
#> 3                         1
#> 4                         1
#> 5                         1
#> 6                         1

Recall is expected overlap under independent uniform choices within exact boundary ties, computed without random draws. Identical fully tied neighborhoods score k/N, not one; boundary sizes and inclusion weights make this visible. Duplicate values remain separate candidates. Self-matches are removed only when explicitly indexed with exclude_reference (zero means no exclusion), never from row names or values. For the same matrix in both arguments, use exclude_reference = seq_len(nrow(query)). The caller must establish independence from fitting or tuning. This diagnostic is uncalibrated and does not establish biological validity or methodological superiority.

The audit evaluates the existing map one row at a time so block size cannot change its reduction path; bulk transformation may differ at floating-point roundoff. Two reference passes bound distance workspace, with an additional k-selection buffer. Caller inputs, validation/coercion copies, and per-query output still use memory. Repeated mapping favors reproducible small CPU audits over throughput.

Update and inspect the result

An update returns a new model. The default fixed budget checks mean squared raw coordinate displacement on the old retained anchors. Infeasible proposals can be shortened or rolled back; inspect the status before interpreting adaptation.

updated <- soren_update(model, x_next)
checks <- soren_diagnostics(updated)
checks$last_update
#> $update_id
#> [1] 1
#> 
#> $status
#> [1] "accepted"
#> 
#> $budget
#> [1] 0.01
#> 
#> $displacement
#> [1] 0.005987233
#> 
#> $maximum_displacement
#> [1] 0.02596878
#> 
#> $constraint_residual
#> [1] -0.004012767
#> 
#> $proposed_displacement
#> [1] 0.005987233
#> 
#> $step
#> [1] 1
#> 
#> $growth_proposed
#> [1] FALSE
#> 
#> $growth_accepted
#> [1] FALSE
#> 
#> $support_score
#> [1] 1.503225
#> 
#> $calibrated
#> [1] FALSE
#> 
#> $iterations
#> [1] 30
#> 
#> $seconds
#> [1] 0.01

A finite-anchor mean budget does not constrain every observation or total movement across many updates. Adaptive mode uses an uncalibrated heuristic; it is not a statistical test of drift. Frozen and unrestricted modes are also available for controlled comparisons.

Retained coordinates and model files

head(soren_embedding(updated, type = "prototypes"))
#>                    [,1]       [,2]
#> prototype-1 -0.01646192  0.1662561
#> prototype-2 -0.30055773 -0.1021498
#> prototype-3  0.21549564 -0.2645561
#> prototype-4  0.30320288  0.2818219
#> prototype-5 -0.29721977 -0.1108774
#> prototype-6 -0.22609334 -0.2475636
checks$retained_slots
#> [1] 40

Replay memory contains bounded reservoir and coverage slots from training and accepted batches. soren_embedding() does not return every historical observation. Use soren_transform() for data stored separately.

Save with soren_save(updated, "model.rds") and reload with soren_load("model.rds"). Only load trusted files. Prototypes and retained replay may contain sensitive derived information and are not anonymized.

Dense finite matrices are supported. Sparse inputs, calibrated drift control, and prototype merging/pruning are not implemented in this version.