Probability Model Architecture: Registry + Per-Stage Selection

Status: Design doc. Specifies how the framework should handle multiple probability mechanisms — naive uniform / simple weighted / population-conditional / domain-specific — and how the right mechanism gets selected at each stage of a walk.

Pairs with: probability-walk-design.md (the computational design that produced corridor BFS + rate weighting), framework-synthesis-cross-domain-and-population.md (the conceptual deepening on cross-domain manifestations + population context), summary-abiogenesis-complete.md §2 (the corpus's three-layer model).

TL;DR: The corpus says probability mechanisms VARY by stage — chemistry rules at R0, autocatalytic threshold at R1.3, vesicle group selection at R1.7, Darwinian post-R2. Our framework currently applies one mechanism (Bernoulli sampling with relative-weighted rates) uniformly. The fix is a probability-model registry where each model is a pluggable function (transition, context) → P_per_trial, and per-stage population_context.v1 declares which model applies. Three baseline models (uniform / rate_weighted / population_sampled) are always available; domain-specific models (autocatalytic_threshold, vesicle_group_selection, chemistry_kinetic, darwinian) plug in when implemented and fall back to baseline when not. Naive analysis always works; sophistication is additive.

1. The problem

summary-abiogenesis-complete.md §2.1 articulates the methodology's three coupled layers:

Topology (Layers 1-3): What CAN happen. The lattice of possible positions, filtered by dependencies, producing the coherent sub-lattice. DETERMINISTIC. Scale-invariant.

Rate (Physics): How FAST it happens. Thermodynamic costs, kinetic barriers, search space sizes, information-theoretic limits. QUANTITATIVE per-transition. Scale-dependent.

Trajectory (Layer 4): What DOES happen. The actual walk, shaped by topology + rate + context. PROBABILISTIC distribution over positions at each time point.

§2.3 specifies that the probability funnel changes shape across stages — different stages have different constraint mechanisms (chemistry rules, template chemistry, autocatalytic feedback, group selection, Darwinian).

Our framework has:

• Topology layer ✓ — corridor BFS with C(k) = F(k) ∩ B(k), conjunction floor + active deps per gate.
• Rate layer ⚠️ — rate.v1 per-transition relative weights with forward + Bayesian backward propagation. Single uniform mechanism applied across all transitions. Doesn't differentiate "chemistry rate" from "autocatalytic threshold" from "group selection."
• Trajectory layer ⚠️ — modal-path overlay shows typical trajectories; sensitivity chart uses Bernoulli P_realized = 1 - (1-p)^N. 1-(1-p)^N is appropriate for "sample N coin flips" — it isn't appropriate for autocatalytic exponential dynamics, vesicle population genetics, or Darwinian fitness.

The gap: per-stage probability MECHANISMS vary; the framework currently doesn't accommodate this.

This is the right gap to address now because:

1. Without it, the rate-weighted analysis bakes in a Bernoulli assumption that's wrong for most abiogenesis stages.
2. The user's directional ask — "naive, simple weighted, more complex models with different parameters and settings... extensible enough to drop in whatever model" — is exactly this gap.
3. The framework will be used across X-genesis (abiogenesis, ontogenesis, software adoption, cultural development); each scope has different natural mechanisms. Hard-coding one is a dead end.

2. The architectural pattern: probability-model registry

2.1 Model interface

A probability model is a function:

def model(transition, context) -> ProbabilityResult

  transition: {
    "from_position": <Position>,           # where the system is
    "to_position":   <Position>,           # candidate next position
    "chain_level":   <str>,
    "primitive_id":  <str>,
    "source_level":  <int>,
    "target_level":  <int>,
  }

  context: {
    "active_population_context": <population_context.v1 doc | None>,
    "active_rate_table":         <rate.v1 doc | None>,
    "active_manifestation":      <manifestation.v1 doc | None>,
    "arrangement":               <arrangement object>,
    "model_parameters":          <dict, model-specific>,
    "elapsed_time":              <float | None>,
  }

ProbabilityResult: {
  "p_per_trial":      <float in [0, 1]>,
  "distribution":     <optional: distribution params>,
  "model_used":       <str, name of model that produced this>,
  "fallback_chain":   <list[str], models tried before this one resolved>,
  "provenance":       <str, source of the value (literature ref, etc.)>,
  "notes":            <str>,
}

The model returns the per-trial probability for the candidate transition given the active context. Implementations are responsible for any internal computation (kinetic rates, search-space integration, population dynamics).

2.2 Model registry

A central registry maps model names to implementations:

# compute/lib/probability_models.py

REGISTRY: dict[str, Callable] = {}

def register_model(name: str):
    def decorator(fn):
        REGISTRY[name] = fn
        return fn
    return decorator

def get_model(name: str) -> Callable | None:
    return REGISTRY.get(name)

The registry is populated at module import time. Adding a new model is a one-decorator-line affair:

@register_model("chemistry_kinetic")
def chemistry_kinetic_model(transition, context):
    # Compute rate from ΔG, concentrations, etc.
    ...

Models live in compute/lib/probability_models.py (baselines) and individual files like compute/lib/probability_models_chemistry.py (domain-specific) when authored.

2.3 Per-stage selection via population_context.v1

The active model at each transition is declared in the population_context's transitions[]:

population_context.v1 (extended)
  ...
  transitions:
    - target: r0p5
      probability_model: "chemistry_kinetic"
      model_parameters:
        delta_g_kcal_per_mol: -7.5
        catalyst_present: "FeS"
        substrate_concentration: 1e-3
      provenance: "Suga lab flexizyme studies, ~10⁻⁵ Kd"
      ...
    - target: r1p3
      probability_model: "autocatalytic_threshold"
      model_parameters:
        fidelity_threshold: 0.85
        baseline_fidelity: 0.70
      provenance: "exploration-genesis-transition-molecular-resolution.md §4.2"
      ...

Each transition declares which model applies and its parameters. The compute layer reads this and dispatches to the registered implementation.

2.4 Fallback chain

When a transition doesn't specify a model, or specifies one that isn't registered, the compute applies a fallback:

1. If transition specifies probability_model AND it's registered → use that model.
2. Else if transition specifies probability_model BUT not registered → use first registered baseline that can produce a value (typically population_sampled), tag with `fallback_chain: ["<requested>", "population_sampled"]`. User sees that the requested model wasn't available.
3. Else if population_context has population.size and per_trial_probability → use `population_sampled`.
4. Else if rate.v1 has weights for this transition → use `rate_weighted`.
5. Else → `uniform_among_coherent`.

Result: the analysis ALWAYS produces a value. Every level of sophistication is additive. The user can run a walk with no rates and no populations and get the naive uniform corridor; adding rates upgrades; adding populations upgrades further; adding domain-specific models upgrades further still.

2.5 Why this is the right shape

• Naive baseline always works — uniform_among_coherent requires no input beyond the corridor itself.
• Simple weighted is one drop-in — author a rate.v1 file, get rate-weighted analysis automatically.
• Population-conditional is one schema field — add population.size and per_trial_probability to a population_context, get Bernoulli-sampled realized probability.
• Domain-specific models are pluggable — implement the function, register, declare in population_context. The compute infrastructure doesn't change.
• Provenance is first-class — every probability value carries model_used, fallback_chain, provenance. No black-box numbers.
• Per-stage selection is data, not code — switching the active model at a stage is a JSON edit, not a compute change.
• Generic across X-genesis — same registry pattern; different model authors register different models for different scopes.

3. The baseline models (implement now)

3.1 uniform_among_coherent

The naive null model. Every coherent forward-tangent move from a position is equally likely.

@register_model("uniform_among_coherent")
def uniform_among_coherent(transition, context):
    arr = context["arrangement"]
    pos = transition["from_position"]
    n_options = len(tangent_set(pos, arr, "forward"))
    p = 1.0 / n_options if n_options > 0 else 0.0
    return ProbabilityResult(p_per_trial=p, model_used="uniform_among_coherent",
                             fallback_chain=[], provenance="combinatorial null model",
                             notes="Every coherent neighbor equally likely; no rate or population information used.")

Use case: bare corridor analysis. Floor for what the structure forces vs. what's combinatorially permitted.

3.2 rate_weighted

Probability proportional to rate.v1 forward weights, normalized across the outgoing tangent set.

@register_model("rate_weighted")
def rate_weighted(transition, context):
    rate_table = context["active_rate_table"]
    if rate_table is None:
        return None  # signal the fallback chain to skip this
    # Sum forward weights across all forward moves from this position
    arr = context["arrangement"]
    pos = transition["from_position"]
    moves = tangent_set(pos, arr, "forward")
    total_w = sum(_rate_for_move(rate_table, m) for m in moves)
    w = _rate_for_move(rate_table, transition)
    p = w / total_w if total_w > 0 else 0.0
    return ProbabilityResult(p_per_trial=p, model_used="rate_weighted",
                             fallback_chain=[], provenance=f"rate.v1 weight {w:.3f} of total {total_w:.3f}",
                             notes="Relative rate weights; assumes Bernoulli per-trial selection among coherent neighbors.")

Use case: the current rate-weighted analysis. Captures relative rate differences without requiring absolute kinetic numbers.

3.3 population_sampled

Bernoulli with population sampling: P_realized = 1 - (1 - p)^N.

@register_model("population_sampled")
def population_sampled(transition, context):
    pc = context["active_population_context"]
    if pc is None:
        return None
    p = pc.transitions[transition.target].per_trial_probability
    N = pc.population.size  # or N × τ if time_available is set
    # Realized probability of the transition occurring at least once in the population
    p_realized = 1.0 - (1.0 - p) ** N
    return ProbabilityResult(p_per_trial=p_realized, model_used="population_sampled",
                             fallback_chain=[], provenance=f"Bernoulli on N={N:.1e} with p={p:.1e}",
                             notes="Independent-trials assumption. Appropriate for discrete sampling of a search space; NOT appropriate for autocatalytic dynamics, group selection, or Darwinian fitness.")

Use case: the current sensitivity chart. Captures how realized probability scales with population size for transitions where the mechanism is "search the configuration space."

4. Future model classes (don't implement now; framework accommodates)

The corpus identifies several stage-specific mechanisms. We declare them as named model classes so population_contexts can request them. Until implemented, each falls back to population_sampled. The fallback is transparent: the output records fallback_chain: ["chemistry_kinetic", "population_sampled"] so the analyst sees the gap.

4.1 chemistry_kinetic

ΔG-based reaction rates. Parameters: ΔG_kcal_per_mol, temperature, catalyst, substrate concentrations. Produces per-trial probability of the reaction occurring at the per-pore level.

Source candidates: reaction rate data from origin-of-life literature; thermodynamic databases.

Stages where applicable: R0 → R0.2 (aminoacylation), R0.2 → R0.5 (template peptide synthesis), and most chemistry-level transitions throughout.

4.2 autocatalytic_threshold

Logistic-like threshold dynamics. Below threshold (~80% fidelity), linear improvement. Above threshold, exponential. Parameters: fidelity_threshold, baseline_fidelity, improvement_per_cycle.

Source: exploration-genesis-transition-molecular-resolution.md §4.2.

Stage where applicable: R1 → R1.3 specifically. The bootstrap loop activation is THE classic autocatalytic threshold.

4.3 vesicle_group_selection

Population genetics on vesicles with differential growth. Parameters: vesicle population size, fitness variance, growth rate, parasite-to-functional ratio.

Source: Eigen 1971 + exploration-physical-compartmentalization-and-probabilistic-walks.md §1.

Stage where applicable: R1.7 → R1.9 specifically. The parasite crisis crossing is group-selection-driven, not Bernoulli-sampled.

4.4 bootstrap_expansion

Sequential dependency expansion (each new amino acid built from existing). Parameters: amino acid recruitment order, biosynthetic complexity.

Source: exploration-code-structure-and-pre-R2-feedback.md §1.3 + 2024 PNAS recruitment-order paper.

Stage where applicable: R1.7 → R1.9, code expansion specifically.

4.5 frozen_accident

One-shot crystallization with irreversibility. Once probability mass passes the threshold, it's locked. Parameters: crystallization rate, irreversibility threshold.

Source: exploration-code-structure-and-pre-R2-feedback.md §6 (frozen accident as a new stability concept).

Stage where applicable: R1.9 → R2 specifically.

4.6 darwinian

Heritable variation × differential reproduction. Parameters: mutation rate, selection coefficient, generation time, population size.

Source: standard population genetics.

Stages where applicable: post-R2 (any walk through biology arrangement at organism-architecture or ecosystem levels).

4.7 empirical_lookup

Table-of-measured-rates. Parameters: literature reference, measurement context, scaling factors.

Source: any kinetic study with measured rate constants.

Stages where applicable: any transition with empirical data. Most useful when the analyst has specific data they want to plug in directly.

5. Per-stage mechanism table for abiogenesis

For each, the framework supports declaring the recommended model in the population_context. Until implemented, fallback to population_sampled produces an answer with the fallback chain noted.

Cross-domain implications: the same table for ontogenesis would have developmental_canalization instead of vesicle_group_selection, etc. For software adoption, network_effects and competitive_displacement. For cultural development, transmission_with_drift. The framework treats these uniformly: name the model, declare parameters, plug in when implemented.

6. Implementation phases

Phase A — Registry skeleton + 3 baselines

Implement:

• compute/lib/probability_models.py with REGISTRY, register_model, get_model, ProbabilityResult dataclass, apply_model_with_fallback.
• The three baseline models: uniform_among_coherent, rate_weighted, population_sampled.
• Integration with corridor compute: compute_corridor gains optional population_context_ref and rate_ref params; passes them as context.

Test: re-run abiogenesis-r0-to-r2 corridor with population_context_ref=abiogenesis-r0p2-hadean.v1.json, verify output matches current population_sampled behavior.

Effort: ~1 session.

Phase B — Author per-stage population_contexts

Author one population_context per Mn in the abiogenesis trajectory:

• abiogenesis-r0-hadean.v1.json — declares chemistry_kinetic for r0→r0p2 with parameters from corpus; falls back to population_sampled until model is implemented.
• abiogenesis-r0p2-hadean.v1.json (extend existing) — declares chemistry_kinetic for r0p2→r0p5 + r0p5→r1.
• abiogenesis-r1-hadean.v1.json — declares autocatalytic_threshold for r1→r1p3.
• abiogenesis-r1p7-hadean.v1.json — declares vesicle_group_selection for r1p7→r1p9.
• abiogenesis-r2-luca-darwinian.v1.json — declares darwinian for r2→diversification.

Each carries the parameters from the corpus + provenance pointers + relevant_sub_manifestations. Most fall back to population_sampled until the named models are implemented.

Effort: ~30-60 min per Mn × 5 = ~3-5 hours.

Phase C — Refactor sensitivity chart to use the registry

Replace inline p_per_trial arrays in plot_abiogenesis_sensitivity.py with reads from population_context files. The chart calls apply_model_with_fallback per transition. Output now includes a column showing which model was actually used (with fallback chain) — the analyst sees what's data-driven vs falling back.

Effort: ~1 hour.

Phase D — Document the per-stage mechanism table

Markdown notes file methodology_strategy/per-stage-mechanisms.md capturing the table above with citations to corpus sources. One section per X-genesis (abiogenesis / ontogenesis / phylogenesis / technogenesis / sociogenesis) showing analogous per-stage mechanism tables. This is the user-facing menu of "what model to declare in your population_context."

Effort: ~2 hours; durable artifact.

Phase E (optional, future) — Implement domain-specific models

When analytical demand arises, implement specific models from §4 above. Each is its own file in compute/lib/probability_models_<domain>.py. Authoring grounds parameters in literature.

Effort: ~1 session per model; deferrable until the analysis warrants it.

Phasing rationale

A is the unblocker — once registry + fallback chain exist, every other phase becomes incremental data work. B-D give us the per-stage selection in production. E is on-demand. The user said "we don't necessarily need to build complex probabilistic models" — phase E is exactly that, kept off the critical path.

7. Validation and correctness

How do we know the architecture is right?

1. Backwards compatibility check. Running with no population_context + no rate.v1 = uniform_among_coherent → corridor width should match structural width (the current corridor.v1.corridor_widths_by_rank). Running with rate.v1 + no population_context = rate_weighted → output should match the current plot_rate_corridor.py. Running with population_context + p_per_trial = population_sampled → output should match the current sensitivity chart.

2. Fallback chain transparency. Every probability value in compute output records the chain. The analyst can audit: "was this from the chemistry model I asked for, or did it fall back?" If too many fallbacks, the analyst knows the picture is partially unimplemented.

3. Provenance traceability. Every probability has a citation (literature ref or "heuristic"). Discipline rule: don't ship analysis where critical-path values are heuristic without flagging.

4. Cross-arrangement portability. The same registry should support a software-adoption walk (with network_effects registered) without any infrastructure changes — only data authoring.

8. Open questions

1. How to handle multi-domain transitions? A transition like R≥6 ∧ Cmp≥2 ∧ P≥1 (parasite crisis) involves three primitives. Does the model apply to each individually or to the conjunction? Probably the conjunction — the model gets the "current position" and the "candidate next position" and decides probability for the joint move.

2. How to handle continuous time vs discrete trials? population_sampled uses discrete N trials. chemistry_kinetic and darwinian are continuous-time. The model interface accommodates both (returning p_per_trial is interpreted as "per-trial-or-per-time-step"), but analyses that mix them need a consistent time semantics. Address via the elapsed_time field in context.

3. How to handle uncertainty/distributions? Baseline models return point values. More sophisticated models might return distributions (e.g., posterior over p given evidence). The ProbabilityResult.distribution field accommodates this; downstream compute either uses point estimate or samples the distribution.

4. Multi-walk integration. When two walks meet (e.g., abiogenesis trajectory + biology arrangement at LUCA), models should compose. Probably via the registry: the joint walk is just a walk in a product arrangement; same registry applies.

5. How does this interact with reverse walks? Currently we have forward + Bayesian backward propagation. The registry should apply to BOTH directions — backward propagation also needs per-transition probabilities. The model interface accepts a direction parameter (forward/backward).

9. Documentation and migration

This design doc supersedes the bare population_sampled assumption baked into the current sensitivity chart. After phase A is done, update:

• framework-synthesis-cross-domain-and-population.md §3 (the population-driven probability section) — reframe to reference the registry as the implementation surface.
• probability-walk-design.md §2.3 (uniform vs physics-weighted model) — update with the registry framing.
• population_context.v1.schema.json — add transitions[].probability_model and transitions[].model_parameters fields per §2.3 above.
• MEMORY.md — note that the probability model is now pluggable.

Once phase B is done, every analysis script that consumes per-trial probabilities reads them through the registry. No more hardcoded values in script bodies.

10. Why this is the right shape architecturally

It mirrors how science actually works: a hierarchy of models from simple to complex, each appropriate to specific analytical contexts. The default model gives you the structural skeleton; sophisticated models add quantitative precision when warranted; the framework accommodates all of them without forcing premature commitment.

It's also future-proof for the X-genesis generalization: ontogenesis will need developmental_canalization; software adoption will need network_effects; cultural development will need transmission_with_drift. None of these require infrastructure changes — just one decorator and one population_context per scope.

The user's framing was right: "the framework should be flexible enough for us to drop in whatever model we need at that point in time, pull the populations, run samples, and see what comes out." The registry is that flexibility, made concrete.

Referenced by the model

Cited as a source by 1 model record (browse the model census):

• abiogenesis-r0-hadean — population_context abiogenesis/sc2/abiogenesis-r0