Comparison, archetypes, and joint manifestations — methodology expansion

Status: APPROVED → IMPLEMENTED. Phases 1A, 1B, 1C all complete for entity arrangement; Phase 2 (Lens B archetype scoring) and Phase 4 (cross-arrangement extension) are next. Open-question decisions captured in §5; implementation record in §9. Scope: Methodology-level. Applies cross-arrangement (not just entity). Touches the analytical compute layer, the data schema, and the canonical architecture rules.


9. Implementation record

Full plan execution in one extended session arc. Phases delivered (vs original §4 roadmap):

PhasePlan §4 scopeStatusNotes
1ASchema metadata + fix instagram-client + tag 2 declarative anchorsDONE7 optional fields added; 0 schema validations broken
1BLens A heatmap + Lens C weighted + Lens E classical clusteringDONE++Lens A + E shipped; Lens C deferred (Lens B more valuable next); ALSO shipped Lens G (signatures), scope-based clustering, stability + meta-stability — 5 analytical lenses total
1CInductive anchor identification + Lens B archetype scoringPARTIAL8 inductive anchors authored; 2 declaratives refined; Lens B deferred to next session
(extra)Scenario projection (entity-system pre-release/modest/maximal)DONENew scenario_projection corpus_role; 2 projection Mns
(extra)Corpus expansion Waves 1+2+3 — 12 new MnsDONESelective adds before anchor authoring; validated cluster boundaries
2Archetype machinery (Lens B + automated identification)NEXT SESSIONLens B is the natural pickup
3Joint Mn compositionDEFERREDSchema metadata is in place; composition library pending
4Cross-arrangement extension (biology, cognition, methodology)DEFERREDApply same arc to other arrangements
5Strategy doc refreshPARTIALThis Status block + Rule 9/10 in canonical-architecture-strategy.md

Empirical findings worth absorbing into canonical-architecture-strategy.md:


1. The problem this addresses

The framework now has 147+ manifestations across 5 arrangements (biology, cognition, entity, methodology, abiogenesis), each authored as a position in a partial-level lattice spanning multiple chain levels. Comparison between Mns is the core analytical operation — but the current comparison machinery is reductive:

The user's framing: "right now the patterns, you just can't tell ... we got to take this and really figure it out because this impacts all the papers and it's critical for us to understand it."


2. Methodological concepts to add

Important framing principle — everything is a manifestation. There are no new object types in this plan; what's new is:

The data model stays uniform: one manifestation.v1.schema.json, one data/manifestations/ directory, one set of compute scripts. Variations are metadata + intent of use, not new schemas.

2.1 Multi-lens comparison

Principle: Don't build one canonical analytical view. Build multiple lenses with explicit names; let the analytical question pick the lens.

Initial lens inventory:

LensPurposeInputsOutput
A — Per-primitive distribution heatmapSee raw structure without aggregationMns × primitives × levelsHeatmap; rows sorted by similarity OR per-primitive rank
B — Archetype scoringConvert distances into named categoriesMns + archetype templatesPer-Mn × archetype score matrix
C — Weighted-similarity / filtered viewAnswer scoped questions ("under the content-addressing lens, which Mns are near git?")Mns + primitive weight profileRe-ranked distance table
D — Multi-facet projectionSeparate substrate / bridge / app-arch / ecosystem distanceMns × chain-level fingerprints2D plots: substrate-distance × bridge-distance, etc.
E — Classical clusteringk-means, hierarchical, DBSCAN as standard tools applied to the framework's vector representationMns × full fingerprintCluster assignments + dendrograms
F — Joint Mn composition"What does A+B+C look like? Which existing Mns is it near?"Mn set + composition ruleSynthetic Mn + comparison to corpus
G — Higher-order primitive combinationsCluster on pair/triad/quad signatures rather than raw single primitivesMns × pair-density (or triad/quad) per chain levelDerived signature vectors, then run through Lens A/B/C/D/E

Lenses are complementary, not alternative. A single Paper-6 figure might use Lens A (heatmap) + Lens B (archetype labels) overlaid. A Paper-11 figure might use Lens D (facet projection) to argue cross-chain coherence. Lens G feeds A-E by transforming the per-Mn representation (raw primitives → pair densities → triad densities).

Lens G — combinatorial signatures (note on scope). Pair-relationships are already encoded in the framework — every domain and bridge JSON has a pair_relationships field describing structurally-significant pairs at that chain level. Triads and quads aren't formalized yet but the combinatorial space exists. Lens G computes per-Mn signatures over these combinations:

Implementation: Each lens is one compute script reading from data/manifestations/* + an optional weight/template profile. CLI flags control lens behavior. Output: figure + JSON result file (for downstream re-analysis).

2.2 Archetypes — canonical-anchor manifestations

Framing correction: An archetype is NOT a separate kind of object. It is a manifestation — specifically, a Sc=2 range-encoded manifestation that we recognize and use as a canonical comparison anchor for a class of Sc=3 unified instances. The "archetype-ness" is in HOW the Mn is used, not in WHAT it is.

content-addressed-data-systems.v1.json and interactive-web-platforms.v1.json are already archetypes by this definition — Sc=2 range-Mns describing regions of the entity arrangement. The contribution of this plan is scaling the practice (more archetypes, especially inductively-derived ones from clustering) and standardizing how they're used in comparison.

Examples that are LIKELY to surface from clustering:

Candidate archetype nameChain levelsPrimitive patternExemplar Mns
event-streaming-substrateentity-system + e2a-bridgeSub=4-5, Inb=4-5, Pg=4-5, M=3-5kafka, slack-substrate, discord
content-addressed-storeentity-system + e2a-bridgeI=3-5, Hi=3-5, Cnt=3-4, Sub=1-2git, ipfs, docker (image layers)
p2p-overlayentity-system + e2a-bridgeP=3-5, Net=4-5, Syn=3-5bitcoin, ipfs, holochain, urbit
reactive-computeentity-system + e2a-bridgeCmp=3-5, Sub=2-4, M=2-4spreadsheets, urbit, smalltalk, nix
social-platform-appapplication-arch + a2e-bridgePg=4-5, Pc=4-5, Pn=4-5, Ap=4-5discord, slack, instagram, figma
infrastructure-as-substratefull chainSa=5, Os=4-5, Ua=4-5, Pp=0linux-posix, http-rest, smtp-email

(Biology candidates: photosynthetic-eukaryote, bilaterian-protostome, chordate, etc. Cognition: organism-individual-cognition, social-band-cognition, cultural-ecosystem. Methodology: qualitative-interpretive, quantitative-formal, iterative-design. These all become Sc=2 range-encoded Mns in the same data model as everything else.)

Lifecycle:

  1. Clustering pass — run Lens A + Lens E on the Mn corpus per arrangement
  2. Candidate identification — clusters with N ≥ 3 Mns and interpretable shape become archetype candidates
  3. Authoring — write a new Sc=2 Mn in data/manifestations/<archetype-name>.v1.json using the existing manifestation schema. Range-encode primitive levels as {min, max} (and optionally modal) per the population-Mn convention already in use. Tag metadata to capture intent (see §2.4 below).
  4. Validation — bound-check + coherence-check using existing tooling (same as any other Mn).
  5. Adoption — new Mns compare against the anchor set (tagged Mns, ~10-30 per arrangement) instead of against the full corpus. Score = 1 - mean(cosine_distance(Mn, archetype_modal_or_centroid)) weighted by archetype-membership confidence.

Relationship to existing population-Mns. None special — they ARE archetypes; they just predate the framing. Once §2.4 metadata is in place, tag the existing ones with corpus_role: anchor and they participate in the new machinery without migration.

Why archetypes matter for papers:

2.3 Joint manifestations

Framing: Like archetypes, joint Mns are NOT a separate object kind. They are manifestations — typically Sc=3 unified instances — whose position is computed from constituent Mns via a declared composition rule, rather than authored directly. The "joint-ness" is metadata (composed_of, composition_rule); the position values look the same as any other Sc=3 Mn.

Composition rules — first cut (to be refined):

Example: joint(postgres, redis, http-rest) should produce a Mn with strong Hi (postgres), strong Sub (redis pub-sub), strong Net (http-rest), AND should activate the bridge primitives that link these — likely Cnt + Tr at e2a-bridge, Ap + Dp at a2e-bridge. Compare the joint Mn signature to existing Mns: which existing system is structurally closest? (Plausibly: github, slack, or one of the messaging platforms.)

Joint Mn — surface vs internals. "Slack-as-surface" (the product) and "Slack-as-internals" (postgres + redis + custom infra) are two views of the same system. Joint Mn composition computes the internals view by composing constituents. The surface view is the existing single-Mn authoring. Both should exist for high-value systems; consistency between them is a methodological check. Both live in data/manifestations/; the distinction is metadata only.

Why joint matters for papers:

2.4 Metadata fields — uniform data model with filterable roles

Per §2.2/§2.3 framing: everything is a manifestation, distinguished by metadata. The schema gets a small set of optional fields that let analyses filter by role and trace provenance.

Proposed additions to manifestation.v1.schema.json:

FieldTypePurposeDefault
corpus_roleenum: instance | anchor | composition | exemplarWhat role this Mn plays in analyses; analyses default to including instances and excluding anchors unless explicitly requestedinstance
originenum: direct_authoring | declarative_population | inductive_clustering | composed_from_constituentsHow the Mn came to be authored, for provenancedirect_authoring
comparison_inclusionenum: default_include | anchor_only | opt_inFine-grained inclusion control for landscape analyses (e.g., archetype Mns set to anchor_only won't appear in normal landscape plots)default_include
exemplarsarray of Mn refs (optional)For anchor-Mns: list of Sc=3 Mns that exemplify this regionomitted
composed_ofarray of Mn refs (optional)For joint Mns: constituents that compose into this Mnomitted
composition_ruleenum (optional): element_wise_max | element_wise_sum_capped | coupling_aware | customThe rule used to compute joint positionomitted
derived_from_clusterobject (optional): { analysis_run, cluster_id, cluster_spread_stats }Provenance for inductively-derived archetypesomitted

Filtering behavior in compute scripts:

This means landscape analyses, fingerprint comparisons, and trajectory work all keep their existing default behavior. Archetype-aware analyses opt in explicitly.

The Sc field already does much of this work. Most landscape analyses already filter to Sc=3 unified instances; Sc=2 range-Mns are excluded by default. The new metadata adds fine-grained control on top of Sc-based filtering — useful when an Sc=3 joint Mn (composed) should be distinguished from an Sc=3 direct-authored Mn.


3. Cross-arrangement applicability

Each concept above applies to all five arrangements (and the abiogenesis fragment), but the content of archetypes + composition rules varies:

3.1 Entity arrangement

3.2 Biology arrangement

3.3 Cognition arrangement

3.4 Methodology arrangement

3.5 Abiogenesis arrangement


4. Implementation roadmap

Ordered by sequential dependency (later items build on earlier).

Phase 1 — Multi-lens compute (1-2 sessions)

Phase 2 — Archetype methodology (2-3 sessions)

Phase 3 — Joint manifestation composition (2-3 sessions)

Phase 4 — Cross-arrangement extension (1-2 sessions per arrangement)

Phase 5 — Methodology document refresh (1 session)


5. Decisions (locked)

5.1 Anchor-Mn granularity — empirical cluster-spread rule

For an inductively-derived anchor-Mn:

For declarative anchor-Mns (existing population-Mns like content-addressed-data-systems): keep existing range encoding; no rule change.

5.2 Composition rule defaults — saturating-max with explicit additive opt-in

5.3 Inductive vs declarative reconciliation — declarative wins on name; range merges

When inductive clustering surfaces a candidate that overlaps with an existing declarative anchor-Mn:

5.4 Cross-arrangement isomorphism encoding — isomorphic_to field, deferred

Add an optional isomorphic_to: [{mn_ref, distance, chain_level}] field to the manifestation schema. Applicable to any Mn (anchor or instance). Populated when cross-arrangement structural twins are verified empirically — e.g., from Finding 19's bridge-fingerprint analysis (biology-to-organism ↔ neural-to-cognitive d=0.237).

Concrete encoding deferred: do not add the field to the schema until we have at least one cross-arrangement isomorphism worth encoding. Add it during Phase 4 as cross-arrangement work surfaces real matches.

5.5 Scaling — confirmed manageable

~6-8 anchor-Mns per arrangement × 5 arrangements ≈ 30-40 anchors total. New Mn comparison becomes O(anchors) ≈ 30 instead of O(corpus) ≈ 147. ~5× reduction in comparison cost and significantly more interpretable output.

5.6 Paper-validation gate — Phase 2 doesn't scale past entity until entity validates

After Phase 2 produces entity-arrangement anchors, validate that:

If (a)+(b)+(c) check out, scale to biology/cognition/methodology. If not, revisit Phase 2 design before extending.

5.7 Metadata vocabulary clarifications

5.8 Schema version handling

The §2.4 metadata additions are all optional fields on manifestation.v1.schema.json. Existing 147 Mns remain schema-valid without updates. No v2 bump needed; add fields and re-validate everything.


6. What this changes for ongoing work

This is genuinely the priority. The backlog items (per-figure L4 captions, dirac-bridge authoring) remain low-priority.


7. Out of scope for this plan (deliberately)


8. Session-start checklist for the next sitting

When picking up next session, work through this in order. Each item is concretely scoped; no further design decisions needed.

Phase 1A — Foundation (single session, ~half-day)

Phase 1B — Lenses (subsequent session)

Phase 1C — Analysis run + first anchors (subsequent session)

Phase 2 — Archetype machinery (subsequent sessions)

After Phase 1 anchors exist:

Phase 3 — Joint composition (parallel to or after Phase 2)

Phases 4-5 — Cross-arrangement extension + strategy doc refresh

Per existing §4 of this plan. Scheduled after Phase 1+2 entity work validates.


Approximate time-cost: Phase 1A = ~half-day; Phase 1B = 1 session; Phase 1C = 1 session; Phase 2 = 2-3 sessions; Phase 3 = 2-3 sessions; cross-arrangement + strategy refresh = additional sessions per arrangement.


Referenced by the model

Cited as a source by 8 model records (browse the model census):