Comparison, archetypes, and joint manifestations — methodology expansion
Status: APPROVED → IMPLEMENTED. Phases 1A, 1B, 1C all complete for entity arrangement; Phase 2 (Lens B archetype scoring) and Phase 4 (cross-arrangement extension) are next. Open-question decisions captured in §5; implementation record in §9. Scope: Methodology-level. Applies cross-arrangement (not just entity). Touches the analytical compute layer, the data schema, and the canonical architecture rules.
9. Implementation record
Full plan execution in one extended session arc. Phases delivered (vs original §4 roadmap):
| Phase | Plan §4 scope | Status | Notes |
|---|---|---|---|
| 1A | Schema metadata + fix instagram-client + tag 2 declarative anchors | DONE | 7 optional fields added; 0 schema validations broken |
| 1B | Lens A heatmap + Lens C weighted + Lens E classical clustering | DONE++ | Lens A + E shipped; Lens C deferred (Lens B more valuable next); ALSO shipped Lens G (signatures), scope-based clustering, stability + meta-stability — 5 analytical lenses total |
| 1C | Inductive anchor identification + Lens B archetype scoring | PARTIAL | 8 inductive anchors authored; 2 declaratives refined; Lens B deferred to next session |
| (extra) | Scenario projection (entity-system pre-release/modest/maximal) | DONE | New scenario_projection corpus_role; 2 projection Mns |
| (extra) | Corpus expansion Waves 1+2+3 — 12 new Mns | DONE | Selective adds before anchor authoring; validated cluster boundaries |
| 2 | Archetype machinery (Lens B + automated identification) | NEXT SESSION | Lens B is the natural pickup |
| 3 | Joint Mn composition | DEFERRED | Schema metadata is in place; composition library pending |
| 4 | Cross-arrangement extension (biology, cognition, methodology) | DEFERRED | Apply same arc to other arrangements |
| 5 | Strategy doc refresh | PARTIAL | This Status block + Rule 9/10 in canonical-architecture-strategy.md |
Empirical findings worth absorbing into canonical-architecture-strategy.md:
-
Selective corpus expansion before anchor authoring is high-leverage. 12 Mns added in 3 Waves of 3-5 each, each Wave probing specific cluster boundaries, totally reshaped the anchor inventory: persistent-database disconfirmed → relational-server-DBMS + consensus-KV split; decentralized-minimal expanded → peer-federation-messaging + consensus-KV-substrate; git singleton resolved → vcs-system cluster. Without corpus expansion, the original 6-anchor pre-Wave-1 inventory would have been wrong on 3-4 of the 6.
-
Mn-pair similarity vs anchor candidacy is a useful distinction. A 2-Mn pair at 10-12/12 co-cluster IS a real structural class (mysql+postgres, etcd+zookeeper, claude-code+obsidian, inferno+plan-9) but doesn't pass the N≥3 anchor-candidacy threshold from plan §5.1. The current solution: author N=2 pairs as anchors with a note that they may expand under corpus growth. Either tighten the N≥3 rule, or accept N≥2 with quality bar = 10+/12 stability.
-
Meta-stability is the right lens for cross-config robustness. Clusters that survive 4 chain levels × 3 signatures × 2 methods (24 configurations effectively) are the load-bearing structural classes. Lens A or Lens E at a single (level, signature) can identify candidates but cannot certify robustness.
-
The Pp (Platform Participation) primitive conflates two semantically distinct cases — "independent tiny app" and "foundational substrate" both end up at Pp=0. Worth a future rename or split (External Platform Dependency + Is-Platform). Not blocking current analysis; flagged for primitive-redesign review.
-
Scenario projections (counterfactual Mns) are analytically valuable for systems whose current snapshot misrepresents their structural-class potential. The entity-system case: pre-release Mn appears as a singleton; modest-adoption projection lands in peer-federation-messaging cluster; maximal-adoption projection lands in foundational-standards cluster. Three Mns capture the ontogenetic span of one system. Cross-level meta-stability is NOT the right lens for scenarios (technical primitives are constant by construction); per-level stability at the variation chain level is.
1. The problem this addresses
The framework now has 147+ manifestations across 5 arrangements (biology, cognition, entity, methodology, abiogenesis), each authored as a position in a partial-level lattice spanning multiple chain levels. Comparison between Mns is the core analytical operation — but the current comparison machinery is reductive:
- Cosine distance on z-scored fingerprints collapses 12-D bridge or substrate vectors to a single scalar similarity, then thresholds for clustering. Most Mns end up in one big cluster (31/35 entity Mns at d=0.15 on entity-to-app-bridge; 34/35 at app-to-ecosystem-bridge). The pattern is invisible because we've compressed it out.
- One clustering frame per analysis. The user might care about content-addressing in one query and messaging-substrate in another — these should produce different distance rankings. Currently they don't.
- No archetypes. Each new Mn gets compared against every other Mn pairwise. This is O(N) per query and produces uninterpretable distances. The framework needs canonical structural archetypes that emerge from real-system clustering and serve as the comparison basis going forward.
- No joint manifestations. Real systems compose: "Redis + Postgres + HTTP server + frontend = a working web app." The composed system has its own signature, distinct from any individual part. The framework currently has no operation for this.
- No methodology-level reuse. Software systems aren't the only place archetypes and joint composition matter. Biology has body-plan archetypes (chordate vs arthropod vs cnidarian). Cognition has cognition-kind archetypes. Methodology has epistemic-style archetypes. All four need the same infrastructure.
The user's framing: "right now the patterns, you just can't tell ... we got to take this and really figure it out because this impacts all the papers and it's critical for us to understand it."
2. Methodological concepts to add
Important framing principle — everything is a manifestation. There are no new object types in this plan; what's new is:
- A standardized vocabulary of analytical lenses (§2.1)
- Recognition that some Mns serve as canonical comparison anchors and explicit metadata tagging them as such (§2.2)
- A composition operation for building joint Mns from constituents (§2.3)
- Schema metadata that lets analyses filter by Mn role (§2.4)
The data model stays uniform: one manifestation.v1.schema.json, one data/manifestations/ directory, one set of compute scripts. Variations are metadata + intent of use, not new schemas.
2.1 Multi-lens comparison
Principle: Don't build one canonical analytical view. Build multiple lenses with explicit names; let the analytical question pick the lens.
Initial lens inventory:
| Lens | Purpose | Inputs | Output |
|---|---|---|---|
| A — Per-primitive distribution heatmap | See raw structure without aggregation | Mns × primitives × levels | Heatmap; rows sorted by similarity OR per-primitive rank |
| B — Archetype scoring | Convert distances into named categories | Mns + archetype templates | Per-Mn × archetype score matrix |
| C — Weighted-similarity / filtered view | Answer scoped questions ("under the content-addressing lens, which Mns are near git?") | Mns + primitive weight profile | Re-ranked distance table |
| D — Multi-facet projection | Separate substrate / bridge / app-arch / ecosystem distance | Mns × chain-level fingerprints | 2D plots: substrate-distance × bridge-distance, etc. |
| E — Classical clustering | k-means, hierarchical, DBSCAN as standard tools applied to the framework's vector representation | Mns × full fingerprint | Cluster assignments + dendrograms |
| F — Joint Mn composition | "What does A+B+C look like? Which existing Mns is it near?" | Mn set + composition rule | Synthetic Mn + comparison to corpus |
| G — Higher-order primitive combinations | Cluster on pair/triad/quad signatures rather than raw single primitives | Mns × pair-density (or triad/quad) per chain level | Derived signature vectors, then run through Lens A/B/C/D/E |
Lenses are complementary, not alternative. A single Paper-6 figure might use Lens A (heatmap) + Lens B (archetype labels) overlaid. A Paper-11 figure might use Lens D (facet projection) to argue cross-chain coherence. Lens G feeds A-E by transforming the per-Mn representation (raw primitives → pair densities → triad densities).
Lens G — combinatorial signatures (note on scope). Pair-relationships are already encoded in the framework — every domain and bridge JSON has a pair_relationships field describing structurally-significant pairs at that chain level. Triads and quads aren't formalized yet but the combinatorial space exists. Lens G computes per-Mn signatures over these combinations:
- For each Mn, compute pair-density vector: for each declared pair (P_i, P_j), the level-product or co-activation score
- Optionally extend to triads (combinations of 3 primitives co-activated) and quads where chain-level primitive count permits
- Scope explodes combinatorially — entity-system has 6 primitives → 15 pairs / 20 triads / 15 quads → 50 features; application-architecture has 12 primitives → 66 pairs / 220 triads → 286+ features. Filter aggressively by analyst preference (only the structurally-meaningful pairs per arrangement) before clustering.
- Output feeds back into Lens A (heatmap on pair signatures) / Lens E (cluster on pair signatures) / Lens C (weighted pair-distance)
Implementation: Each lens is one compute script reading from data/manifestations/* + an optional weight/template profile. CLI flags control lens behavior. Output: figure + JSON result file (for downstream re-analysis).
2.2 Archetypes — canonical-anchor manifestations
Framing correction: An archetype is NOT a separate kind of object. It is a manifestation — specifically, a Sc=2 range-encoded manifestation that we recognize and use as a canonical comparison anchor for a class of Sc=3 unified instances. The "archetype-ness" is in HOW the Mn is used, not in WHAT it is.
content-addressed-data-systems.v1.json and interactive-web-platforms.v1.json are already archetypes by this definition — Sc=2 range-Mns describing regions of the entity arrangement. The contribution of this plan is scaling the practice (more archetypes, especially inductively-derived ones from clustering) and standardizing how they're used in comparison.
Examples that are LIKELY to surface from clustering:
| Candidate archetype name | Chain levels | Primitive pattern | Exemplar Mns |
|---|---|---|---|
| event-streaming-substrate | entity-system + e2a-bridge | Sub=4-5, Inb=4-5, Pg=4-5, M=3-5 | kafka, slack-substrate, discord |
| content-addressed-store | entity-system + e2a-bridge | I=3-5, Hi=3-5, Cnt=3-4, Sub=1-2 | git, ipfs, docker (image layers) |
| p2p-overlay | entity-system + e2a-bridge | P=3-5, Net=4-5, Syn=3-5 | bitcoin, ipfs, holochain, urbit |
| reactive-compute | entity-system + e2a-bridge | Cmp=3-5, Sub=2-4, M=2-4 | spreadsheets, urbit, smalltalk, nix |
| social-platform-app | application-arch + a2e-bridge | Pg=4-5, Pc=4-5, Pn=4-5, Ap=4-5 | discord, slack, instagram, figma |
| infrastructure-as-substrate | full chain | Sa=5, Os=4-5, Ua=4-5, Pp=0 | linux-posix, http-rest, smtp-email |
(Biology candidates: photosynthetic-eukaryote, bilaterian-protostome, chordate, etc. Cognition: organism-individual-cognition, social-band-cognition, cultural-ecosystem. Methodology: qualitative-interpretive, quantitative-formal, iterative-design. These all become Sc=2 range-encoded Mns in the same data model as everything else.)
Lifecycle:
- Clustering pass — run Lens A + Lens E on the Mn corpus per arrangement
- Candidate identification — clusters with N ≥ 3 Mns and interpretable shape become archetype candidates
- Authoring — write a new Sc=2 Mn in
data/manifestations/<archetype-name>.v1.jsonusing the existing manifestation schema. Range-encode primitive levels as{min, max}(and optionallymodal) per the population-Mn convention already in use. Tag metadata to capture intent (see §2.4 below). - Validation — bound-check + coherence-check using existing tooling (same as any other Mn).
- Adoption — new Mns compare against the anchor set (tagged Mns, ~10-30 per arrangement) instead of against the full corpus. Score =
1 - mean(cosine_distance(Mn, archetype_modal_or_centroid))weighted by archetype-membership confidence.
Relationship to existing population-Mns. None special — they ARE archetypes; they just predate the framing. Once §2.4 metadata is in place, tag the existing ones with corpus_role: anchor and they participate in the new machinery without migration.
Why archetypes matter for papers:
- Paper 6 (convergent evolution) is fundamentally an archetype-comparison argument — "these systems converged on the content-addressed-store archetype." Currently we hand-wave this; identifying explicit Sc=2 anchor-Mns formalizes it.
- Paper 11 (structural methodology) needs to ARGUE that structure-comparison is the right analytical move; archetypes are the unit of comparison.
- Paper 0 (the six primitives) gains clarity if anchor-Mns are surfaced as "regions of the primitive-combinatorics that real systems land in" rather than "all possible primitive combinations are realizable."
2.3 Joint manifestations
Framing: Like archetypes, joint Mns are NOT a separate object kind. They are manifestations — typically Sc=3 unified instances — whose position is computed from constituent Mns via a declared composition rule, rather than authored directly. The "joint-ness" is metadata (composed_of, composition_rule); the position values look the same as any other Sc=3 Mn.
Composition rules — first cut (to be refined):
- Element-wise max for primitives where the strongest constituent dominates (e.g., if A has Sub=5 and B has Sub=0, joint has Sub=5)
- Element-wise sum capped at max-level for additive primitives (e.g., if A and B both have D=3, joint has D=min(6, max_level))
- Coupling-aware at bridge primitives: the bridge between A and B activates specific primitives based on the coupling declared in A's or B's
couples_withfield - Coherence enforcement: after composition, validate that the joint position remains coherent under all dependency constraints. If not, the composition is invalid (which is a finding — "these systems don't compose cleanly").
Example: joint(postgres, redis, http-rest) should produce a Mn with strong Hi (postgres), strong Sub (redis pub-sub), strong Net (http-rest), AND should activate the bridge primitives that link these — likely Cnt + Tr at e2a-bridge, Ap + Dp at a2e-bridge. Compare the joint Mn signature to existing Mns: which existing system is structurally closest? (Plausibly: github, slack, or one of the messaging platforms.)
Joint Mn — surface vs internals. "Slack-as-surface" (the product) and "Slack-as-internals" (postgres + redis + custom infra) are two views of the same system. Joint Mn composition computes the internals view by composing constituents. The surface view is the existing single-Mn authoring. Both should exist for high-value systems; consistency between them is a methodological check. Both live in data/manifestations/; the distinction is metadata only.
Why joint matters for papers:
- Paper 6 — convergent evolution arguments are fundamentally about composed systems. The git ecosystem aggregate (Finding 7) is a primitive joint operation; full joint Mn composition generalizes this.
- Paper 9 (application architectures) — designing a real application means composing constituents; the framework should support this design move.
- Cross-arrangement applicability — a multicellular organism is a joint Mn of its tissue types; a research ecosystem is a joint Mn of research methods + institutions + funding structures.
2.4 Metadata fields — uniform data model with filterable roles
Per §2.2/§2.3 framing: everything is a manifestation, distinguished by metadata. The schema gets a small set of optional fields that let analyses filter by role and trace provenance.
Proposed additions to manifestation.v1.schema.json:
| Field | Type | Purpose | Default |
|---|---|---|---|
corpus_role | enum: instance | anchor | composition | exemplar | What role this Mn plays in analyses; analyses default to including instances and excluding anchors unless explicitly requested | instance |
origin | enum: direct_authoring | declarative_population | inductive_clustering | composed_from_constituents | How the Mn came to be authored, for provenance | direct_authoring |
comparison_inclusion | enum: default_include | anchor_only | opt_in | Fine-grained inclusion control for landscape analyses (e.g., archetype Mns set to anchor_only won't appear in normal landscape plots) | default_include |
exemplars | array of Mn refs (optional) | For anchor-Mns: list of Sc=3 Mns that exemplify this region | omitted |
composed_of | array of Mn refs (optional) | For joint Mns: constituents that compose into this Mn | omitted |
composition_rule | enum (optional): element_wise_max | element_wise_sum_capped | coupling_aware | custom | The rule used to compute joint position | omitted |
derived_from_cluster | object (optional): { analysis_run, cluster_id, cluster_spread_stats } | Provenance for inductively-derived archetypes | omitted |
Filtering behavior in compute scripts:
- Default: include all
corpus_role: instanceMns; excludeanchorandcompositionMns --include-anchorsflag: include anchor-Mns alongside instances (for archetype-overlay visualizations)--anchors-onlyflag: only the anchor set (for archetype-vs-archetype comparison)--include-compositionsflag: include joint Mns (for analyses where composed systems are subjects)
This means landscape analyses, fingerprint comparisons, and trajectory work all keep their existing default behavior. Archetype-aware analyses opt in explicitly.
The Sc field already does much of this work. Most landscape analyses already filter to Sc=3 unified instances; Sc=2 range-Mns are excluded by default. The new metadata adds fine-grained control on top of Sc-based filtering — useful when an Sc=3 joint Mn (composed) should be distinguished from an Sc=3 direct-authored Mn.
3. Cross-arrangement applicability
Each concept above applies to all five arrangements (and the abiogenesis fragment), but the content of archetypes + composition rules varies:
3.1 Entity arrangement
- Multi-lens comparison: applies as outlined
- Archetypes: event-streaming, vcs, p2p, reactive-compute, social-platform, infrastructure (~6-8 archetypes for 34 Mns)
- Joint Mns: composition over
couples_with+ bridge integration. Surface-vs-internals dual encoding for high-value systems.
3.2 Biology arrangement
- Multi-lens: same lens infrastructure; primitive sets differ (G, T, R, P, Reg, Mem at substrate)
- Archetypes: body-plan archetypes (chordate, arthropod, cnidarian, plant, fungus, prokaryote) likely the right granularity. Could also be ecological-niche archetypes (photoautotroph, decomposer, predator).
- Joint Mns: multicellular organism = joint of tissue types + organ systems; ecosystem = joint of populations. Plenty of biology cases for testing the composition rule.
3.3 Cognition arrangement
- Multi-lens: per-organism-kind heatmaps + cultural-artifact heatmaps as separate facet groups (per Rule 8 Mn-kind discipline)
- Archetypes: organism-individual, social-band, cultural-ecosystem, civilization, research-ecosystem already exist as authored Mns — these are partial archetypes. Inductive clustering should refine them and surface new ones (e.g., is there a "primate-troop-cognition" archetype distinct from "rodent-colony"?)
- Joint Mns: distributed cognition examples — "a research team" = joint of individual cognitions + cultural-artifact (the lab's shared knowledge structure)
3.4 Methodology arrangement
- Multi-lens: methodology spread is the widest (Finding 13, 0.52) — clustering should reveal sub-groups
- Archetypes: qualitative-interpretive (ethnography, phenomenology), quantitative-formal (statistical-inference, math-proof), iterative-design (design-thinking, lean-canvas), strategic-frame (SWOT, OKRs), systemic (cybernetics, complexity)
- Joint Mns: methodological pluralism — "ethnography + statistical-inference" applied jointly to a problem. The composition might be informative about what real research practice does.
3.5 Abiogenesis arrangement
- Smaller scope (3 chain levels populated); fewer Mns
- Archetypes: pre-life chemistry → proto-replication → proto-compartmentalization stages already authored as trajectory positions. Could surface as 3-4 archetype regions.
- Joint Mns: probably not the most productive arrangement for joint Mn work; better to focus on the other four.
4. Implementation roadmap
Ordered by sequential dependency (later items build on earlier).
Phase 1 — Multi-lens compute (1-2 sessions)
- Fix instagram-client: author its a2e lightly so it's not a clustering artifact. Mark
mn_kind: client-instancefor future filtering, but include in standard analyses. - Lens A (per-primitive heatmap):
compute/scripts/plot_per_primitive_heatmap.py— rows = Mns, columns = primitives at one chain level, sort options (alphabetical, by primitive rank, by similarity-to-target-Mn). - Lens C (weighted distance): extend
plot_structural_fingerprint_landscape.py(or new script) with--weightsflag accepting primitive weight dict, OR--profile <name>for saved weight profiles indata/comparison_profiles/. - Lens E (classical clustering):
compute/scripts/cluster_classical.py— k-means, hierarchical, DBSCAN; one CLI argument for method choice. Output: cluster assignments + dendrogram + silhouette score.
Phase 2 — Archetype methodology (2-3 sessions)
- Schema metadata extension: add the §2.4 fields (
corpus_role,origin,comparison_inclusion,exemplars,composed_of,composition_rule,derived_from_cluster) tomanifestation.v1.schema.jsonas optional fields. No new schema file. All existing Mns remain valid (fields default in). - Tag existing population-Mns: set
corpus_role: anchor,origin: declarative_population,comparison_inclusion: anchor_onlyoncontent-addressed-data-systemsandinteractive-web-platforms. Addexemplarslists for each. No migration; just metadata. - Inductive archetype identification pipeline:
compute/scripts/identify_archetypes.py— runs clustering with multiple methods, surfaces clusters with N≥3 and interpretable shape, outputs candidate archetype Mns (Sc=2, range-encoded) for human review. The output is a draftdata/manifestations/<archetype-name>.v1.jsonfile withorigin: inductive_clusteringandderived_from_clustermetadata populated. - Author archetypes for entity arrangement first (~6-8 candidates), then biology body-plans, then cognition kinds, then methodology epistemic-styles. Each archetype is just a new Sc=2 Mn in
data/manifestations/with appropriate metadata. - Lens B (archetype scoring):
compute/scripts/score_archetypes.py— for each Mn × anchor-Mn, compute membership score (cosine to anchor centroid/modal, weighted by range-coverage). Default includes onlycorpus_role: anchorMns as comparison targets. - Update standard compute scripts to respect new metadata: default behavior filters
corpus_role: anchorandcompositionout of landscape/fingerprint analyses unless--include-anchors/--include-compositionsis passed.
Phase 3 — Joint manifestation composition (2-3 sessions)
- Composition rule library:
compute/lib/joint_composition.py— element-wise max + sum-capped + coupling-aware + coherence validation. Multiple composition modes selected viacomposition_rulefield. - Schema metadata for joint Mns: already added in Phase 2 (
composed_of,composition_rule,corpus_role: composition,origin: composed_from_constituents). No further schema work. - Pilot composition cases: postgres + redis + http-rest (web app); slack-internals = joint(postgres-like, redis-like, websocket-like); cognition: research-team = joint(individual-cognition × N, cultural-artifact). Each pilot produces a new Sc=3 Mn in
data/manifestations/withcorpus_role: compositionmetadata. Compare each joint to nearest surface Mn using Lens A + Lens B. - Surface-vs-internals consistency check: for systems with both a surface Mn AND a joint-internals Mn, compute the distance and flag inconsistencies. Two Mns with different
corpus_role(instancevscomposition) but the same underlying system are expected to be CLOSE structurally; large divergence is a finding.
Phase 4 — Cross-arrangement extension (1-2 sessions per arrangement)
- Apply Phases 1-3 outputs to biology, cognition, methodology
- Author arrangement-specific archetypes (body plans, cognition kinds, epistemic styles)
- Test joint Mn composition rules in each arrangement
- Surface cross-arrangement archetype isomorphisms — does "event-streaming-substrate" in entity have a structural twin in biology or cognition? (Finding 19 already shows cross-arrangement bridge isomorphism; this would extend to substrate + ecosystem level.)
Phase 5 — Methodology document refresh (1 session)
canonical-architecture-strategy.mdgains new sections:- §2.5 (Mn-kind discipline) is already in place; the new metadata (
corpus_role,origin,comparison_inclusion) extends the same uniform-data-model principle. Note that anchor/composition Mns share the schema with instance Mns; what differs is metadata + intent of use. - §2.6 (Comparison machinery) — new — multi-lens framework, anchor-mediated comparison, default filtering behavior
- §3 Rule 9 — "compare via anchor-Mns first, full corpus second"
- §3 Rule 10 — "joint Mns must validate coherence and declare composition rule"
- §4 use-case expansion for cross-arrangement anchor/composition work
- §2.5 (Mn-kind discipline) is already in place; the new metadata (
framework-synthesis.md§5 (architecture as developed) extended with the comparison/anchor/joint layermethodology.md(the source doc) referenced for the anchor / canonical-region concept — likely already touches this in the L3 abstraction section; needs alignment
5. Decisions (locked)
5.1 Anchor-Mn granularity — empirical cluster-spread rule
For an inductively-derived anchor-Mn:
- Eligibility: cluster must have N ≥ 3 member Mns. Smaller groupings are exemplar-pairs, not regions, and don't get anchor status.
- Range bounds: per primitive,
min= empirical minimum across cluster members;max= empirical maximum. No padding. The actual observed spread IS the range. - Modal value: per primitive,
modal= median across cluster members (mode if categorical-like). - Tightness signal: if all members fall within 1 level on a primitive, encode the range as
{min: M, max: M, modal: M}(degenerate range — that primitive is the cluster's signature). If spread is ≥ 2 levels, encode as proper range. Both forms valid.
For declarative anchor-Mns (existing population-Mns like content-addressed-data-systems): keep existing range encoding; no rule change.
5.2 Composition rule defaults — saturating-max with explicit additive opt-in
- Default rule for all primitives: element-wise max. Reflects saturating semantics ("level of capability provided") which fits ~all of the framework's primitives. If A=4 and B=2 on the same primitive, joint=4 (A's contribution sets the level).
- Additive rule (
sum_capped_at_max): only used when a primitive is explicitly tagged in its domain/bridge schema withcomposition_mode: additive. No primitive is currently tagged. We add the field if/when a real composition case needs it — not pre-emptively. - Coupling-aware rule: deferred to Phase 3 refinement. When element-wise max produces a joint Mn that doesn't capture the coupling structure between constituents (e.g., postgres+redis joint should have higher Cnt at e2a-bridge than either alone, reflecting the merge), we'll introduce coupling-aware logic — but only after the max baseline shows where it's insufficient.
- Per-joint declaration: every joint Mn declares its
composition_ruleexplicitly in metadata, so any non-default rule is transparent.
5.3 Inductive vs declarative reconciliation — declarative wins on name; range merges
When inductive clustering surfaces a candidate that overlaps with an existing declarative anchor-Mn:
- Existing anchor keeps its name.
- Range bounds update to the union of declarative bounds and empirical cluster spread (declarative is intent; cluster is realized data; union captures both).
exemplarslist extends to include the cluster members.origintag becomesdeclarative_population(unchanged) but addsrefined_from_cluster: <analysis_run_id>inderived_from_clustermetadata for provenance.- No new file; existing Mn file gets updated.
5.4 Cross-arrangement isomorphism encoding — isomorphic_to field, deferred
Add an optional isomorphic_to: [{mn_ref, distance, chain_level}] field to the manifestation schema. Applicable to any Mn (anchor or instance). Populated when cross-arrangement structural twins are verified empirically — e.g., from Finding 19's bridge-fingerprint analysis (biology-to-organism ↔ neural-to-cognitive d=0.237).
Concrete encoding deferred: do not add the field to the schema until we have at least one cross-arrangement isomorphism worth encoding. Add it during Phase 4 as cross-arrangement work surfaces real matches.
5.5 Scaling — confirmed manageable
~6-8 anchor-Mns per arrangement × 5 arrangements ≈ 30-40 anchors total. New Mn comparison becomes O(anchors) ≈ 30 instead of O(corpus) ≈ 147. ~5× reduction in comparison cost and significantly more interpretable output.
5.6 Paper-validation gate — Phase 2 doesn't scale past entity until entity validates
After Phase 2 produces entity-arrangement anchors, validate that:
- (a) Paper 6's convergent-evolution argument can be tightened by referring to specific anchor-Mns rather than hand-waving "these systems are structurally similar"
- (b) Paper 0's six-primitives framing benefits from explicit anchor-Mns showing realized regions of the combinatoric space
- (c) Anchor-mediated Mn comparison produces interpretable signal (not just compression for compression's sake)
If (a)+(b)+(c) check out, scale to biology/cognition/methodology. If not, revisit Phase 2 design before extending.
5.7 Metadata vocabulary clarifications
mn_kind(existing, per Rule 8) captures content type (entity-application, cultural-artifact, organism-cognition, etc.). Unchanged.corpus_role(new, §2.4) captures analytical role (instance, anchor, composition, exemplar). Orthogonal tomn_kind. An anchor can have anymn_kind; a joint Mn can have anymn_kind.- Anchor-Mns do NOT get a special name prefix. Descriptive names only (e.g.,
event-streaming-substrate.v1.json, notanchor-event-streaming-substrate.v1.json). Thecorpus_role: anchortag discriminates. exemplarsfield stores Mn refs as file paths (data/manifestations/<name>.v1.json), matching existing reference convention.
5.8 Schema version handling
The §2.4 metadata additions are all optional fields on manifestation.v1.schema.json. Existing 147 Mns remain schema-valid without updates. No v2 bump needed; add fields and re-validate everything.
6. What this changes for ongoing work
- Paper 6 body integration (currently the largest open gap) gains a load-bearing analytical move: archetype-comparison as the basis for convergent-evolution claims. Significantly stronger than the current "structurally similar" hand-waving.
- Paper 11 (structural methodology) gains its central methodological argument: comparison-via-archetype is the framework's analytical contribution.
- Paper 0 (the six primitives) gains the right framing for the primitive-combinatorics: archetypes are the realized regions of the combinatoric space; not all combinations are equally populated.
- All paper-shipping work waits on Phase 1-2 of this plan, because the analytical machinery currently in place is insufficient for the claims the papers need to make.
This is genuinely the priority. The backlog items (per-figure L4 captions, dirac-bridge authoring) remain low-priority.
7. Out of scope for this plan (deliberately)
- Refining the methodology source doc (
methodology.md) — should follow this plan, not lead it - Lean formalization of archetype concept (a future direction; not blocking)
- Migration of existing Findings to archetype-based framing (after Phase 2 is done)
- Visual design of paper figures using new infrastructure (after Phase 1-3 complete)
8. Session-start checklist for the next sitting
When picking up next session, work through this in order. Each item is concretely scoped; no further design decisions needed.
Phase 1A — Foundation (single session, ~half-day)
-
Author instagram-client's
app-to-ecosystem-bridgeto fix the n=1 clustering artifact. Light values reflecting client-instance ecosystem participation (deployed via app stores, versioned releases, etc.). -
Add §2.4 metadata fields to
data/manifestations/manifestation.v1.schema.json:corpus_role,origin,comparison_inclusion,exemplars,composed_of,composition_rule,derived_from_cluster. All optional. Runmake validateto confirm existing Mns remain valid. -
Tag the two existing declarative anchors:
content-addressed-data-systems.v1.json—corpus_role: anchor,origin: declarative_population,comparison_inclusion: anchor_only, populateexemplarsfrom existing Sc=3 content-addressed Mns (git, ipfs, docker, datomic, etc.)interactive-web-platforms.v1.json— same pattern, exemplars include the social-platform-app candidates
Phase 1B — Lenses (subsequent session)
-
Lens A —
compute/scripts/plot_per_primitive_heatmap.py. CLI:--arrangement <name> --chain-level <name> [--sort-by similarity|rank|alphabetical] [--include-anchors]. Output: heatmap PNG + JSON of values. Default excludes anchors. -
Lens C — extend
compute/scripts/plot_structural_fingerprint_landscape.pywith--weights <primitive>=<w>,...flag, OR--profile <name>reading saved weight profiles fromdata/comparison_profiles/<name>.v1.json. Recompute distances under weighting. -
Lens E —
compute/scripts/cluster_classical.py. CLI:--method {kmeans,hierarchical,dbscan} --k <n>(for kmeans) or--eps <d> --min-samples <n>(for dbscan). Output: cluster assignments + dendrogram + silhouette score. Respectscorpus_rolefiltering. -
Lens G (deferred to Phase 1D or Phase 2) —
compute/scripts/compute_pair_signature.py. Readspair_relationshipsfrom arrangement/domain/bridge JSONs; computes per-Mn pair-density vector; outputs JSON suitable as input to Lens A/E. Skip in Phase 1B if Lens A/C/E results from raw-primitive signatures already surface the structure we need; add when raw-primitive view hits a ceiling.
Phase 1C — Analysis run + first anchors (subsequent session)
- Run Lens A on entity-system + entity-to-app-bridge + application-architecture + app-to-ecosystem-bridge separately. Visually identify candidate anchor regions.
- Run Lens E (all three methods) on the same chain levels. Compare cluster assignments.
-
Manual: synthesize 6-8 candidate entity-arrangement anchors per §2.2's example table. Each becomes a draft Sc=2 Mn in
data/manifestations/withorigin: inductive_clustering. - Bound-check + coherence-check each anchor candidate. Iterate ranges if needed.
Phase 2 — Archetype machinery (subsequent sessions)
After Phase 1 anchors exist:
-
Lens B —
compute/scripts/score_archetypes.py. For each Mn × anchor, compute membership score (cosine-to-modal, range-coverage-weighted). Output: per-Mn anchor profile. -
compute/scripts/identify_archetypes.py— automate the cluster-to-anchor pipeline (clustering + N≥3 filter + range derivation per §5.1 + draft Mn file emission). - Paper validation gate (§5.6) — verify entity anchors strengthen Paper 6 + Paper 0 before scaling to other arrangements.
Phase 3 — Joint composition (parallel to or after Phase 2)
-
compute/lib/joint_composition.py— element-wise max as default; sum-capped opt-in; coherence validation after composition. -
Pilot composition:
joint(postgres, redis, http-rest)→ Sc=3 joint Mn withcorpus_role: composition. Compare to nearest surface Mn. - Surface-vs-internals consistency check for at least one system that has both views.
Phases 4-5 — Cross-arrangement extension + strategy doc refresh
Per existing §4 of this plan. Scheduled after Phase 1+2 entity work validates.
Approximate time-cost: Phase 1A = ~half-day; Phase 1B = 1 session; Phase 1C = 1 session; Phase 2 = 2-3 sessions; Phase 3 = 2-3 sessions; cross-arrangement + strategy refresh = additional sessions per arrangement.
Referenced by the model
Cited as a source by 8 model records (browse the model census):
- commercial-saas-platform —
manifestationentity/sc2 - consensus-kv-substrate —
manifestationentity/sc2 - content-infrastructure —
manifestationentity/sc2 - editor-tools-pair —
manifestationentity/sc2 - os-substrate —
manifestationentity/sc2 - peer-federation-messaging —
manifestationentity/sc2 - relational-server-dbms —
manifestationentity/sc2 - vcs-system —
manifestationentity/sc2