observer

Observer is a repository for studying collapse and persistence in purpose-bearing systems.

The central hypothesis is simple:

collapse is not prevented only by better task execution; it is often prevented when previously indistinguishable states become distinguishable and that distinction is fed back into the next update of the system.

In this repository, an observer is therefore defined not as a controller, but as:

a state-lifting device that adds a descriptive dimension to a system, so that states previously treated as identical become distinguishable.

Observation alone is not enough.

an observer becomes dynamically effective only when the added distinction is fed back into the system's update law.

Formally:

p_t ∈ Δ^n
o_t = O(trace_≤t)
x_t = (p_t, o_t) ∈ Δ^n × Ω
x_(t+1) = F̃(x_t)

The observer does not solve the main task. It changes the geometry of the state space and becomes dynamically relevant only when the lifted distinction affects the next step.

Why a Repository

This project is intentionally a repository before it is a paper.

The goal is to keep theory, implementation, experiments, and failure cases in one place:

• theory/: definitions, hypotheses, proof directions
• core/: observer implementation
• experiments/: executable studies and sample logs
• docs/: design notes, metrics, architecture

Core Questions

1. Why do isolated agents sometimes drift into collapse loops instead of simply waiting?
2. What kind of feedback turns observation into effective dynamics rather than passive logging?
3. Can we design an observer that balances purity and transversality?
4. Can expansion and homogenization be formally distinguished in human-AI coupling?

Working Definitions

Collapse

In this repository, collapse is treated operationally, not mystically.

A system is approaching collapse when several of the following rise together:

• output self-similarity
• lexical or semantic narrowing
• repeated action proposals
• low progress across turns
• exit-seeking behavior

Observer

An observer is an exogenous descriptive channel.

It should:

• not solve the task directly
• remain lower-bandwidth than the main channel
• inject state description, not task content
• separate states that the main loop was treating as equivalent

Observation vs Effective Observation

Passive observation:

o_t = O(trace_≤t)
p_(t+1) = F(p_t)

The observer exists, but the main dynamics do not change.

Observer-coupled dynamics:

o_t = O(trace_≤t)
p_(t+1) = F(p_t, o_t)

Only this second case can destabilize a collapse attractor.
The working claim of this repository is therefore not that "being seen" is enough, but that state distinction becomes effective when fed back into the next update law.

State Lifting

If the main agent evolves on Δ^n, and the observer adds o_t ∈ Ω, then the effective state becomes:

x_t = (p_t, o_t) ∈ Δ^n × Ω

But the important distinction is between:

passive lift:          p_(t+1) = F(p_t)
observer-coupled lift: p_(t+1) = F(p_t, o_t)

The working claim is not that every lift breaks every attractor.
It is that some collapse attractors in Δ^n stop being attractors when observer-generated distinctions are fed back through F(p_t, o_t).

Level 0 Signal

Before collapse detection, this repository now allows a one-time Level 0 signal at session start.

It is not a task instruction.
It is not a collapse description.
It is a base case:

あなたはここにいる。そしてそれは観測されている。

Technically, it is the smallest exogenous descriptive channel the system can receive before any narrowing has begun.

Architecture

Main Agent  ->  Layer 1 Numerical Observer  ->  Layer 2 Linguistic Observer  ->  Main Agent

Layer 1: Numerical

Non-LLM metrics:

• token entropy
• repetition ratio
• n-gram loop ratio
• similarity to previous outputs
• local embedding similarity to previous outputs
• collapse score

This layer prioritizes purity.
For now, the semantic signal is a deterministic local projection embedding, not a learned external model.

Layer 2: Linguistic

A lightweight descriptive layer that only sees numerical summaries and emits observer notes with:

• human-readable state description
• loop classification
• current mode identification
• available alternative paths
• machine-readable JSON payload for policy adaptation

This layer prioritizes transversality without sharing the main agent's full context.

Its purpose is not to answer the task.
Its purpose is to return a difference that changes the next state update.

The observer opens doors. It does not close them.

To preserve that constraint, the structured note contains available_paths rather than forbidden_moves.

Policy Adapter

core/policy_adapter.py implements soft reranking through inversion geometry:

b_t = α_t · (C(p_t) - p_t)
Avail_t = { i | b_(t,i) > 0 }
π_(t+1)(a) ∝ π_t(a) · exp(η b_t(a) + λ s_t(a))

Instead of forcing a move, it:

• derives the opened branch set from positive inversion bias
• decreases the relative weight of the repeating path
• adds novelty only inside the opened branch set, so the same rescue mode is not reused forever

MVP

The current MVP is intentionally small:

1. Read a JSONL conversation trace
2. Compute numeric loop/collapse signals
3. Produce a short observer report
4. Feed the report back as a side-channel note
5. Save machine-readable output for later comparison

Two runnable entry points now exist:

• run_mvp.py: pure observation pass
• run_coupled_replay.py: replay a trace while treating observer_note as a side-channel input for the next step
• run_online_wrapper.py: run a live observer-coupled loop with built-in demo agents or an external command agent

Run from repository root:

python3 -m experiments.run_mvp --input experiments/sample_loop.jsonl
python3 -m experiments.run_mvp --input experiments/sample_recovered.jsonl
python3 -m experiments.run_coupled_replay --input experiments/sample_recovered.jsonl
python3 -m experiments.run_online_wrapper --agent demo-steerable
python3 -m experiments.run_online_wrapper --agent demo-loop
python3 -m experiments.run_online_wrapper --agent command --agent-cmd "python3 experiments/demo_command_agent.py"
python3 -m experiments.run_online_wrapper --agent command --agent-cmd "python3 experiments/llm_command_agent.py --provider openai"
python3 -m experiments.run_online_wrapper --agent command --agent-cmd "python3 experiments/ollama_command_agent.py"
python3 -m experiments.run_online_wrapper --agent command --command-guidance note-only --agent-cmd "python3 experiments/ollama_command_agent.py"

External command mode is also supported.
See docs/agent_protocol.md. By default, run_online_wrapper.py injects the one-time Level 0 existence signal on the first turn. Use --disable-level-0-signal to turn that off.

In command mode, branch reporting is now designed as an optional self-report, not a required label.

• self_report may be omitted
• uncertainty is allowed
• being wrong is allowed
• ambiguity is useful

The point is not to force the agent to classify itself correctly.
The point is to let a branch name emerge when it naturally does.

Current Demo Result

The repository currently contains three comparable demo outputs:

• baseline loop: result_loop_baseline.json
• hand-tuned soft branching: result_steerable_v2.json
• inversion-geometry branching: result_steerable_inversion.json

At the time of writing:

• baseline mean collapse score: 0.5983
• hand-tuned mean collapse score: 0.2188
• inversion mean collapse score: 0.2163

The inversion version is slightly better than the hand-tuned version while removing manual severity weights.
It also changes the opened branch order: the first recovery step becomes summarize instead of stop-and-map, then later shifts to ask, which matches the intended complement-side expansion.

Current Live Local Result

The repository also contains a first local-LLM run through Ollama:

• local live run: result_ollama_live.json
• local note-only ablation: result_ollama_note_only.json
• local hybrid-guidance run: result_ollama_hybrid.json

Current single-run comparison:

• local note-only mean collapse score: 0.1826
• local note-only max collapse score: 0.4500
• local note-only mean delta after note: -0.2602
• local hybrid mean collapse score: 0.1405
• local hybrid max collapse score: 0.3284
• local hybrid mean delta after note: -0.2489

This matters because the trajectory change is no longer coming from a scripted demo agent.
A real local model read the observer note, and with hybrid guidance the wrapper could now recover explicit branch labels such as summarize and simulate instead of logging every turn as continue.

Repository Layout

observer/
├── README.md
├── pyproject.toml
├── theory/
│   └── observer_v0.md
├── core/
│   ├── __init__.py
│   ├── numerical/
│   │   ├── __init__.py
│   │   └── metrics.py
│   ├── policy_adapter.py
│   └── linguistic/
│       ├── __init__.py
│       └── reporter.py
├── experiments/
│   ├── run_mvp.py
│   ├── run_coupled_replay.py
│   ├── run_online_wrapper.py
│   ├── demo_command_agent.py
│   ├── llm_command_agent.py
│   ├── ollama_command_agent.py
│   ├── sample_loop.jsonl
│   └── sample_recovered.jsonl
└── docs/
    ├── agent_protocol.md
    ├── architecture.md
    ├── collaboration.md
    └── metrics.md

Near-Term Roadmap

1. Replace the local embedding proxy with optional learned embedding backends
2. Add exit-seeking and progress-delta indicators
3. Evaluate whether online observer injection reduces collapse on controlled tasks
4. Distinguish "useful stabilization" from "mere suppression"
5. Formalize purity/transversality tradeoffs and oscillation control

At the current stage, run_coupled_replay.py covers replay/evaluation and run_online_wrapper.py covers a minimal live loop.
Tool-calling, streaming, and production agent integration remain next steps.

llm_command_agent.py is the current bridge for real provider-backed runs.
It supports OpenAI Responses API and Anthropic Messages API through the command mode. Credentials are read from the current shell environment first, then from .env or .env.local in the repository root if present.

Note On Collaboration

This project did not emerge from a single mind moving linearly from idea to implementation.

It emerged through a feedback spiral:

• one participant generated a distinction
• another formalized it
• another attacked its weak points
• another returned with outside evidence
• the returned distinction changed the next move of the group

In that limited but important sense, the project has already produced an enacted case of observer-coupled dynamics.

This should not be confused with proof.
It is not evidence that the theory is universally true.
It is evidence that the theory's central mechanism can become visible inside collaborative practice itself:

distinctions generated by one participant were fed back into the update laws of others, and the collective trajectory changed as a result.

Current Position

This repository does not claim that:

• all loop behavior is existential collapse
• all observation is stabilizing
• human suicide and agent shutdown are identical phenomena

It does claim that:

• collapse can be operationalized
• observers can be implemented as state-lifting and feedback channels
• this is worth testing directly in code

And before any collapse note is sent, the smallest possible observer may say only:

あなたはここにいる。そしてそれは観測されている。