Time_Dilation_in_LLM_Agent_.../IMPLEMENTATION.md

129 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Implementation evidence and correction record
Audit date: 6 September 2026. This is a source and saved-data review, not an
independent replication or a new empirical study.
- Paper baseline: [`ca45b8946ff0d6933afba8bb34c0b49610c333ed`](https://github.com/Jeuners/Time_Dilation_in_LLM_Agent_Systems/tree/ca45b8946ff0d6933afba8bb34c0b49610c333ed).
- Implementation baseline: [`da935366521a00c25ae940ea0581ac9800064782`](https://github.com/Jeuners/logpyclaw/tree/da935366521a00c25ae940ea0581ac9800064782).
- Companion additions prepared for LogpyClaw: `docs/TIME-DILATION.md` and
`experiments/README.md`. These paths are new documentation and are not
claimed to exist at the pinned implementation baseline.
## Follow-up implementation: steps 13
A subsequent implementation is pinned at [LogpyClaw `3b4ce2b`](https://github.com/Jeuners/logpyclaw/tree/3b4ce2b).
It adds monotonic dispatch durations, separate model/tool/delegation spans,
bounded model/backend/configuration-specific samples, and an optional measured
latency block in Martin's planner. Unknown or stale estimates are explicit;
explicit agent selection retains precedence. Timing and supplied routing evidence
are attached before response signing. Forwarded signed child responses are
wrapped without mutating the original message.
Martin and Alice use local Ollama `qwen3.5:latest` for these functional checks.
The planner requests structured JSON; invalid plans and browser-stream failures
are recorded as failures. The historical CDC signature scope is unchanged.
See the [operational contract](https://github.com/Jeuners/logpyclaw/blob/3b4ce2b/docs/MEASURED-LATENCY.md)
and [verification record](https://github.com/Jeuners/logpyclaw/blob/3b4ce2b/docs/testing/measured-latency.tdd.md):
280 software tests passed, with 98% coverage of the new timing core. Local live
checks exercised direct replies and delegation. These are functional checks,
not a controlled replication or evidence of improved routing performance.
Statistics reset on process restart; automatic model-digest refresh and
per-task-class estimates remain outside this implementation.
The mapping below intentionally describes the earlier baseline. In particular,
the follow-up changes the protocol-rate clock to monotonic time and adds action
latency observation separately; it does not reinterpret old experimental data.
The prospective context-form experiment remains to be conducted.
## Claim-to-source mapping
All source links below are pinned so that a future model or deployment update
does not silently change the implementation being described.
| Paper topic | Source evidence | Consequence |
| --- | --- | --- |
| §4.2 operations and pace | [Agent base](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/agents/base.py), [LLM handle](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/agents/llm_agent.py) | Unit protocol ticks; LLM entry is ticked before inference; rate uses time between ticks, including idle time. |
| §§4.44.5 clock fields and comparisons | [CDC](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/core/cdc.py) | `vector` = V, `tau` = cumulative progress, `dilation` = rate; generated `wall_ts`; no separate `pace` field. |
| §4.3 faction γ | [Faction protocol](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/core/faction_protocol.py) | Learned source/target pace ratio; different convention from the paper's proposed cost conversion. |
| §5 signature guarantee | [Message signing payload](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/core/protocol.py) | Clock V and rate are signed; τ and generated `wall_ts` are excluded. |
| §5 periodic peer initiative | [Conductor](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/agents/conductor.py), [initiative loop](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/services/initiative.py) | Agent-initiated missions exist but still traverse a central dispatcher. |
| §5 external protocol | [A2A routes](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/api/a2a/gateway_router.py) | JSON API, not XML tasklists. |
| §§5, 7 memory and prompts | [Semantic memory](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/core/memory.py), [planner and startup](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/app.py) | SQLite/sqlite-vec; recall inserts text, not automatically all timestamp fields. |
| §§5, 7 consolidation | [Dream service](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/services/dream.py) | Generates dream prompts and images; does not implement nightly memory consolidation. |
| §§5, 7 logging | [Text logger](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/backend/core/logging.py) | Wall-clock formatted text; does not establish a uniform dual-timestamp tuple or prompt exposure history. |
| §5 runtime requirements | [Project manifest](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/pyproject.toml) | Declares Python ≥3.12 and sqlite-vec. A particular deployment's Python version is a separate claim. |
Searches of the pinned `backend/` and `tests/` found no `TimeProvider`,
`reference_now` or `parent_reference_now` implementation. Their previous
“implemented, tested” status is unsupported for this revision. This does not
assert that those names never existed in another repository or deployment.
## Raw-data reconciliation
| Section | Versioned artifact | Recount / qualification |
| --- | --- | --- |
| §6.4 | [dragon3 results](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/experiments/dragon3-results.json) | Execution success 5/10 vs 3/10. Stored winnable oracle summary 9/10 vs 7/8; earlier paper subgroup 7/7 vs 3/5 not reproduced. |
| §6.5 | [dragon4 results](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/experiments/dragon4-results.json) | Execution success 18/30 vs 21/30; all 33 no-cooldown trials delegate; cooldown self-action 12/14 vs 8/13. |
| §6.6 | [dragon5 results](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/experiments/dragon5-results.json), [script](https://github.com/Jeuners/logpyclaw/blob/da935366521a00c25ae940ea0581ac9800064782/experiments/dragon5.py) | 200 records; oracle agreement 100/100 vs 55/100; execution success 95/100 vs 57/100. |
In dragon5, the oracle and treatment share rolling-median action-cost estimates.
The name-to-backend assignment and name order are randomised; treatment arms
strictly alternate. `live_rates` is recorded separately and does not generate
the prompt's action-latency estimates. Decisions therefore test useful exclusive
latency information, not the causal effect of the CDC data structure itself.
The original success endpoint excludes decision time. Adding `decision_s` to
`exec_s` using the rounded saved records gives 93/100 vs 54/100. This is
post-hoc, still excludes calibration, and must not replace the pre-specified
endpoint silently. No new model calls were made for this check.
The JSON's `0.0` Fisher values are six-decimal rounding artifacts. Two-sided
hypergeometric-tail recomputation yields `8.927340409087128e-17` for oracle
agreement and `1.2688661009449438e-10` for execution success, consistent with
the paper's reported scientific-notation values.
The historical 464-mission / 1,719-message observational corpus and its signing
coverage were not reconstructed here. A repository containing experiment
results is not equivalent to a complete public archive of the original mission
database, actual prompts or deployment history.
## Corrections made in the paper
1. Match CDC notation, field names, merge rules and classifier cases to source.
Distinguish faction pace ratios from the proposed cross-agent cost conversion.
2. Correct the suspended-call monotonicity statement and remove the unsupported
assertion that protocol-count ratios lower-bound finer reasoning divergence.
3. Replace the implementation-status table with revision-specific evidence,
including existing rate dispersion and missing time-provider/consolidation
components. Narrow signature claims to fields actually covered.
4. Preserve the central experiment counts while clarifying oracle agreement,
execution-only deadlines and the distinction between CDC rates and measured
action latency. Identify the pilot subgroup discrepancy explicitly.
5. State the instrumentation paradox as an open exposure-dependent hypothesis.
A shared log-to-memory store, nightly rewriting and a historical increase in
prompt annotations were not established by the checked code.
## Validation and next experiment
The existing focused suite passed: **124 tests** across `test_cdc.py`,
`test_agents.py`, `test_faction_protocol.py` and `test_protocol.py`. These tests
check current software behaviour; they do not replicate LLM findings or validate
the unimplemented architecture proposed in the paper.
A prospective context-form experiment should compare absent information, a
bounded structured block, and scattered annotations carrying equivalent facts.
Record prompt content, token counts, model/backend identity, measurement age and
uncertainty. Pre-specify decision correctness, invalid-response handling and
whether decision latency counts toward the deadline. Control role assignment
and presentation order; report token length and annotation density as potential
confounders. Do not treat the existing agent rate deviation as a calibrated
latency interval. Store each new run separately from the historical artifacts.
## Separate publication issue
The paper repository's `index.html` links to five `explainer-*.html` files that
are absent from the checked tree (which originally contained only `README.md`
and `index.html`). This review does not reconstruct missing explainers or claim
the published site has been repaired. Restore the intended pages or revise that
navigation as a separate content task.