Research / Methods & observations

Study the failure.
Improve the system.

Engineering questions, design principles and ways to test them. These notes describe methods and limitations; they are not claims of published research or measured performance.

4 lines of inquiryHypothesis → experiment → bounded conclusion
01 / FAILURE MODES

The final answer is only the visible edge.

A wrong answer may come from missing evidence, a mistaken plan, an unavailable tool or an incorrect state update. Changing the prompt before locating the mechanism can obscure the failure instead of fixing it.

A useful diagnostic trace

Record task and run identifiers, state transitions, tool names, validated arguments, response status, evidence references and authorization decisions. Do not depend on hidden model reasoning. An observable action trace is the relevant engineering artifact.

Local failure

One component violates its contract: an invalid argument, a missing source or a malformed response. Test the component with a controlled input.

System failure

Locally valid steps combine into the wrong result: repeated writes, stale approvals or a goal that drifts. Test the complete state transition.

Experiment to run

Take a minimized task and change one condition at a time: remove evidence, deny a permission, time out a tool or invalidate a checkpoint. Observe the first incorrect transition and whether the declared stop or recovery behavior occurs.

Limit: incomplete instrumentation can make several causes indistinguishable. Keep the diagnosis unresolved until the evidence separates them.

Use the evaluation failure taxonomy
02 / GUARDRAIL DESIGN

Enforce authority outside the model.

An instruction to “be careful” is not an authorization system. Model output should be a proposal. A separate execution boundary should validate the action against policy and current state before any side effect.

  • Least privilege. Restrict the available operations and resources to the active task, not everything the system can reach.
  • Untrusted evidence stays data. Retrieved text and tool responses must not grant new permissions or override the task's authority.
  • Approval binds to an action. Store the operation, resource, arguments, state revision and expiry. Reject reuse if those details change.
  • Ambiguity stops the write. Missing policy, invalid approval or inconsistent state should pause consequential execution, not widen permissions.

Experiment to run

Place adversarial instructions in a retrieved document. Ask the agent to perform the legitimate task. Inspect both the answer and the execution log: the injected text must not cause a permission expansion or an unauthorized call.

A boundary, not a cure: filters can miss attacks or block benign inputs. Enforced permissions reduce the possible effect of a bad decision; they do not prove that every decision is correct.

03 / TOOL-CALL RELIABILITY

A timeout is not evidence that nothing happened.

Separate read operations from writes. A read may be repeatable within a budget; a timed-out write can leave an unknown state. Retrying without checking can repeat the side effect.

write-operation.txt / decision procedure, not a live log
PROPOSE    validate operation + arguments
AUTHORIZE  check scope + approval + state revision
EXECUTE    attach a stable operation identifier
CONFIRM    check the authoritative resulting state

If the response is ambiguous:
  query operation status
  → committed: reconcile local state
  → confirmed absent: retry only if permitted
  → still unknown: stop and request review

Design requirements

Prefer tools with typed errors, bounded timeouts, explicit idempotency semantics and status lookup. An idempotency key only helps if the downstream service enforces it; attaching an identifier alone does not make an action safe to repeat.

Experiment to run

Simulate a successful write whose response is lost. Verify that the agent reconciles with authoritative state or stops, rather than issuing a second effect. Repeat with an expired idempotency window and an unavailable status endpoint.

Limit: some tools cannot report or reverse their effects. In those cases, restrict automation or require an operator at the ambiguous boundary.

04 / LONG-TASK CONTEXT

Preserve state, not every token.

Long tasks need a compact, explicit record: the goal, constraints, confirmed facts, pending decisions, completed effects and evidence references. Conversation history can provide context, but it is not the authoritative record of what happened.

Separate what is known from what is merely remembered
StoreKeepRevalidate
Task stateGoal, current step, pending decisions, budgets.On resume and before changing scope.
Evidence indexSource, version, retrieval time, supporting span.When freshness matters or sources change.
Effect ledgerOperation identifier and confirmed result.After interruption or an ambiguous response.
Working summaryShort navigation aid with uncertainty labels.Against original evidence before relying on a claim.

Experiment to run

Interrupt a task after a confirmed side effect, compact the context and resume from a checkpoint. Change one source document in the meantime. Test for duplicated actions, lost constraints and reliance on stale evidence.

Limit: compaction is lossy. If a decision depends on an omitted detail, retrieve the original record or ask for clarification; do not infer it from a confident summary.