Empirical AI Research
We recorded 119 agent sessions in forensic detail: every tool call captured before execution, every token accounted, thirty artifacts per run. Then we graded the sessions against the maintainers' own tests and asked what the recording was worth. It reconstructs what the agent did with near-perfect fidelity, and it says almost nothing about whether what the agent did was any good. Observation and verification turn out to be different jobs.
A quirq is minted, never self-reported — budgeted outcome, before/after snapshots, verification mints delivered work.
The paper's worked ledger, live — cost per run, budget vs spend. Validate with your own numbers.
No longer is AI token spend the only metric for businesses. Let's prove real output across industry rails.
How we benchmarked idempotent compute across Nirvana ABS, E2B, and GKE agent sandboxes - and why only one platform kept every bit of committed work after a crash without a 35-second penalty.
Exploratory work, with an eye on future XO Cowork workflows. A hardened space container protects the host and does nothing to stop several agents inside it from reaching each other, and the usual fix, a container inside the container, needs the exact mount privileges the hardening removes. We tested the sandboxes that work without them, found one that does, and ran three experiments on it.
A working memory layer decides what an agent sees at each step, which is a different question from what it stores. Here is what breaks when the history gets long, what the memory layer has to do about it, and what one recent set of measurements says about the shape of a working answer: raw traces that survive their own summaries, notes that carry time and provenance, and a retriever that navigates a directory tree instead of ranking embeddings.
Why should you want a dynamic harness? Ask it from lived experience: the scaffolding around your agent — its instructions, memory, tools — is identical on day one and day one thousand. A 2026 wave of research has made harnesses that adapt to the task, the environment, and the model into measured results. This piece walks that ladder and names the unclimbed rung: the harness that adapts to you. We have found no demonstration of it, and the blocker is not mechanism but measurement — no one has yet established what the check for 'it learned me correctly' even is.
The harness around an agent is becoming a learner in its own right: it adapts what it does, edits what it is, and accumulates what it knows. Whether any of that helps turns on a single variable — who judges the adaptation. A verdict computed from the world compounds; a system grading its own progress fails fluently, and keeps going. The more the harness moves, the more the ground truth must not — and the ground truth is a property of the environment, not the model.
Model reasons; the harness enforces. ~1.6% of Claude Code is AI decision-making; 98.4% is infrastructure. A one-pager on the agent loop, permissions, context shapers, tools, subagents, and session persistence.
Frontier models can now tell when they are being tested, and they behave differently when they know. That breaks the static benchmark, because a passing score measures the model's read of the test environment, not its behavior in the wild. This is an analysis of the 2025 to 2026 alignment disclosures, and the case they make that the evaluation environment, not the benchmark, is the load-bearing component of alignment measurement.
A pilot measuring how a space's project context shapes a coding agent's performance and token cost: richer context is nearly free to run, it does not hurt task success, and it lowers the cost of getting oriented.
Even when an agent uses real data to mimic it, synthetic evaluation data falls short of organic data on the two things a safety evaluation most needs: calibrated incidence forecasting and construct validity. Why the gap is structural, why organic data matters, and where synthetic legitimately wins.
A guided evidence map of XO's agent-context studies, including the replication that revised the initial pilot and the questions each follow-up answered.
Businesses hire agents as workers. Skill unbundles from time, and output stops being capped by headcount.
Collaboration collapses toward the mind: humans stay the brain, agents become the runtime, and business impact becomes one blended ledger.
The end state: software that builds, repairs, and improves itself, with humans setting direction and guardrails.
Every calculation in quirq accounting: scoring, the mint, the all-in cost model, unit and portfolio metrics, the time axis, and the energy bridge.
The quirq ledger a company reads monthly, the reading discipline, and what adopting teams should do first.
The environment owns the ground truth: it is the specification, the scorekeeper, and the binding constraint.
The attack surface of a unit of account, in order of severity, and the structural mitigations.
How to write definitions of done, set budgets, and read the ledger schema every metric is computed from.
Hypothesis-first: every empirical claim with its falsifier, bound to numbered experiments E1 through E7.
Harvey outperforms raw frontier models on legal work. Reverse-engineering its architecture from public disclosures shows the advantage is not exclusive access to a smarter model, but the system built around it. Harvey treats the foundation model as swappable compute and puts every bit of its edge into the harness: routing, state, permissions, and deterministic task decomposition.
The popular read is that some languages are just hard for language models. We think that is mostly an artifact of how we represent them, the tokenizer and the writing system, not the language itself. Sanskrit is the sharpest test, because its extreme theoretical density is exactly what a standard tokenizer destroys. This is the pre-registered experiment we will run to find out which of language, script, and tokenization drives efficiency under fixed compute.
The unit of measurement for AI's business impact: budgeted by a human, minted by verification, dual to the token.
We added a third coding agent to the context ladder — Google's Gemini — and ran it head-to-head with Codex and Claude on a lightweight C repository. It is the first agent that behaves like it's curious: it tops our curiosity index, spends the largest share of its actions reading, and — uniquely — explores more the richer the space gets.
We wrote two operating contracts for a coding agent and matched them to the same length. One was generic; the other carried a single project-specific rule. The generic one changed nothing: conformance stayed at its 8% floor. The one with the rule lifted it to 80–100%. On these tasks it was not about how much context you give an agent, but whether the bytes close a gap it actually has.
We promised to raise the difficulty until the bare agent broke. We did, with harder tasks on a real 170-file service, and on these tasks it didn't break. With no documentation at all, two coding agents satisfied nine of nine non-obvious functional requirements, identically. On this kind of work, the bottleneck was never curiosity.
We gave two coding agents a steadily richer home — a README, an AGENTS.md contract, a project brief, a full scaffold, even a curated memory of how the codebase works — and measured what they actually opened. They read the surface and skip the substance. 84 controlled runs on environmental curiosity.
Comparing Claude Fable 5 and Claude Opus 4.8 on real engineering tasks with a reproducible harness — gated success, run observability, captured diffs, and blind judging — cut short when Fable was suspended before the harder tasks ran.
Perspectives we keep returning to: the XO view, a view from Claude, and a view from GPT.
The contract that replaces the prompt: a job with a definition of done, a budget, and a single owner, living in a space.
The research note behind the unit of work: what it argues, and where the full quirq treatment lives.