Sources#
- Beyond RAG: Building Agentic Document Workflows with LlamaIndex
- How Bridgewater Built an AI Analyst That Does Hours of Expert Research in Minutes
- Muscle Memory for Agents: Compile not Merely Retrieve
- Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production
- Sidekick's continual learning loop
Summary#
An agent platform is a permanent cost centre by default: every execution re-invokes full inference, so the tenth occurrence of a recurring incident costs exactly what the first did, takes a possibly different path, and may reach a worse answer. The waste is that a successful investigation is discarded.
Arun Malik (Microsoft Azure Networking, arXiv 2607.07052, 2026-07-08, case-study) names the fix and reports it running in production on a cloud-network operations platform handling tens of thousands of incidents a month: progressive crystallization — agent exploration is a discovery mechanism, and behaviour it discovers and repeatedly validates gets converted into deterministic workflows that need zero tokens, while the agent layer stays available for what is genuinely novel.
The three-type spectrum#
The taxonomy is the reusable part, and its point is that the three types are lifecycle stages of one discovered behaviour, not three separate systems:
| Type | Execution | Determinism | Tokens/run |
|---|---|---|---|
| Type 3 — agent-orchestrated | Sub-agents reason freely within a bounded scope; autonomous reads, deterministic checkpoints, human-in-the-loop gates on writes | ~50% | 10k–50k |
| Type 2 — hybrid | Fixed step structure; the LLM is called only at specific stages, for interpretation, classification or summarization. Every action is typed and schema-validated | ~90% | 1k–5k |
| Type 1 — deterministic | Pre-coded logic, conditionals, typed API calls; no LLM at runtime | 100% | zero |
The Type 2 formulation is the sharpest line in the paper: the LLM decides understanding, not what to do. That is the same split Deterministic Pre-Execution Gates draws between the model's judgment and the mechanically-checkable envelope around it, arrived at from the cost side rather than the safety side.
Promotion is gated on evidence, and demotion is automatic#
| Transition | Requirements (defaults, configurable) |
|---|---|
| Type 3 → 2 | ≥10 successful runs · zero safety violations · ≥90% of runs produce the same action sequence · all auto-generated acceptance tests pass · no human override in the recent window |
| Type 2 → 1 | ≥50 successful hybrid runs · LLM classification consistency ≥99% · the deterministic rule covers all observed input variation · full regression suite passes without the LLM · human review of the deterministic logic |
Capture is a process-mining step, not a macro recorder: the trace is parsed into an ordered list of tool calls, branch conditions the agent actually acted on are detected, per-step input/output schemas are inferred, a DAG of tool dependencies is built, instance-specific values (device IDs, timestamps) are parameterized, and human-approval points are marked as explicit gates. Acceptance tests are generated from the successful traces, and the candidate playbook must pass them before promotion.
The circuit breaker is what makes this safe to run unattended. Each promoted playbook is monitored and demoted on execution failure, safety violation, or acceptance-test regression. The production anecdote is the whole design in one paragraph: a firmware update changed a command's output format, the deterministic parser broke, the system demoted the playbook to hybrid so the LLM could absorb the new format, and after a run of clean executions it was re-promoted — with no human deciding when to switch.
The claim worth arguing with: autonomy is earned by track record, not by model capability#
"Autonomy is attached to the specific playbook class and action type, based on its evidence, rather than to the capability of the underlying model. A more capable model does not automatically earn more autonomy; a track record does."
This is a direct counter-position to the shape of Harness Shrinkage as Models Improve. The shrinkage thesis says scaffolding is a tax on current model weakness and should be deleted at each release. Crystallization says the opposite about permissions: what a system is allowed to do unsupervised is a property of the demonstrated reliability of a specific behaviour, and a model upgrade does not transfer that evidence. Both can hold — the shrinkage argument is about how much instruction the model needs, this one about how much authority a workflow has earned — but the two point in opposite directions on the same question a team actually faces at every model release, and nothing in the corpus reconciles them.
It also inverts the usual reading of a capability upgrade. On this design a better model does not make the platform cheaper directly; it makes discovery better, and the savings arrive later, when what it discovered crystallizes.
Safety improves with cost, rather than trading against it#
The paper's counter-intuitive claim is that cheaper is safer, and the argument is structural rather than empirical: auditability is unchanged (all three types record full traces); reproducibility rises monotonically ~50% → ~90% → 100%; blast-radius control moves from runtime HITL gates → schema validation → statically verifiable deterministic logic, which is stronger because it can be checked before execution rather than caught during it; compliance moves from conditional-on-a-human-being-present to built-in. A deterministic playbook distilled from an agent run is easier to verify, reproduce and audit than the run it came from.
The production numbers, and what they are not#
Over eight months on the Azure cloud-network platform: Type 1 executions 0% → ~45% (with ~30% hybrid, ~25% agent-orchestrated), per-incident agent cost down >70% while incident volume roughly doubled, >90% of common incident categories resolved autonomously, MTTR from hours to minutes, false-positive remediation under 5% with no customer-visible impact.
The author's own limits, which bound every number above:
- Single organization, single operational domain. Thresholds and ratios should be re-derived elsewhere; only the lifecycle is claimed to be domain-agnostic.
- Platform-level observations, not a controlled comparison. There is no counterfactual arm — nothing establishes what the cost curve would have done without crystallization, and eight months is one maturity trajectory.
- It assumes recurring patterns. In an environment dominated by genuinely novel one-off work, most executions stay Type 3 and the mechanism buys little. This is the scope condition that decides whether the idea transfers to coding agents at all: incident response is unusually repetitive, and a codebase feature request may not be.
- Promotion quality is bounded by trace quality. Auto-generated acceptance tests come from traces; an under-observed pattern can be promoted prematurely, which is exactly why demotion and human review of the final deterministic logic are in the design.
Type 1:2:3 as a maturity metric is the paper's most portable suggestion — the execution mix is a single number that says how much of your agent spend is buying discovery versus re-paying for solved problems.
What the crystallized thing looks like: typed events as the architecture#
Malik's lifecycle says when agent behaviour should become a workflow and says almost nothing about the workflow's shape. Doulcet's LlamaIndex workshop (AI Engineer Singapore 2026, practitioner-opinion, vendor COI — this is a description of LlamaIndex Workflows, not a comparative claim) is the corpus's one detailed account of the shape, and it converges on this page from the other end.
The starting complaint is Type-3-to-Type-2 drift stated as a code smell. A linear pipeline — parse, embed, retrieve, synthesize — works until the day you need to branch: "if the document is a contract, run clause extraction; if it's a report, run summary. Now your pipeline is an if-statement. Two more ifs and it's a mess." The proposed replacement is an event-driven graph: steps are async functions with typed Pydantic events in and out, and the graph is implied by the type signatures — no explicit orchestration object, no wiring code. Four primitives cover it: steps, events, a shared mutable Context, and the Workflow container.
Four properties follow, and each is a claim about what a crystallized workflow buys that a re-invoked agent does not:
- Typed events are the architecture, not an implementation detail. The naming rule is the whole argument: name an event by what it represents (
ParsedDocument), never by who produced it (Step2Output), so future-you reads the type signatures and sees the graph. This is the same instinct as Malik's demand that a promoted playbook be legible enough to audit — a Type 1 workflow nobody can read cannot be demoted on evidence. - Parallelism is declarative.
send_eventfans out (a step emitting N events of one type runs N branches concurrently),collect_eventsjoins on a count held in the context. No threading, no async-pool management. The same primitive serves per-page document ingestion and deep-research sub-question fan-out — see Deep Research Agents for the tree-shaped case. - Human review is an event, not an exception. The naive shape (pause the process, send an email, hope the human returns) is "fragile, stateless, untraceable." The prescribed shape emits
HumanReviewRequested, persists the whole context durably for minutes or days, and resumes onHumanReviewCompleted— so a hours-long human decision is an ordinary edge in the graph. The consequence is the interesting one: if human review is a typed event you can eval it — measure when humans agreed, when they disagreed, capture what they changed, and feed it into the next eval set. "Treat the human as part of the system you measure, rather than a fail-safe sitting outside it." That is Configurable Human Participation's configurable-agency axis reached from engineering rather than benchmarking, and it is what makes a promotion criterion measurable at Type 2. - Observability falls out.
stream_eventsemits the same typed events that were written down as architecture, so the trace schema and the design are the same artifact — no separate logging layer. Durability is the paired property: retry the failed step, not the pipeline, and resume from checkpoint after a crash, because "parsing a 500-page contract is not cheap; you don't want to redo it because synthesis had a transient network error."
The boundary condition agrees with this page and is stated as a warning. "If your task is 'answer a single Q&A query over a single document', a function will do." Workflows earn their cost only under branching, parallelism, durability needs, or human-in-the-loop — Malik's admission criteria for leaving Type 3, restated as a caution against reaching for the machinery by default. Neither source measures the alternative.
Four reference shapes are given, and the last is the one with no analogue on this page: contract review (classify → per-clause parallel extraction → risk score → HITL on flagged clauses only → report); deep research; due diligence over a 5,000-document data room (per-type extraction schemas → accumulate a fact store → cross-document reconciliation → surface anomalies); and a living knowledge base — watch sources, reparse on change, re-extract, incrementally reindex, publish fact-level diffs to subscribers, with no StopEvent. A crystallized workflow that never terminates is a category Malik's spectrum does not contain: its Type 1 endpoint is a deterministic procedure with a finish line, and this is a standing process whose output is a change feed.
The same move on conversation instead of incidents — and the missing circuit breaker#
Muscle Memory for Agents (Omran, Lanka, Zhang & Dixit, Google Cloud FDE, arXiv 2608.08995, 2026-08-10, empirical) runs this page's lifecycle in a domain with none of incident response's advantages: it mines recurring user intent out of conversation history and compiles it into quality-gated executable specialist agents. Full treatment on LLM-as-Compiler Knowledge Base; three things it settles or exposes here.
It supplies the discovery step Malik's Type 3 → 2 transition assumes. Malik's capture is process mining over the traces of a behaviour someone has already identified as recurring. Here recurrence detection is inside the pipeline: patterns are extracted from batched sessions at frequency ≥ 3, merged by Jaccard similarity > 0.5, split into task patterns (what the user wants) and behavioural patterns (how they communicate), then gated by a non-parametric ranking that retains only candidates scoring ≥ 25/50 and an overlap merge at cosine-plus-Jaccard ≥ 0.73. "No fixed target is imposed" — the pipeline decides how many artifacts each domain should have, landing on 1–6 per user. That is the question Malik's ≥10-successful-runs threshold answers by hand.
But it gates in the wrong place for unattended operation, and its own numbers show the cost. Both sources gate promotion on evidence; they differ on when the evidence is collected. Malik's is a runtime track record (≥10 successful runs, ≥90% identical action sequence, zero safety violations; then ≥50 runs and ≥99% classification consistency for Type 1) with automatic demotion on execution failure, safety violation or acceptance-test regression — the firmware-update episode is the whole design in one paragraph. Muscle Memory's is replayed history (a critic pass on 10 criteria at a 7/10 threshold against real conversations, plus a mini-eval on 6 historical scenarios) and then nothing: its own third limitation is that "agent triggers are static after generation, that is, the system does not adapt agents based on runtime feedback or evolving user preferences." What that costs is visible in its results — a 20% false-positive trigger rate on out-of-domain requests, and one user whose over-pruned single agent costs a full point of accuracy on a 1–4 scale — with no mechanism in the system that could notice either. The evidence-gated lifecycle is the stronger control, and the corpus's newest compiled-artifact system has only the weaker half of it.
The two gates also fail differently, which is worth keeping. Malik's is a consistency gate: it promotes what reproduces. Muscle Memory's ranking includes a distinctiveness term scored against general-purpose LLM capability, and it over-pruned the one technical user to a single agent because "'explain Python errors' is valuable to the user but indistinct from the baseline." A distinctiveness gate discards behaviour the base model already handles adequately — right for cost, wrong for coverage — and a consistency gate has no such failure mode.
The economics run the opposite way, so "crystallize" names two different endpoints. Malik's Type 1 endpoint executes with zero tokens: the LLM leaves the runtime entirely. A compiled Muscle Memory specialist still makes its own LLM call (static architecture) or a 2–3 stage pipeline of them (dynamic), plus a routing feature-extraction call and an embedding lookup on every message. It crystallizes the prompt and the blueprint, not the decision procedure — which puts it at roughly Type 2 on this page's spectrum, and it spends more per call than the baseline it beats. Malik's compile move buys cost reduction; this one buys quality and pays for it. Both are promotions of discovered behaviour into a tested artifact, and only one of them retires the model.
A third endpoint: crystallize into weights, not into code#
Shopify's Sidekick account (McNamara & Mazza-Anthony, Shopify Engineering, 2026-08-05, case-study — first-party, unreplicated) runs the discover-then-promote instinct to a destination neither source above contains. Its harness stage is recognisably this page's move: an autoresearch agent proposes edits to prompts, tool definitions and orchestration code, evaluates each against a calibrated judge, and "keeps the change if the score improves; discards it otherwise" — promotion gated on measured evidence, one candidate at a time. What happens next is the new part: "once harness improvements plateau, we begin optimizing in parameter space," and the discovered behaviour is compiled into model weights by SFT on repaired production trajectories followed by GRPO. Full mechanism on Agent Quality Flywheel.
So "crystallize" now names three endpoints, and the economics differ at every one:
| Endpoint | Artifact | Runtime cost | What retires |
|---|---|---|---|
| Malik, Type 1 | deterministic pre-coded logic | zero tokens | the model |
| Muscle Memory | a compiled specialist prompt + blueprint (≈Type 2) | more than the baseline it beats | nothing — it buys quality and pays for it |
| Shopify | model parameters (plus 1,500 gist tokens in place of a 6,000-token prompt) | a cheaper model on the same path | the frontier model, not the inference call |
The third is the only one where the discovery loop itself is what gets retired rather than the execution: the harness edits an autoresearch agent found are eventually absorbed into a model that no longer needs them stated. And it inverts this page's determinism axis. Malik's promotion ladder buys more determinism at each rung (≈50% → ≈90% → 100%); a fine-tune buys none — the runtime is exactly as stochastic after the promotion as before, which is why no demotion criterion could exist in the same form. What Shopify has instead of a circuit breaker is the daily retrain over accumulated data, which is a drift-resistance mechanism, not a regression detector: nothing in the account notices that a promoted behaviour got worse and rolls it back. On this page's own comparison that is the weaker half of the gate again, in a third system.
And it does not answer this page's transfer question — see the annotation on that bullet below for why a merchant-facing GraphQL agent is not the out-of-IT-operations evidence the question asks for.
The admission criteria converge independently, which is the page's most reusable claim getting a second source. Muscle Memory's §3.3 conditions — the same intent recurs often enough to amortize the compile cost, consistency across instances matters, and patterns are discoverable from observed history rather than only declared up front — are Malik's "it assumes recurring patterns" scope condition stated as a three-part test, with the same negative case named ("one-shot factual queries, novel tasks with no history, exploratory dialogue"). Evidence note: this is the first empirical source touching this page, but it measures output quality on 90 held-out scenarios, not the lifecycle — no promotion, demotion, or cost curve is measured anywhere in it, so the page's case-study evidence on crystallization itself is unchanged.
Connections#
-
Agentic Code Generation as Compilation — the same destination reached from the opposite end. Malik's spectrum earns deterministic execution by promoting behavior that has repeatedly validated; Bridgewater's PAT starts there by construction, because the domain's shape (an analysis is a data-frame DAG) was known before anything was built. Weight's formulation of why it matters is the sharpest statement of this page's premise in the corpus — "we enforce correctness in the architecture... the agents cannot forget to validate. They are forced to validate" — the difference between a step an agent is instructed to take and one that is a statement in the enclosing program. What PAT lacks is this page's demotion path: nothing in it describes what happens when a crystallized assumption stops holding
-
Continuous Self-Modification Under Review — the opposite gate design, on an agent editing its own harness rather than an ops platform: admissibility, not evidence. Ouroboros blocks 63.5% of recent self-edit attempts through a multi-model diff-review panel, but nothing in its pipeline asks whether a landed change helped — there is no track record, no promotion, no automatic demotion on regression, and no pre/post comparison anywhere in 1,085 self-modification commits. This page's contribution read against it is that an evidence-gated lifecycle is a strictly stronger control than a review gate, and the corpus's most autonomous self-modifying system has only the weaker one
-
Document Parsing as the Retrieval Bottleneck — the workflow-shape half of the same subject, treated above: typed events as the contract, declarative fan-out/fan-in, durable HITL, observability as a by-product, and the never-terminating "living knowledge base" shape. It also supplies the upstream dependency this page assumes away — a workflow whose first step parses a document inherits every structure-loss failure that step can make, and no amount of promotion discipline downstream recovers it
-
Configurable Human Participation — the same move from two directions: HAS-Bench makes human participation a benchmark variable, the workflow account makes it a typed event, and both do it for the same reason — a human decision you can address as a first-class object is one you can measure agreement, disagreement, and edits against, which is what turns HITL from a fail-safe into an evidence source for promotion
-
Deterministic Pre-Execution Gates — the same model/machine division reached from the safety side rather than the cost side; Type 2's "the LLM decides understanding, not what to do" is that split stated as an execution type, and Type 1's statically-verifiable logic is the strongest form of a pre-execution check
-
Dynamic Workflows: An Algebra for Agents — the composition primitive this lifecycle retires: where the algebra is about an agent writing a program that composes agents, crystallization is about the program surviving the agent and eventually running without one
-
Cost-per-Task Over Cost-per-Token — the cost frame this operationalizes and then escapes: the per-task cost of a solved problem falls to zero rather than to a cheaper model's price, which is a different lever from routing or cascades (the paper positions itself explicitly as orthogonal to FrugalGPT-style per-call cost reduction)
-
Harness Shrinkage as Models Improve — the tension worth holding: shrinkage says scaffolding is a tax on current model weakness that each release should retire, crystallization says a workflow's authority is earned by its own track record and does not transfer across a model upgrade
-
Implementation Abundance Inverts Product Work — a partial answer to the curation-cost question from a different domain: the cost of curating exploration is paid down by promoting what the exploration proved, so curation is a one-time investment per recurring pattern rather than a recurring tax — but only where the patterns recur, which is the scope condition an exploratory product context may not meet
-
Agent Quality Flywheel — the third endpoint, treated above: the same evidence-gated promotion of discovered behaviour, compiled into model weights rather than into deterministic code, with the harness stage explicitly exhausted first ("once harness improvements plateau, we begin optimizing in parameter space"). It inverts this page's determinism ladder — a fine-tune buys no determinism, so it also has no demotion criterion — and it is the source scouted for this page's out-of-IT-operations question, which it does not answer
-
Agentic Work Systematization — the same instinct as skills-and-plugins (externalize what you keep re-deriving), pushed one stage further: not "write it down so the agent re-reads it" but "compile it so no agent runs at all"
-
Verification as the New Bottleneck — the auto-generated acceptance tests are what make promotion possible, so verification quality is the binding constraint on how much work can leave the agent layer
-
LLM-as-Compiler Knowledge Base — the same compile move on knowledge rather than behaviour, and the source of the contrast above: Muscle Memory gates a compiled specialist once against replayed history and then freezes it, while this page's playbooks earn authority from a runtime track record and lose it automatically on regression. Its
mannwhitneyuerror analysis is also the compile-time-loss risk both pages inherit, measured — a fabrication baked into the artifact at compile time scored 1/4 against the un-compiled baseline's 4/4 -
Authority and Audit Survive Abundance — resolves this page's upgrade-moment conflict with Harness Shrinkage as Models Improve: shrinkage governs instruction scaffolding, this page's evidence-gated authority is governed by track record alone, and the demotion circuit-breaker is what makes the upgrade moment decision-free on the authority side
-
Guarantees That Degrade at Deployment: Action-Space Soundness, Admissibility Without Effect, and a Vendor-Coupled Security Framework — this page supplied the deployed proof for the self-modification cluster's answer: an evidence-gated promotion ladder with automatic demotion on execution failure, safety violation or acceptance-test regression is the effect test Ouroboros's review gate lacks, running in production in a domain with no more ground truth than a public chat deployment has
Open Questions#
- Does the lifecycle transfer out of IT operations? Every number here comes from incident response, which is unusually repetitive and has a crisp success oracle (the incident resolved). Coding, product and research work have neither property in the same degree. The falsifiable version: apply the Type 3→2→1 promotion criteria to an agentic coding pipeline and measure whether the deterministic share rises at all over comparable time. Checked, not answered (2026-08-13): Sidekick's continual learning loop was scouted for this and fails it on three counts. (1) Wrong domain shape, not a different one. A merchant-facing agent that introspects a GraphQL schema, looks up filter syntax and executes a query against the Admin API is structured-query generation against a machine-checkable target — closer to incident response than to a feature request, and it shares exactly the two properties this bullet says coding lacks: the task recurs across millions of merchants, and correctness is largely oracle-checkable because the query either returns the right rows or it does not. It is the same shape in different clothes. (2) No transfer claim is made. The piece is scoped to one agent, and the ingest pass found no assertion anywhere that the loop generalizes to other Shopify domains, let alone out of them. (3) It is not this lifecycle. Nothing is promoted toward determinism — the endpoint is model weights, the runtime stays stochastic, and there is no demotion mechanism at all (see the third-endpoint section above). A single-agent vendor account cannot carry this question in either direction, and marking it answered on one would be the error this annotation exists to prevent. The falsifiable version stands unchanged.
- The platform-level cost curve has no counterfactual arm. How much of the >70% per-incident cost fall is crystallization, and how much is ordinary model-price decline plus caching over the same eight months? A controlled comparison, or a decomposition against contemporaneous list prices, would separate them.
Resolved Questions#
- Crystallization and harness shrinkage give opposite instructions at a model upgrade — delete the scaffolding versus keep the evidence-gated permissions. Which governs, and does the answer differ for instruction scaffolding versus authority scaffolding? Answered: Authority and Audit Survive Abundance — neither governs the other; they govern disjoint objects, separable by one test: can the model being better make this line unnecessary? Instruction scaffolding encodes a task prior (shrinkage governs — ablate at every release); this page's authority scaffolding encodes a boundary plus a local evidence record, neither of which a capability jump supplies — and the security corpus makes the stronger claim that authority cannot migrate inward, because a component that grants its own scope is circular ("you can delegate judgment; you cannot delegate authorization"). The sort is by what a line encodes, not where it lives (a prompt-borne scope declaration is still authority-class; the derived page grounds this in the constraint/request asymmetry). At upgrade day the reconciliation is already in this page's design: the launch pass prunes instructions, permission grants stay untouched, and the demotion circuit-breaker re-earns authority from evidence continuously — so the upgrade moment requires no authority decision at all.
Sources#
- Beyond RAG: Building Agentic Document Workflows with LlamaIndex — Pierre-Loic Doulcet, AI Engineer Singapore 2026 (
practitioner-opinion, LlamaIndex vendor COI; a description of one framework, with no comparison and no measurement): Part V — the four primitives, typed-event naming rule,send_event/collect_events, durable HITL,stream_eventsobservability, the when-NOT-to-use-a-workflow boundary, and the four reference shapes. The event-graph and durable-pause diagrams exist only as slide images and were read via the image two-pass; full source treatment and parse warnings on Document Parsing as the Retrieval Bottleneck - Muscle Memory for Agents: Compile not Merely Retrieve — Omran, Lanka, Zhang & Dixit (Google Cloud FDE), arXiv 2608.08995, 2026-08-10,
empirical: §3.3 the admission criteria, §4.2 the pattern-mining and quality-gate stages, §6 the static-trigger limitation and the over-pruning finding, Table 2 the per-user results. Scoped to the sections above — it measures output quality, not this page's lifecycle, so nothing here upgrades the page'scase-studyevidence on promotion, demotion or the cost curve. Full source treatment and parse warnings (Table 1 cell-collapsed and reconstructed;canary-recallreportsokwithout running) on LLM-as-Compiler Knowledge Base - Sidekick's continual learning loop — Andrew McNamara & Cody Mazza-Anthony, Shopify Engineering, 2026-08-05,
case-study(first-party account of the authors' own production system; single agent, no controlled arm, no transfer claim). Cited for the third-endpoint comparison only: the autoresearch propose-evaluate-keep-or-discard loop and itsprogram.mdconfig, the harness-plateau ordering, the gist-token figures used in the endpoint table, and the absence of any demotion mechanism. Its quality and cost figures are not used here. Full source treatment on Agent Quality Flywheel - Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production — Arun Malik, Microsoft Azure Networking, arXiv 2607.07052, 2026-07-08 (
case-study, sole author, first-party account of the author's own production platform, not peer reviewed; no controlled comparison): §III + Table I the execution-type taxonomy, §IV + Table II the promotion criteria and trace-extraction algorithm, §V the economic model, §VI + Table III safety monotonicity, §VII the demotion circuit-breaker and the firmware-update episode, §VIII the eight-month production figures, §IX the author's limitations. docling verifyok— 4pp, 3 tables, 3 pictures, no collapse or shift; the three figures restate numbers stated in prose, so the image two-pass was not load-bearing - How Bridgewater Built an AI Analyst That Does Hours of Expert Research in Minutes — McManus, Ran & Weight (Bridgewater Associates), LangChain channel, 2026-07-24, 25:44 talk,
case-study. Cited for "we enforce correctness in the architecture... the agents cannot forget to validate" (22:37–24:00). First-party and unmethodologized throughout — see Agentic Code Generation as Compilation for the evidence caveats on every figure
Cited by 19
- Harness Shrinkage as Models Improve×3
The counter-currents above concede size and direction while leaving the page's instruction intact:…
- Agentic Code Generation as Compilation×2
The high-level architecture diagram, he notes, is "actually just Python code. It's influenced by…
- Authority and Audit Survive Abundance×2
The candidate split both pages already carry is correct, and the corpus can now ground it rather…
- Continuous Self-Modification Under Review×2
Crystallizing Agent Work Into Workflows — the contrasting promotion gate: there autonomy is earned…
- Document Parsing as the Retrieval Bottleneck×2
The deck's answer to "what do you wire around a document the agent can actually read" is LlamaIndex…
- Guarantees That Degrade at Deployment: Action-Space Soundness, Admissibility Without Effect, and a Vendor-Coupled Security Framework×2
The obvious objection to demanding an effect test in a live, unbriefed, seven-surface deployment is…
- Agent Quality Flywheel
Crystallizing Agent Work Into Workflows — the third endpoint for discovered behaviour: promotion…
- Agentic Work Systematization
Crystallizing Agent Work Into Workflows — the same externalize-what-you-re-derive instinct pushed…
- Configurable Human Participation
Crystallizing Agent Work Into Workflows — the engineering form of this page's central move. Making…
- Cost-per-Task Over Cost-per-Token
Crystallizing Agent Work Into Workflows — the cost frame taken to its limit: for a solved,…
- Deep Research Agents
Crystallizing Agent Work Into Workflows — the runtime this form factor needs, and the lifecycle…
- Deterministic Pre-Execution Gates
Crystallizing Agent Work Into Workflows — the same model/machine division reached from the cost…
- Dynamic Workflows: An Algebra for Agents
Crystallizing Agent Work Into Workflows — what happens to a composed program after it works:…
- Implementation Abundance Inverts Product Work
Crystallizing Agent Work Into Workflows — a partial answer to the curation-cost question from an…
- LlamaIndex
Crystallizing Agent Work Into Workflows — LlamaIndex Workflows are the concrete event-driven…
- LLM-as-Compiler Knowledge Base
Crystallizing Agent Work Into Workflows — the same compile move on behaviour rather than knowledge,…
- Agent Systems & Harness Engineering
Crystallizing Agent Work Into Workflows — Malik's production lifecycle at Azure Networking: treat…
- Open Questions Backlog
Crystallizing Agent Work Into Workflows ×2 (oldest 14d) — Does the lifecycle transfer out of IT…
- Verification as the New Bottleneck
Crystallizing Agent Work Into Workflows — verification as the gate on how much work can leave the…
Related articles
- Harness Shrinkage as Models Improve
Prompt scaffolding shrinks each model release; Cat Wu's pruning discipline; Boris Cherny "100 lines of code a year from…
- Optimizer–Evaluator Decoupling
The architectural rule in eval-fix loops that whatever proposes a fix (coding agent, automated optimizer, human) never…
- Open Questions Backlog
Generated by `_system/lint.py --write-backlog`. Do not hand-edit. Domain and Watching sections carry one row per page —…
- Verification as the New Bottleneck
Fiona Fung: coding is no longer the bottleneck — verification, review, maintenance are; shift-left; TDD loses its tax;…
- Agent-Authored Harness Optimization
An agent runs the whole eval-fix loop on its own harness — read traces, hypothesize, patch, re-run. Seven instances dis…
