Building a Harness That Lets an LLM Explore the Blockchain

Executive Summary

Most attempts to point an LLM at the blockchain wire a model up to a block explorer. It reads the transaction list for an address and writes a summary. That works until you point it at anything adversarial… which onchain is most of what's interesting!

We wanted an agent that investigates the way a security researcher does: pull the bytecode, notice it's a proxy, resolve the implementation, read the storage slots, trace the money, check who holds which role, and decide whether it's looking at a wallet drainer or a keeper doing its job.

This post describes the harness we built to do that and the infrastructure underpinning it.

What a Harness Actually Is

Claude Code is the harness most people are familiar with. The underlying models used by Claude Code are the same ones available through Anthropic's API, but much of Claude Code's "magic" comes from the harness. It can read and write files, run shell commands, see the output, grep the codebase, spin up subagents, run the tests, read the failures, and try again. A harness is the scaffolding around a model that turns a text generator into an agent, something that can actually do work.

This means a harness encodes opinions about how to approach a problem. Do you hand the whole investigation to one agent with every tool, or split it into finely scoped specialists that each do one thing? What runs first, and what does the next step get to see as context? When does the agent decide it has enough and stop, and when does it escalate to a human?

Our harness is the blockchain equivalent of Claude Code's filesystem and shell. The rest of this post goes over the tools in it and the infrastructure that makes it trustworthy and fast enough to detect exploits at the largest scale.

Our Detection System: Traditional ML First, Agents Above the Risk Threshold

Traditional ML is fast and runs in real time on every transaction; LLM-powered agents are slow and cannot keep up with the throughput of a blockchain. This was our bread and butter long before GPT-3 was released.

So we don't send everything to an LLM-powered agent. A fast, explainable ML model scores every transaction in real time, and the agents kick in when that score crosses a risk threshold. Below the threshold, the ML verdict ships at millisecond latency. Above it, the transaction gets a full investigation. The agent is slower and more expensive, so it earns its keep on exactly the transactions where being wrong costs the most.

The LLM verdicts also feed back into our traditional ML classifier, which keeps our real-time (millisecond) capabilities state of the art.

Underneath, the system is built in two layers. The data layer is an indexing pipeline that decodes or simulates every transaction across 30+ chains and keeps a queryable picture of chain state: wallets and EOAs, contracts and proxies, transfers and funding flows, parameter updates, contract upgrades, ownership changes, signatures, and attribution labels. The detection layer reads from it, running the ML scoring and the agent investigations described above.

What Crosses the Threshold

Escalation is triggered by the risk score rather than by the model's uncertainty. Because the machine learning model is explainable, every score comes with a breakdown of which features moved it and by how much. For a flagged transaction, that might look something like this:

Each row shows a feature, its value for this transaction, its signed contribution to the score, and the direction it pushes. Here a single feature, a large mint-to-supply ratio that fits the classic freshly-minted-token pattern, is pushing hard toward "exploit," while the address's age, transaction count, and lack of a bribe all pull toward "benign." (These are illustrative; the real model weighs many more features.)

Those contributions sum to the risk score that decides the transaction's path. When a case crosses the threshold, the breakdown travels with it, so the agent starts out knowing which features carried the score and what it needs to go verify. Here that means checking whether the large mint-to-supply ratio is really evidence of something malicious, or an artifact of how the feature reads this particular transaction. (We'll walk through an example of this at the end.)

Learn more about Blockaid's AI Exploit Detection →

Building the Harness

Our harness lets a central orchestration agent powered by a frontier reasoning model coordinate specialist subagents over a shared tool and data layer.

Among them are a contract behavior analyst that reasons about what bytecode does (privileged roles, hidden owner powers, unauthorized fund movement, upgradeability risk, obfuscated control flow), a documentation crawler that reads protocol docs and verified source, an offchain investigator that pulls social and threat-intel signals (including a feed of drainer-linked X accounts and new scam domains), and a cross-protocol analyzer that follows money and ownership across bridges and chains.

The tools matter just as much as the prompts, because the tools bound what the agent can figure out deterministically and reliably. Beyond bytecode decompilation and the transaction-scan APIs, the set includes:

Raw RPC through a gateway. The agent issues eth_getStorageAt, eth_getCode, eth_call, eth_getLogs, and debug_traceTransaction with a callTracer across every supported chain through one authenticated endpoint. This is what lets it read a storage slot to resolve the EIP-1967 implementation behind a proxy itself, rather than describing an explorer page.
Proxy and storage resolution. First-class handling of EIP-1967 implementation, beacon, and admin slots, since a naive agent will otherwise analyze the proxy stub instead of the real code.
Execution tracing. Full internal call-tree reconstruction, so the agent sees the keeper → vault → agent → adapter → bridge flow rather than the top-level call alone.
Role and ownership checks. Direct hasRole() and owner lookups, so "the caller is authorized" is a verified fact.
Simulation. Running a transaction in a sandbox to see what it would do, with no gas and no state change.
Historical pattern lookup. Checking whether the same transfer shape has occurred before. This is the single most effective way we have of clearing false positives.

Underneath the subagents, the investigative units are small, named, versioned feature prompts. A drainer investigation composes checks like is_behavior_change_by_time (does behavior branch on block number or time?), has_obfuscation (is msg.sender or tx.origin being disguised?), function_selector (is the dispatcher hand-rolled, doing arbitrary calls or gas manipulation?), and a top-level is_drainer synthesis. Each has an ID, a version, and a prompt. Versioning lets us test one check in isolation, roll it from v1 to v2 without touching the rest, and trace any verdict back to the exact check that produced it.

The Agent Proxy + Evals

All agent traffic, including every subagent call, tool invocation, and model request, routes through a proxy we run in front of the system. It does three things:

It enables evals. Because every request passes through one place, we can replay historical cases against a new model or a new feature version and measure the difference before anything ships. The eval set is the backlog of past verdicts. The proxy also captures the full investigation trace for offline scoring.
A rigorous replay environment turns out to be where most of the real insight comes from. Many of our sharpest calls about which model, prompt, or harness design actually works only became visible once we could measure them against history.
It handles routing. URLs are config placeholders resolved at runtime against the environment rather than hardcoded, so the same investigation runs unmodified from a cloud agent or a researcher's laptop.
It gives us safety and observability. The investigation path is read-only by construction, so the agent can only trace, simulate, and read state. Every call is logged, attributable, and rate-limited (so we don't get any surprise Anthropic/OpenAI/Google bills that bankrupt us).

The Loop: The Agent Teaches Our Faster, Cheaper, Real-Time Models

Escalation is only half the value we get from the agents. Every LLM verdict is a labeled example, produced with evidence, that flows back into the model the next time it trains. And the agent only spends effort on the transactions that crossed the risk threshold, which are exactly the ones worth training on.

A clean false-positive verdict becomes a confirmed benign label, and it carries more than the label alone. The agent explains why the model was wrong, often pointing at a specific feature that read incorrectly for that transaction. Aggregate enough of those explanations and you stop fixing individual rows and start fixing the feature itself.

Even cooler is what happens when the agent keeps doing the same onchain check by hand. If it repeatedly resolves a proxy admin, confirms a caller holds a role, or matches a transfer against historical patterns, that check is a strong candidate to become a precomputed feature, and work that needed an LLM last quarter might be handled by our ML classifier this quarter.

And every verdict writes durable labels onto the entities involved (this proxy is a verified protocol contract, this address is a known keeper, this deposit address is a legitimate bridge adapter), which are themselves features the next transaction reads for free. The net effect is a ratchet, where the expensive path is also the teacher for the cheap one, and the cheap one keeps absorbing more of the work.

Here's a simplified version of the feedback loop:

The Alerts We Don't Raise

The most important output of this system is the alerts the agent doesn't raise. Flagging a ~$994K transfer leaving a pool is easy; any dollar threshold can do it. Correctly not flagging it, and producing the chain of evidence for why, is the work that makes this so valuable to our security research team.

Most of the value here is the ability to clear a suspicious-looking transaction with confidence, at scale. It lets our human experts encode their knowledge and repeatable research processes into something automatic and scalable, and spend their actual cognitive bandwidth on the edge cases that need it.

Example: A $994K "Exploit" That Wasn't

To make all of this concrete, here is one escalation end to end.

A wallet drainer and a legitimate withdrawal can produce nearly identical traces. Both move large sums, both call into proxies, and both touch the same kinds of contracts. The difference lives in context that no single transaction contains: who deployed the contracts, whether the caller actually holds the role it's invoking, and whether the same transfer has happened many times before.

An alert fires on a ~$994K transfer out of a lending pool, flagged as a possible exploit. A few minutes later the agent reaches a verdict:

Verdict: false positive. This is a legitimate Syrup USDC pool withdrawal, not an exploit. The user called their own smart wallet, which routed through the Syrup pool (a verified proxy whose admin is the Syrup Pool Manager). The pool released the lender's $1M principal back to them and collected ~$6.2K in fees, for a net of +$993,838. The "unfair trade" the detector flagged was the loan vehicle losing $994K, but that contract is part of the protocol releasing the lender's own principal.

To get there, the agent reconstructed the USDC transfer trace hop by hop, resolved the smart wallet's owner and the pool's proxy admin, pulled the sender's history (hundreds of transactions, a ~$190K portfolio, no exploiter labels), and checked the target's existing labels, which already mark it as the kind of verified protocol contract that produces this exact class of intra-protocol P&L. The model has to go and get each fact, then decide what to look at next based on what it just found.

It also caught the traditional ML feature that misfired. The one signal pushing the classifier toward "exploit" was "exploit from a very low nonce address," but the actual sending nonce was 498, nowhere near low. That correction is exactly the kind of signal that flows back into the model through the loop we talked about above.

Not every case is a false positive, and not every one is this shallow. Other alerts have taken the agent through six-hop call flows across bridges and adapters, OpenZeppelin hasRole() checks, and deployment-history lookups across four related contracts before it could say with confidence whether a large transfer was a drain or a keeper doing its job.

Conclusion

We set out to build an agent that reasons about the chain, and spent most of the time on the gateway, the proxy, the versioned features, and the eval loop that make the reasoning repeatable and worth trusting. The model supplies the reasoning, but the harness decides whether that reasoning is grounded, checkable, and cheap enough to run at blockchain scale.

That investment is also what makes the system easy to grow. Adding a chain means adding it to the gateway. Handling a new attack class means writing a versioned feature and evaluating it against history. When the cheap model plateaus, the agent's verdicts are already training the next version of it.

If you're also building agents that need to reason about messy, adversarial state (rather than chat/hallucinate about it), we'd love to hear from you!

Request a Demo →

About Blockaid

Blockaid is the onchain security platform trusted by the largest companies operating in Web3. Built by veterans of elite intelligence and cybersecurity units, Blockaid provides end-to-end protection for financial institutions, protocols, and end users, combining direct wallet and dApp integrations with real-time monitoring, detection, and response across smart contracts, infrastructure, and externally owned accounts. Since 2025, Blockaid scanned over 6.3 billion transactions and blocked 585 million attacks. Blockaid is the security infrastructure behind Coinbase, MetaMask, Uniswap, Safe, and dozens of the most widely used platforms in the industry.

Learn more at blockaid.io, and follow us on Twitter and LinkedIn.