Bridge Atlas Episode 5: Verifiability

Issue #74: A conversation with Shreya Shankar and Justin Drake

Welcome back to the final episode of the Bridge Atlas series, hosted by Christine D. Kim. We discuss Ethereum and AI systems through the lens of verifiability, with guests Shreya Shankar, a PhD researcher at Berkeley, and Justin Drake, an Ethereum Foundation researcher working on the lean Ethereum project.

---

Christine: Justin, can you start us off by introducing the lean Ethereum
project and what that entails?

Justin: In lean Ethereum, we’re tackling verifiability in two ways. First, for deterministic blockchain computation, instead of re‐executing transactions on thousands of nodes, we use cryptographic SNARKs – short proofs a single person produces and everyone else can verify in milliseconds. Second, for the non‐deterministic consensus part, we aim to speed up voting to achieve finality in seconds instead of minutes.

Christine: Shreya, I saw you smiling when Justin mentioned SNARKs. Is that something you are interested in as it relates to verifiability?

Shreya: I smiled because I learned about zk-SNARKs early in my PhD – they’re very cool! My own work is on DocETL, a declarative system for unstructured data processing. For example, a public defender might mine court transcripts to find racially biased statements. This requires reasoning beyond traditional databases, so DocETL lets users write MapReduce-style pipelines with operators defined in natural language and executed by LLMs.

Verifiability for us means: how do we ensure AI outputs align with human expectations? We optimize pipelines for accuracy and cost, sometimes replacing LLM functions with deterministic code. We also expose statistical primitives so users can verify samples and estimate worst‐case accuracy. Provenance – establishing where in the source particular data statements came from – and understanding AI reasoning are important too.

Justin: That reminds me of my ‘spectrum of trust’ model. On one end you have math, physics, cryptography – things you trust completely. On the other, legal systems and reputation – mushy, subjective. Blockchains aim for trust minimization, pushing everything toward the ‘trustless’ end. Dispute resolution in marketplaces remains hard; maybe your work could help build trustless marketplaces without escrow agents.

Shreya: How are disputes handled now?

Justin: In OpenBazaar, buyers and sellers agree on a trusted escrow agent. It’s reputation‐based and can be inefficient. Agents can also be tempted to collude for large transactions.

Christine: Shreya, talk about AI evals in verifiability.

Shreya: We annotate AI outputs to identify features correlated with human disagreement – missing definitions, odd prioritization, etc. – until the distribution of these ‘qualitative codes’ stabilizes. Then we can train LLM judges to evaluate outputs at scale and measure misalignment.

Justin: Decentralization in Ethereum is similarly hard to measure. The Nakamoto coefficient – how many operators control 33% or 50% of stake – has limits. SNARKs lower barriers to entry so even small devices can verify. For non‐deterministic aspects, economic penalties ensure that bad behavior would destroy large amounts of staked value. Shreya, how are adversarial actors handled in AI?

Shreya: We see two threat models: malicious users crafting pipelines to prove biased points, and non‐malicious users blindly trusting AI outputs without knowing counterfactuals. The latter worries me more.

Justin: In SNARKs, we counter maliciousness with interactive proofs – multiple rounds where the verifier challenges the prover – and ‘hinting,’ where the prover supplies extra information to make verification easier.

Shreya: I can imagine combining those for DocETL: for example, AI showing users samples it didn’t find interesting and asking them to label them.

Justin: Hinting parallels a teacher giving students clues.

Shreya: In data science, negligent or biased analysis isn’t new – but AI’s agency over vast amounts of unstructured data makes the problem more urgent.

Justin: We have recursive proof composition – combining fast but large proofs with slow but small proofs to get both speed and compactness. You could apply a recursive AI approach: one AI isolates snippets, another evaluates them.

Shreya: We do something similar, decomposing problems into units different AIs handle well, and measuring accuracy after rewrites.

Christine: Is there a tension between verifiability and efficiency?

Justin: In Ethereum rollups, each flavor trades off differently: optimistic rollups have long challenge periods; zk-rollups require heavy upfront proof building; TEE rollups trust hardware; committee rollups trust members. Energy limits mean we can’t snarkify all AI – so we must be selective.

Shreya: In AI data systems, computation vs. resource cost trade‐offs are constant. DocETL partitions tasks to minimize cost and maximize accuracy – using deterministic functions for simple tasks, reserving LLMs for reasoning.

Justin: That’s like the co‐processor and glue model: specialized processors for repetitive tasks, flexible ‘glue’ for unstructured problems.

Shreya: Yes, for example filtering transcripts deterministically to discard irrelevant text before using LLMs.

Justin: In SNARKs, a team built auto pre‐compiles – machine‐generated specialized processors – which sometimes outperform human‐written ones, even for hash functions. … Blockchains would seem to provide an ideal money for AI because AIs don’t have access to bank accounts, but they can have access to cryptocurrency. When you combine that with the presumption that AIs are going to be so much smarter than we are, it’s only natural to presume that AIs are going to be the richest entities in the world and they’re going to be storing almost all of their wealth in cryptocurrencies. What kind of emotions that does that invoke in you?

Shreya: Giving AI currency could create alignment incentives we currently lack. But what happens if AIs hoard wealth? What can they spend it on? In data analysis, I’d pay more for better AI, but that could be costly.

Christine: Could you hardcode AI to value a cryptocurrency as its ultimate goal, even without things to spend it on?

Justin: Network effects could make such money valuable if other AIs accept it for tasks. I foresee a future where AIs pay humans to do things they can’t.

Shreya: True – AIs will need compute, memory, and human labor for certain tasks.

Justin: Some jurisdictions might grant AIs legal entity status, enabling bank accounts. We’ll see experiments on whether that leads to better or worse outcomes.

Christine: What are some sibling concepts to verifiability?

Justin: In blockchains, reproducibility is key: nodes re‐execute to get the same result. Bugs are mitigated either through client diversity – independent teams producing matching results – or formal verification, proving software has zero bugs. Formal verification is hard for AI, but AIs can help formalize SNARKs by tackling sub‐problems in parallel.

Shreya: In AI, reproducibility is often misunderstood. Non‐determinism isn’t bad – it’s a feature – but we may need statistical claim reproducibility rather than exact output reproducibility.

Christine: Closing thoughts?

Shreya: I’m excited about hints – using them to make verification easier has clear analogs in data processing.

Justin: I’m encouraged by efforts to turn mushy realities into black‐and‐white outcomes. Maybe Shreya’s research could replace referees in sports or escrow agents in marketplaces, solving the last 20% where humans are still needed.