Why Mock Arbitrations Persist: The Rational Market for an Irrational Tool

In an earlier piece, we argued that mock arbitrations fail as predictive instruments. This article addresses the natural follow-up: if the method doesn't work, why do parties still pay substantial sums for it?

American trial practice abandoned informal surrogate-group testing decades ago, starting in the mid-1970s when David Boies, then a young Cravath partner, commissioned what is credited as the first systematic scientific jury research by a corporate defendant, in IBM’s antitrust defense. Scientific medicine made the same transition earlier; arbitration never did.

The easy explanation is that arbitration lawyers are behind the times, but that isn't remotely true. The lawyers running high-stakes international arbitrations sit atop the profession, and many adopted jury research without hesitation on the litigation side. The real question is why these same sophisticated lawyers choose differently once the forum shifts from jury to tribunal.

We think the answer lies with the market, not the lawyers. High-stakes legal services are a “credence good”—quality can’t be verified even after the service is consumed. A general counsel (“GC”) who loses a $500 million arbitration cannot separate bad lawyering from bad luck, so buyers fall back on proxies: prestige becomes that proxy, and mock arbitrations sit on top of it. That's why they persist.  But this situation isn’t inevitable.

A more reliable alternative now exists: rigorous behavioral research combined with structured empirical testing of how a tribunal’s cognitive mechanisms actually operate, measuring scientifically what mock arbitrations only pretend to. The question is why the market hasn’t switched.

The GC's Real Decision Problem

In major disputes, the GC pursues two goals at once: winning the case and protecting their own professional standing. These usually align, but diverge exactly when stakes are highest and the path forward is unclear, turning winning and self-protection into separate problems.

The behavioral economics are simple. A GC's reference point is their current standing, and losses from it hurt roughly twice as much as equivalent gains feel good. This is a ratio Kahneman and Tversky documented and anyone who has watched corporate politics will recognize. A loss from bad facts or an unpredictable tribunal is survivable; a loss from the GC's own judgment is not.

Picture this: six weeks before the hearing, outside counsel concludes the damages theory built over nine months won’t hold up and recommends reframing. The original approach was approved by the CEO and CFO, on the GC’s advice, at seven-figure cost. The GC must now explain the change to the executive suite: who signed off, what are the new numbers, and whether the reason for switching sounds like more than gut instinct.

In that moment, which approach can be defended internally if it fails matters more than which is more likely to win. A pivot grounded in rigorous research can be defended; one grounded in intuition cannot. The CFO doesn't want a “feeling”; the CEO wants evidence.

This is also why hiring a white-shoe firm is rational even when not empirically optimal: its prestige is “social proof” that solves the GC's real problem—internal defensibility. Mock arbitrations inherit the same logic: their weaknesses are invisible to internal audiences, even though the exercise itself is visible in budgets and memos. Buying one is the rational move given the incentives a GC faces.

What the GC is rewarded for is visible conformity to the preparation rituals their peers use. This article asks them to trade that for something that actually works.

Why Firms Resist Scientific Arbitration Testing

The supply side has a parallel problem, and it isn't really about cost or logistics.

Litigators systematically overestimate their own cases’ strength; research shows the gap between predicted and actual outcomes doesn’t close with seniority. The “Wyatt Earp effect”—the belief, from beating the odds a few times, that invulnerability has become real, sets in, and sunk-cost reasoning compounds it, keeping weak cases alive past the point a detached analyst would recommend settlement. The team that built a theory is not best positioned to judge whether it still holds.

Behavioral research counters these tendencies, and this is the reframe that matters for the GC. Rigor is protection against cognitive biases counsel cannot fully correct from inside, not a second opinion on their judgment. A mock arbitration run by the firm that built the theory can't provide that counterweight; an independent empirical inquiry into the strategy's cognitive foundations can.

The medical parallel holds: no surgeon refuses diagnostic imaging on the grounds that it questions their expertise, and no surgeon's standing suffers when imaging reveals something unanticipated. Why does the reverse standard apply to elite arbitration counsel?

What Rigorous Research Actually Delivers

If GCs take the path we recommend, what does this research produce?

For the GC, the payoff is a defensible evidentiary foundation for the hardest decisions that are hardest to defend: pivots that can't be justified by pointing to what worked last time. If a case is lost after such a pivot, the GC can show the board it occurred early, saved cost, and rested on evidence rather than intuition. The loss then sits with the tribunal’s uncertainty, not the GC’s judgment.

More importantly, rigorous research does what mock arbitration is marketed as doing but cannot: it produces findings that generalize, rather than showing only how one surrogate group reacted on one day. It reveals how the cognitive mechanisms an argument depends on actually work: whether the framing activates the intended heuristics, whether the damages theory survives the tribunal’s reasoning, whether the narrative holds up against how decision-makers process disputed evidence. The mock gives a test performance; empirical research gives data about the minds that will decide the case.

For the firm, diagnostic rigor becomes a mark of sophistication rather than weakness. Leading medical practices made this transition a generation ago, and the trial lawyers who adopted jury research earliest redefined prestige rather than losing it. The same move is available in arbitration.

Signaling and the Bad Equilibrium

Why hasn't the market replaced mock arbitrations with rigorous behavioral research? The equilibrium is stable for reasons that have little to do with the merits.

Firms signal quality through observable inputs, like prestige, price, offices, lateral hires announced in the trade press, because quality itself can’t be observed; signaling is how elite legal services are sold. Bringing in outside behavioral experts feels, from inside a firm, like importing authority from a discipline it doesn’t own.

The feedback loop runs both ways: clients reward prestige because they can't see quality, and firms invest in prestige because clients reward it. Mock arbitrations persist as visible, if unreliable, evidence that preparation happened.

The equilibrium has lasted because it’s stable, but stable and good are different things: the result is a market that cannot reliably predict what will work. Nobody needs to be foolish for the system to produce foolish outcomes.

The Diffusion Pattern

The cleanest analogy is evidence-based medicine. When it emerged in the early 1990s, largely through David Sackett and colleagues at McMaster, clinicians objected that experience beats data and judgment can’t be reduced to statistics. Within a generation, EBM went from heresy to orthodoxy, as did similar shifts in agriculture, finance, and marketing.

These transitions share a shape: diffusion-of-innovation research finds that once early adopters reach roughly a fifth of a market, the curve steepens—remaining holdouts convert under competitive pressure rather than persuasion, because the old practice has become indefensible to their own clients and peers.

Defending mock arbitrations in 2035 will sound like defending evidence-free medicine sounded in 2005. The window for early-adopter status is finite.

Conclusion: The Case for Replacement

The argument for change doesn't rest on any claim that GCs and firms resist science. They are rational actors in a market shaped by opacity, asymmetric reputational risk, and signaling incentives that make prestige a better investment than rigor. Reframing the incentives changes the system.

When disputes routinely involve hundreds of millions of dollars, a preparation method that cannot reliably predict outcomes is a systemic market failure, not an intellectual failure of its participants. Fixing it serves everyone, including the firms whose incentives currently prevent them from fixing it.

The correction is specific and available now: replace the mock arbitration with rigorous behavioral research. Stop paying for a dress rehearsal before a surrogate panel whose reactions generalize to nothing, and start paying for structured empirical testing of the cognitive mechanisms the case depends on. The methodology has been mature on the American trial side for nearly fifty years, and its findings are defensible to boards, CEOs, and audit committees.

The leverage sits with early adopters on both sides. GCs who commission this research before their peers gain a defensible foundation for their hardest decisions and a method that actually predicts outcomes. Firms that adopt it as a differentiator gain prestige that will survive a transition their industry hasn’t made yet—but will.

That transition is coming, as it came for medicine, finance, and agriculture, whose practitioners once believed judgment couldn’t be reduced to data and discovered, a generation later, that data had reshaped judgment itself. Arbitration is next; the only question is who moves first.

The concepts presented here and in the companion article are explored in greater depth in the forthcoming Kluwer publication Success in Arbitration: Winning with Applied Psychology, scheduled for December 2026