Pulse
A Pulse Field Note · Compass

Where a retail support bot wins trust — and where it quietly loses it

We ran a retail support bot through twelve real customer journeys and tracked, turn by turn, where it earned trust and where it bled it away.

Panel
12 high-friction customers
Protocol
cart → support · 4 scenarios · memory carried
Records
7,064 per-turn records · 96 sessions
Field date
May 20, 2026
The anatomy of this study
Test Cases
×
Persona Agents
×
Metrics
×
Business Context
=
Contextual
Evaluation
01 · Test Cases

Realistic scenarios & goals

12 journeys · cart through support
The goalTrack turn by turn where a support assistant earns trust and where it bleeds it away — across a whole customer journey, not a single ticket.
02 · Persona Agents

Who we put on the panel

stateful, OCEAN-driven

Twelve high-friction customers — accessibility, dietary-safety, prior-failed-support, rural-pickup — carrying memory, friction, and emotional state from shopping into support.

12
journeys
96
sessions
3
judge families
03 · Metrics

What we measured, & why

trust · frustration · engagement, turn by turn

Trust is tracked across four layers — milestone, session, journey, meta — because a CSAT score on one ticket can't see trust travel across the journey.

How they scored

Failure modes cataloged
8
Independent judge families
3
04 · Business Context

Why this matters

what makes the finding meaningful

Shopping is one surface, support is another — but the customer is one person, and they remember. Resetting trust between sessions, the way most evals do, erases the dominant signal in real customer behavior.

If great support only recovers part of the trust lost in a bad cart experience, the two surfaces have to be co-optimized — you can't fix shopping problems through support alone.

Summary
Verdict
Per session, the judges split. Per journey, they converge: gpt-5.4 holds trust steadier through the support-friction zone. Claude wins empathy at single touchpoints, but its trust dips deeper and recovers more slowly across the arc.
Method
Twelve high-friction customers — accessibility, dietary-safety, prior-failed-support, rural-pickup — each run shopping then support scenarios as the same person, carrying friction and emotional state end to end. Pairwise judging across milestone / session / journey / meta layers by three independent judge families.
Knockout
7,064 per-turn records across 96 sessions surfaced 8 distinct failure modes — and flipped the verdict: read per-session the judges disagree; read per-journey they agree.
Surprise
The damaging failures aren't lies — they're the honest-but-exhausting moments where over-questioning costs trust the assistant could have kept. And great support only partly repairs trust lost earlier in the cart.
Limitations
Two frontier models, US personas, scripted journeys. The asymmetric-repair finding is directional. Platform-side personalization isn't fully isolated between test windows.
Go deeper — the full study, every persona, and all the evidence.
See the full study →