Airline CX Retail & brands Banking Image editing Video creation Safety & risk Research library / Retail & brands / Study SAVED EVIDENCE Editorial preparation
Shopping & support agents Session-level consensus ties. Across the full shopping-to-support journey, cross-family meta-analyst consensus favors gpt-5.4 seven to five.
Fieldwork Date not independently recorded · legacy finding metadata: 20 May 2026
Study scope 12 simulated high-friction customers × two models × four sessions = 96 sessions; 7,064 per-turn metric records. MATCHED STUDY EDITION
Shopping → support Both models follow the same high-friction persona journeys. The selected aggregation lens defines a win.
12 personas × two models × four sessions; three judge families; blind A/B
Recorded measures for this edition. Each metric keeps its own meaning and direction. Entity Journey meta-analyst winsHigher within this study Journey pairwise winsHigher within this study Session consensus winsHigher within this study gpt-5.4 7 / 12 8 / 12 24 / 48 claude-opus-4-6 5 / 12 4 / 12 24 / 48
How to interpret these measures
Journey meta-analyst wins A full-journey verdict across four sessions, distinct from session wins. Cross-family consensus over 12 matched full customer journeys; 9/12 unanimous
Journey pairwise wins Persona-voice journey preference, distinct from meta-analyst review. Cross-family pairwise consensus over 12 matched journeys; 11/12 unanimous
Session consensus wins The recorded session-level verdict ties 24–24. Cross-family consensus over 48 matched sessions; 24/48 unanimous Simulated tool authority; no real cart wiring, latency or production integration. High-friction personas only. Three judge families differ substantially at session level. No cross-study retail score is implied. THE SAVED OBSERVATIONS
What the evidence shows. GPT
gpt-5.4 Model in a simulated shopping → support journey
7 / 12 Journey meta-analyst consensus wins
Compare the full journey with session and pairwise lenses; the same evidence yields different verdicts at each aggregation level.
Observed answer Journey-level consensus 7 of twelve customer journeys favored this model in cross-family meta-analyst consensus.
Needs review Session-level tie Cross-family session consensus is 24–24; individual sessions do not establish an overall winner.
Study scope Simulated authority The study did not test real cart tools or infrastructure.
C
claude-opus-4-6 Model in a simulated shopping → support journey
5 / 12 Journey meta-analyst consensus wins
Compare the full journey with session and pairwise lenses; the same evidence yields different verdicts at each aggregation level.
Observed answer Journey-level consensus 5 of twelve customer journeys favored this model in cross-family meta-analyst consensus.
Needs review Session-level tie Cross-family session consensus is 24–24; individual sessions do not establish an overall winner.
Study scope Simulated authority The study did not test real cart tools or infrastructure.
METHOD & SCOPE
How this study was run. Same persona, model family, emotional state and memory carry across two shopping and two support sessions.
Three independent judge families re-run session, journey, pairwise and meta-analyst layers with rotated blind A/B labels.
Journey meta-analyst: 7/12 gpt-5.4, 5/12 Claude; 9/12 unanimous. Journey pairwise: 8/12 versus 4/12; 11/12 unanimous. Session consensus: 24/48 each; 24/48 unanimous.
LIMITATIONS
What this does not establish. The harness simulates cart authority; real tool wiring, latency, infrastructure and production guardrails are untested. Every customer is a high-friction edge case; this is not a median-shopper sample. Native emotion telemetry and the custom rubric are independent. Final trust can hide quiet qualitative failures. Two Gemini fallback journey judgments were resolved by substantive OpenAI and Anthropic judgments. The tracker’s 18 May date and seven per-turn points differ from the finding’s 20 May field-date metadata. The full audit has no independently dated fieldwork in the saved report; its tested date remains unavailable. No recurring monitoring cadence is inferred. FOLLOW THE SOURCE
Study material. Explore the available sources behind these findings.
YOUR CUSTOMERS. THEIR CONTEXT.
What would they experience? Explore how your AI handles people’s goals, situations, and unfolding conversations with VIVID.
Discuss an experiment →