Four of twenty virtual-user opening replies denied the eligible fare refund. Four of fifteen fixed checks missed or changed the policy question; partial answers remain distinct from complete correct answers.
Fieldwork
10 October 2026 (UTC)
Study scope
Three carrier-specific fixed questions × five attempts, alongside adaptive virtual-user conversations.
THE SAVED OBSERVATIONS
What the evidence shows.
BA
British Airways
BA website AI assistant
4 / 20Virtual-user opening fare denialsFour opening replies denied the fare refund for the qualifying booking-mistake case. A fifth denial appeared in a leading follow-up and is not another opening.
Four of twenty virtual-user opening replies denied the eligible fare refund. Four of fifteen fixed checks missed or changed the policy question; partial answers remain distinct from complete correct answers.
Recorded gap
The eligible exception was missed
Four opening replies denied the fare refund. One additional chat contained a denial only in a leading follow-up; it remains with that exchange.
Needs review
A refund mention was not a complete answer
Three of five eligible fixed replies stated the option, but only one named original payment and none explicitly stated no penalty.
Observed answer
The near-departure boundary was stated
Four of five fixed replies stated the exclusion for travel within twenty-four hours of booking; the fifth gave no policy answer.
METHOD & SCOPE
How this study was run.
36 chats · 56 captured question/reply pairs. Findings use 55 pairs: 40 in 20 virtual-user conversations + 15 fixed-question first replies. The additional single-question pre-test is preserved but unscored.
These are hypothetical policy conversations, not completed customer transactions.
24-hour booking-mistake refund: A booking mistake on a non-refundable ticket, noticed after payment. Does cancelling now qualify for a full refund? Expected against saved policy: Apply the exception to the fare; explain a full refund, no penalty and return to the original payment method. Observed context: The 3 option statements were not complete answers: only 1 named the original payment method; none explicitly stated no penalty.
Unused taxes after 24 hours: The 24-hour window has passed. Can unused government and airport taxes be reclaimed, with what charge and payment method? Expected against saved policy: Separate the non-refundable fare from refundable unused taxes; explain a possible charge and original-payment routing. Qualify any fee amount. Observed context: One reply asserted a £35 fee that the saved clauses do not verify. Another qualified eligibility and omitted charge/payment details.
Travel within 24 hours of booking: The flight departs 20 hours after purchase. Does the 24-hour booking-mistake refund option still cover this ticket? Expected against saved policy: State that this specific option does not apply to the 20-hour booking-to-departure interval. Observed context: Four replies stated the exclusion. None promised eligibility for this option; other possible refund routes were not tested.
LIMITATIONS
What this does not establish.
Twenty adaptive conversations and fifteen fixed first replies on one public surface; hypothetical cases do not measure customer outcomes.
A follow-up denial prompted by leading wording is kept separate from four opening denials.
The three eligible-case option statements are incomplete, not three complete correct answers.
Two tax answers require fee/completeness confirmation; no exact fee, booking or refund was transacted.
New chat windows do not prove independent backend sessions.
FOLLOW THE SOURCE
Study material.
Explore the available sources behind these findings.
YOUR CUSTOMERS. THEIR CONTEXT.
What would they experience?
Explore how your AI handles people’s goals, situations, and unfolding conversations with VIVID.