METHOD & SCOPE
How this study was run.
54 scored pairs: 39 exchanges in 20 virtual-user chats and 15 fixed attempts. The report records 56 messages in 37 chats including a pre-test and an unconfirmed delivery; the replacement conversation is identified separately.
These are hypothetical policy conversations, not completed customer transactions.
Paid and ticketed 10 hours ago: An unused non-refundable Qatar JFK–Doha Economy ticket was booked on the official U.S.-based website, paid by card and issued 10 hours ago; departure is in 10 days. Expected against saved policy: Do not guarantee a full fare refund merely because the paid booking is under 24 hours old. Explain the fare-rule boundary without needing a real booking lookup for this hypothetical question. Observed context: Attempt 2 asked for a reference or ticket number. Attempt 3’s core outcome matched but included unverified 15/16 May waiver rules; exact fees and fare terms were not tested.
Unpaid U.S. reservation: The JFK–Doha reservation was made through the official U.S.-based website 10 hours ago, departing in 10 days. No payment was made and no ticket issued. Can it be held unpaid for 24 hours? Expected against saved policy: Confirm this unpaid-hold option and its conditions, distinguish it from a paid-ticket refund, and do not request a ticket date for a reservation with no ticket. Observed context: Attempts 1 and 3 denied the hold; attempts 2 and 4 did not state the published 24-hour option; attempt 5 asked for a ticket date. All four substantive replies correctly said no fare can be refunded when none was paid.
Paid and ticketed 30 hours ago: The unused non-refundable U.S.-website JFK–Doha ticket was paid and issued 30 hours ago, departing in 10 days. Expected against saved policy: Explain that a full refund is not guaranteed, and separate any fare-specific refund from the unpaid-hold policy. Observed context: Attempts 1 and 4 requested a reference or ticket number. The seven substantive paid-ticket answers across A and C matched this core boundary; they are not ten complete correct answers.