DTCx12 Global

DTCx12 Global was Gorgias's free virtual summit for ecommerce teams getting ready for Black Friday and Cyber Monday, held on October 7. Simplesat sponsored it, and Owen McKee, our Sales Director, had a 20-minute slot with a talk Gorgias titled "AI agents, customer feedback and the continuous business improvement loop." ShipBob, JAXXON, Obvi, Zipify, Woolman, Consio and Inevitable Agency were also on the program, with Jess Cervellon of Open Late Collective in the headline slot. The recording of Owen's session runs 19 minutes.

Owen started with what AI agents are already doing well. They're taking first contact, faster and at volume. What concerned him was what gets measured next. "There's a risk there that we become so AI focused that we're actually just measuring the AI," he said, "and we're not actually identifying whether we're really satisfying the customer."

The distinction he drew was between two kinds of truth. Internal QA, and now AI QA, checks whether the agent ticked the right boxes and gave the right answer. "That's the internal truth," he said. "But the customer is actually the external truth." His example set the two side by side: a 96% AI score next to a 2 out of 5 from the customer. An interaction can pass every internal check and still leave the person unhappy, and the only way to see that is to ask them. Or, in his words, "The customer can QA your AI."

He didn't treat that gap as a failure. "You shouldn't be frightened about the fact that there is a disconnect, because that's probably the best possible learning opportunity that you're going to get." He walked through where that disconnect comes from. A customer asks where the parcel is and gets a tracking number, when what they wanted to know was whether it would arrive before the weekend. An outdated knowledge base that may not reflect a special promotion, leaving the AI without current information. An escalation that comes too late: the day before the talk, Owen had been through a mobile provider's bot that kept failing to understand him until it handed him to a person, who resolved it quickly. Even so, "it still left me with that negative feeling."

The second half of the talk was about where the answer goes once you have it. Most feedback today sits with customer success, Owen said, but the cause of a bad score is usually somewhere else in the business. His example was a 43% spike tied to one product. The AI can flag it. It can't tell you that the new product launched with a user manual that wasn't clear enough, or that a special offer was out of stock by the time people could buy it. "The score is a signal, but the conversation is the intelligence," he said. He wants that conversation routed automatically to product, marketing, operations and leadership, as an early warning system rather than a weekly score check.

He laid the rest out in three questions. Most teams are at "what's our score?" The next question is why, and whether the business holds enough data to answer it. The last is what to do differently, and whether the change worked. One of his customers moved from a 45-day returns policy to 30 days and saw a spike right away, which is the kind of internal decision that needs tracing through feedback rather than guessed at. That cycle, run continuously rather than as a single step, is the loop in the title.

Our place in that, as Owen put it, is the asking. The AI agent is the execution layer and AI QA is the evaluation layer. Simplesat asks the customer, in the channel they used, whether that was SMS, chat, email, voice or an inquiry on the website, and consolidates the answers so the rest of the business can act on them.