AIChatGPT Is a Demo. Your Users Think It's a Product.
OpenAI put a chat interface in front of a language model on the thirtieth of November. By the end of the first week it had a million users.
I’ve watched this technology develop for a decade and written about most of the papers underneath it. I did not expect this. Not the capability, which is roughly what I’d have predicted, but the speed at which it became a thing that people outside the industry have opinions about.
Three weeks in, here’s what I think is actually happening and what it means for anyone who has to build something.
The Technology Is Not the News
Let me get the deflationary point out of the way, because it matters for planning.
There’s no new capability here. The underlying model is an iteration on what’s existed since 2020. The technique that makes it feel helpful rather than merely fluent is the preference tuning I wrote about in April, where human rankings train a reward model that the language model is then optimised against.
Both pieces were public. What changed is that someone put a text box on the internet and let anyone try it, for free, with no signup friction and no explanation required.
“The breakthrough was the interface. Everything behind it had been sitting in papers for two years, being read by a few thousand people who couldn’t convey what it felt like.” — Sameer Gupta
That’s not a criticism. Making a capability legible to ordinary people is genuinely hard and genuinely valuable. But it means the correct read is not “the technology jumped.” It’s “the technology became visible,” and those require different responses.
The Expectation Shock
Here’s the actual business problem, and it lands in January.
Every executive in your company has now used this. So has every customer. And the experience is extraordinarily convincing: fluent, responsive, apparently knowledgeable about everything, and instant.
That sets a reference point. When someone asks why your support system can’t do that, they’re comparing your product to an experience that had no accuracy requirement, no integration with live data, no audit trail, no liability, and a demo user who was playing rather than trying to accomplish something with consequences.
You will spend 2023 explaining this. I’d start preparing the explanation now.
Why “Just Add a Chatbot” Is Dangerous
The proposals are coming. Some will be good. Here’s what I’d want examined before any of them gets funded.
It is confidently wrong, and the confidence is the problem. It produces plausible, well-structured, authoritative text that is sometimes simply false, with no tonal difference between the two cases. Every previous generation of this technology failed obviously. This one fails invisibly, and a customer cannot tell.
It doesn’t know anything about your business. It knows what was in its training data, which stopped at some point in the past and never included your pricing, your policies, or your inventory. Making it useful about your business is a substantial engineering problem, not a configuration setting.
There’s no source and no citation. When it tells a customer something, you cannot determine where that came from or why. For anything regulated, that’s not a limitation, it’s a blocker.
You cannot test it the way you test software. The same input can produce different outputs. There’s no assertion you can write. Quality assurance for these systems is an unsolved problem and anyone claiming otherwise is selling something.
It will be manipulated. People are already discovering they can talk these systems out of their instructions. If yours can quote a price or make a commitment, someone will make it commit to something you didn’t intend, and they’ll post the screenshot.
“Ask what happens when this system is confidently wrong to a customer in writing. If the answer involves a regulator or a refund, you don’t have a chatbot project. You have a liability project.” — Sameer Gupta
What I’d Actually Do in January
Not nothing. The capability is real and dismissing it would be as wrong as over-committing.
- Start where being wrong is cheap. Internal tools. Drafting that a person reviews. Summarising documents for someone who’ll read the original anyway. Search over your own documentation. All valuable, none catastrophic when it errs.
- Keep a human between the model and the customer, for now, in anything with money or obligations attached.
- Build the evaluation before the feature. Assemble a few hundred real questions with known good answers, and score against them. Without this you have no way to tell whether a change helped, and you will make changes.
- Ground it in your own content rather than relying on what it absorbed in training. Retrieving your actual documents and having the model work from those is the pattern that’s emerging, and it’s considerably more defensible.
- Decide your disclosure position now. I wrote about this when Google Duplex landed. Tell people they’re talking to a machine, at the start, without being asked.
- Write down what you won’t use it for. A short list, agreed by someone senior, saves a great many arguments later.
Final Thoughts
I’ve spent years arguing that this field’s pace is overstated in the press and understated where it matters. This is one of the rare moments where the press is closer to right.
But what changed was accessibility, not capability, and that distinction determines whether your response is sensible. The models will improve. The hard parts, which are grounding, evaluation, accuracy, and accountability, are exactly as hard as they were a month ago and no product launch has touched them.
The companies that do well next year will be the ones that got excited about the interface and stayed disciplined about the engineering. That’s a harder balance than either pure enthusiasm or pure scepticism, and both are going to be well represented in your building in January.