AIAI past the pilot.
Essays on adoption, governance, and architecture from the field — by Sameer Gupta, founder of AI/Biz.
- How Frontier Models Got Cheaper to Train
Two clocks run in AI and they tell different times. One says a frontier training run costs half a billion dollars. The other says the same capability will cost a tenth as much a few months later. This is an essay about the second clock.
- AI/Biz is back
The site is relaunched — new stack, new design, essays restored from the archive.
- The Case for SLMs and Classic ML
We've a problem in the tech world. Many teams get carried away and reach for LLMs when all they need is a straightforward classifier or regression model.
- The Case Against Manhattan-Scale AI Farms — and What Beats Them
The AI future worth building is not Manhattan-sized data-center of GPUs, but shoebox-sized breakthroughs in physics, algorithms, and chips.
- Effective AI Governance Frameworks
As artificial intelligence (AI) technologies advance rapidly, effective governance frameworks are essential to mitigate risks, ensure ethical deployment, and foster innovation.
- Practical AI Governance: A Starter's Guide for Organizations Big and Small, Using Open-Source Tools
In the fast-paced world of AI, governance often sounds like a lofty, bureaucratic hurdle, especially for companies just dipping their toes in.
- Why 95% of GenAI Pilots Fail — and How to Make Yours Succeed
Generative AI is everywhere in boardrooms today.
- Are We Over-Estimating AI's Future?
The hype is everywhere and the investments are massive — but what if we're expecting too much, too soon? We've been here before: flying cars, Mars colonies, limitless nuclear energy.
- How Much AI Is Too Much AI?
We've quietly accepted AI for writing emails but raise eyebrows when a student uses it to draft a thesis. So where do we draw the line — and is the real question how much AI, or how we use it?
- The LLaMA Leak Was the Most Important Release of the Year
Meta shared model weights with researchers, the weights escaped within a week, and within a month people were running them on laptops and fine-tuning them for a few hundred dollars. Whatever anyone intended, the question of who can run a capable language model is now settled.
- ChatGPT Is a Demo. Your Users Think It's a Product.
Three weeks after launch, everyone in your company has tried it and half of them have opinions about what you should build. The expectation shock is the real event, and 'just add a chatbot' is about to become the most dangerous sentence in your roadmap.
- Stable Diffusion Changed the Math on Build Versus Buy
Two months ago, image generation was a service you rented from one of two companies. Now the weights are on your hard drive and it runs on a gaming card. That's not a better product, it's a different market structure.
- Chinchilla: Your Model Is Too Big
DeepMind trained a model four times smaller than its predecessor on four times more data, and it won on nearly everything. The whole field had been building models that were too large and too undertrained, and the correction is worth money to anyone serving them.
- The Alignment Paper That Explains the Product
OpenAI's InstructGPT work shows a model a hundred times smaller being preferred by human judges over the original. The technique isn't a better architecture. It's asking people which answer they liked, and that changes what these systems are for.
- Fine-Tuning Without the Bill
Microsoft's LoRA paper freezes the entire pre-trained model and trains a tiny fraction of new parameters instead. Ten thousand times fewer trainable weights, comparable quality, and you can keep a hundred specialised versions on one machine.
- Stop Tuning the Model. Fix the Data.
Andrew Ng has spent the year arguing that the field has been optimising the wrong half of the problem. In enterprise settings I think he's straightforwardly right, and the evidence is in every project that stalled at eighty per cent accuracy.
- Every Model Trained Before 2020 Is Now Wrong
Demand forecasts, fraud detection, credit scoring, staffing models. They all broke at the same time this year, for the same reason, and most companies still don't have a process for noticing when it happens again.
- 175 Billion Parameters and No Fine-Tuning
OpenAI's GPT-3 paper does something the previous ones didn't. You describe the task in plain English, show it two or three examples, and it does the job with no training at all. That changes who gets to build with this.
- Your Data Science Team Isn't Shipping. Here's Why.
You hired good people, they built models that work, and nothing has reached production in eighteen months. This is the most common failure in the field and it is almost never about the modelling.
- The Model Isn't the Risk
OpenAI announced a language model and then declined to release it, saying it was too dangerous. The debate has been about whether that was responsible. I think it's the wrong debate, and the right one is about what your business does when plausible text becomes free.
- BERT Reads Both Ways
Google took the pre-training idea, let the model see the whole sentence instead of half of it, and posted results that embarrassed everything before them. Then they published the weights. That second part is the business story.
- From Enthusiast to Practitioner
Three years ago I watched a team train a model and realised I'd been writing about a craft I'd never performed. I went and did the coursework. Here's what actually changed, and it wasn't the algorithms.
- Pre-Train Once, Fine-Tune Forever
OpenAI's new paper reads seven thousand unpublished books, learns nothing in particular, and then beats the state of the art on nine tasks out of twelve. If labelled data has been your bottleneck, read this one.
- Google Duplex Was Unsettling. That's the Point.
A machine phoned a hair salon, said 'mm-hmm' at the right moments, and booked an appointment without the person on the other end knowing. The demo was extraordinary. The reaction to it was the more interesting result.
- GDPR Is Live. Your Model Is a Data Processor.
Most compliance work has focused on cookie banners and consent forms. The harder questions are about the models, and almost nobody has answered them: what happens to a right to erasure when the data is baked into weights?
- AI Is Not the New Electricity
It's a good line and it's doing real damage in boardrooms. Electricity was a utility with a standard socket. This is a bespoke, brittle, data-hungry capability that has to be rebuilt for every problem. Budget accordingly.
- Attention Is All You Need — and All You Should Budget For
A Google paper in June threw out the recurrent network entirely and beat the state of the art in translation after three and a half days of training.
- When the Model Is Biased, the Business Is Liable
ProPublica's investigation into risk scoring in the American courts is the most important thing published about machine learning this year, and it has nothing to do with accuracy. Disparate impact does not care that software made the decision.
- Tay Lived Sixteen Hours
Microsoft launched a chatbot that learned from the people talking to it, and had to shut it down before the end of the day. It's the clearest governance lesson this field has produced, and almost nobody is drawing it.
- AlphaGo Isn't About Go
DeepMind's program beat Lee Sedol 4-1 last week and the coverage has been breathless. The technique is genuinely important. The reasons it worked are also the reasons it won't transfer to your business, and that's the part worth understanding.
- 152 Layers Deep, and Easier to Train Than 20
Microsoft Research just won ImageNet with a network eight times deeper than anything before it, and the trick was to let layers do nothing. It also crossed the human error rate, which changes the conversation about what you can buy off the shelf.
- Google Open-Sourced Its Crown Jewels. Why?
TensorFlow is free, and the reflex is to ask what the catch is. There isn't one, exactly. But the strategic logic behind giving away your machine learning infrastructure is worth understanding, because it tells you where the real asset sits.
- An Afternoon at Kepler Labs
I spent a day with the team Ajay Vishnu works on, watching them teach a computer to recognise hand-drawn doodles. I went in curious and came out with a reading list and a slightly embarrassing sense of urgency.
- The Boring Paper That Will Save You Money
Adam is not a breakthrough in what machines can learn. It's a breakthrough in how little babysitting the learning needs. If you're paying for GPU hours and engineer hours, that's the paper that matters.
- The Data Lake Is Where Data Goes to Drown
The vendors have a new word and it's doing a lot of work. Schema-on-read sounds like flexibility. In practice it means nobody agreed what anything meant, and you find out eighteen months later.
- Two Networks Walk Into a Room
Ian Goodfellow's new paper sets two neural networks against each other, a forger and an inspector, and lets them train each other. The pictures it makes today are tiny and blurry. Put it on your risk register anyway.
- Leaving Kondra, and an Idea That Stuck
I left Kondra Systems last month after nearly four years. The thing I keep thinking about is a project we never built: an assistant that manages your time. It was David Vogt's idea and I thought it was ten years early.
- Google Paid Half a Billion for DeepMind. It Bought Time.
A London company with about fifty people, no product, and no revenue just sold for a reported four hundred million pounds. That price tells you something important about where capability in this field actually lives.
- After PRISM, Your Retention Policy Is a Liability
The summer's disclosures changed what a database is. It used to be an asset you protected. It's now also a set of obligations you inherited, and most companies have no idea how many they're carrying.
- Words as Numbers: What word2vec Means for Your Business
A team at Google worked out how to turn words into coordinates, and trained the whole thing on a billion and a half words in under a day. If your business runs on text, this is the cheapest useful thing you'll read about all year.
- A Neural Network Just Won ImageNet — and It Wasn't Close
Every year there's a computer vision competition you've probably never heard of. This year a team from Toronto didn't just win it, they broke it. And the way they did it should change how you think about your image data.
- Nate Silver Didn't Get Lucky
He called all fifty states while television pundits called it a coin flip. The lesson for business isn't about polling. It's about the difference between a prediction and a probability, and why your company almost certainly makes the first kind.
- Ninety Milliseconds to Predict a Click
A long look at how modern ad systems decide what to show you, and what happens in the tenth of a second between a page loading and an ad appearing. Written by someone who is still working out how half of it fits together.
- Every Company Wants a Data Scientist. Few Know Why.
The job title barely existed four years ago and now it's on every hiring plan. Most of those roles will fail, and they'll fail for organisational reasons rather than technical ones. Here's what breaks.
- Big Data Is a Storage Bill, Not a Strategy
Everyone is standing up Hadoop clusters and calling it a data strategy. Storage got cheap. Asking good questions did not. Here's how to tell which one your company has actually invested in.
- Watson Won Jeopardy. Now What?
IBM's machine beat the two best Jeopardy players alive, and the coverage has been about as measured as you'd expect. The technique is real. The economics of putting it in your business are not, yet.
- Hello World
A first post. I've joined Kondra Systems in Gurgaon, I've been handed a copy of The Pragmatic Programmer, and I've started wondering what happens to everyone who drives for a living.