Own the learning loop and rent the model underneath it

Satya Nadella posted a line that I keep coming back to: a frontier without an ecosystem is not stable. I read it as advice for any company building with AI. The best model changes every few weeks, so a strategy built on picking the best model is a strategy you will be rewriting all year.

What lasts is a learning loop on top of the models, where your people and your AI get better together on your own work. Each time someone corrects an output or explains why an answer was wrong, that knowledge should land somewhere the next model can use. If it disappears into a chat window, the company has paid for the lesson and kept none of it.

Two kinds of capital

I find it useful to think about two kinds of capital. Human capital is the judgment and pattern recognition your people have built up over years in your business. Token capital is the AI capability your company builds and owns: the prompts, the retrieval, the evaluations and the workflows that encode how your company does its work.

The test for whether you own any token capital is blunt. Could you rip out the model underneath and keep the expertise you built on top? If the answer is no, what you have is a dependency on someone else’s model, with your company’s knowledge locked inside prompts that only behave on that one model.

The model layer is becoming replaceable

The major clouds already treat models as parts you can swap. Azure AI Foundry has a model router. AWS Bedrock gives you many models through one API, with automatic prompt routing. Models also change and get replaced on the provider’s schedule, which you do not control.

The practical rule is to avoid hardwiring your business to one model. Put an abstraction between your workflows and the model, route each request to a model that does the job well enough for the price, measure the results continuously, and be ready to swap when something better or cheaper arrives. None of this has to be elaborate on day one, but the seam has to exist before you need it.

What sits under the router

The durable value is everything under the router, the parts that stay when the model changes. This is the list I would want a company to be able to point to:

• Your evals. The test sets and scoring that tell you whether a new model is actually better at your work specifically.
• Your traces. The record of what the system did on real cases, which is the raw material for every improvement you make later.
• Your workflows. The steps and approvals that turn a model’s output into finished work.
• Your institutional knowledge. Your documents and past decisions, plus the corrections your experts make along the way.
• Your governance and your definition of good. Who approves what, and the written standard a result has to meet before anyone relies on it.

Most companies today have some pilots and a ChatGPT login. Neither accumulates much when the model changes, because the knowledge sits with individuals and the vendor. Starting the loop usually means picking one workflow and writing down what good looks like before you build, then keeping every trace and correction from the first day.

My opinion is that this is where an AI budget should go first. A team that has its own evals can test a new model against its own cases and know whether to switch. A team without them has to take the provider’s word for it, or run a fresh pilot every time something new ships.

The caveat is that abstraction is not free. Models are not perfectly interchangeable, and a prompt tuned for one can behave differently on another, so swapping without evals can make things quietly worse. A router is also one more component to run and secure. For a single narrow use case, calling one model directly can be the right choice for now, as long as you keep the traces and evals that would let you move later. If you want to talk through where your own loop would start, get in touch.