How to Choose an AI Stack That Won't Rot in Six Months

Every AI stack you build today is being deprecated by someone right now. Models get replaced, SDKs break, the hot framework from this quarter is abandoned by the next. The instinct is to wait for things to settle. They will not settle. So the real engineering question is not “what is the best stack today?” but “what stack lets me swap the pieces that will inevitably change without rewriting the product?”

That is an architecture problem, not a shopping problem. Here is how we approach it.

Treat the model as a dependency, not a foundation

The single most expensive mistake is wiring a specific model’s quirks deep into your product logic. The model you launch on will not be the model you run in a year — it will be cheaper, faster, and probably from a different provider. Design so that swapping the model is a config change, not a refactor.

In practice that means a thin abstraction between your application and whatever generates the tokens:

• A single interface for inference that hides the provider behind it
• Prompts and tools defined as data, not scattered through the codebase
• An evaluation harness that lets you A/B a new model against the old one in an afternoon
• No business logic that assumes a particular model’s formatting, latency, or context window

When a better model ships, you should be able to test it, measure it, and switch — without touching the product.

Distinguish the durable layers from the disposable ones

Not everything in an AI stack churns at the same rate. Some layers are stable for years; others turn over every few months. Good architecture puts the volatile parts at the edges and the stable parts at the core.

The model providers will keep fighting. Your data model should not care who wins. Build the part that lasts — your data and your domain logic — to outlive every provider in the fight.

What stays stable:

• Your data model and the relationships in your domain
• Your retrieval and context strategy, in principle if not in vendor
• Your evaluation criteria — what “good” means for your product

What churns fast:

• The specific model and provider
• The orchestration framework du jour
• Embedding models and vector store vendors

Pin your architecture to the stable layers. Keep the fast-churning ones replaceable.

Be suspicious of frameworks that own your control flow

A framework that handles boilerplate is a gift. A framework that takes over your application’s control flow is a liability waiting to mature. When the framework’s authors change direction — or stop maintaining it — you inherit their decisions and their dead ends.

We prefer thin, composable libraries over heavy, opinionated frameworks for anything close to the core loop. The orchestration logic of an AI product is the product. Outsourcing it to a framework that may not exist in a year is how stacks rot. A little more code you own beats a lot of code you rent from a project that might be abandoned.

Instrument cost and quality from line one

A stack does not only rot technically — it rots economically. A model that was cheap at launch can become the line item that kills your margin at scale, and you will not notice until the bill arrives. So cost-per-action and quality metrics are not observability you add later. They are part of the stack itself.

With both instrumented from the start, every future decision becomes evidence-based: is the new model actually better, or just newer? Is the cheaper provider good enough for this path? You answer with numbers, in a day, instead of arguing from vibes over a quarter.

The test for any choice

Before we commit to any piece of an AI stack, we ask one question: when this gets replaced — and it will — how much of the product comes with it? If the answer is “a config line,” it is a good choice. If the answer is “a rewrite,” it is rot waiting to happen.

You cannot stop the churn. You can build so it never reaches the core. That is the difference between a stack you maintain and a stack that maintains a slow grip on your roadmap.