> ## Content Index
> Fetch the complete content index at: https://www.davidcost.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# What Questions Should an Executive Ask Before Approving an AI Implementation?
- URL: https://www.davidcost.com/what-questions-should-an-executive-ask-before-approving-an-ai-implementation/
- Published: 2026-09-10T14:00:29.000Z
- Updated: 2026-09-10T14:00:31.000Z
- Description: Executives don’t need to predict which AI model will win. They need to ensure their AI implementations can adapt. Here are the questions to ask about model routing, abstraction and evals before approving an AI investment.
- Author: David Cost
- Tags: Enterprise AI, AI Architecture, AI Transformation

### How model routing, abstraction and evals can preserve flexibility as the AI landscape changes  

**Before approving an AI implementation, executives should ask whether the organization can change the underlying AI models without rebuilding its applications, use different models for different tasks, keep prompts and enterprise context portable, and evaluate new models against its own requirements. The goal isn't to predict which AI model will win. It's to preserve the organization's ability to choose.**

## The Best AI Model Will Keep Changing  

If you're making a significant enterprise AI investment, there is an important question to ask before approving the implementation:

**How hard will it be to change the AI model?**

The AI model landscape is changing incredibly quickly.

A year ago, many people assumed OpenAI would emerge as the dominant model provider. Today, Anthropic has enormous momentum. A year from now, the landscape may look different again.

Your implementation shouldn't require you to predict the winner.

Yet that is effectively what an organization does when it builds applications, workflows and enterprise knowledge too tightly around a particular model or model provider.

This isn't simply about avoiding vendor lock-in. It's about maintaining the flexibility to take advantage of a technology market that is likely to continue changing rapidly.

Two concepts are particularly important: **routing and abstraction.**

## What Is AI Model Routing?  

At its simplest, model routing allows an AI application to use different models for different tasks rather than sending every request to the same model.

A router sits between the application and the available models. When a request arrives, the router determines where that request should go.

That decision can be based on factors including:

- capability
- cost
- latency
- task complexity
- specialization
- security or governance requirements
- availability

A complex reasoning problem might go to a highly capable frontier model.

A simple, high volume task might go to a smaller and less expensive model.

A specialized task might go to a model designed for a particular domain.

If the preferred model is unavailable, the request might be routed to another provider.

And when a better model emerges, the organization has the ability to begin routing appropriate work to it.

Conceptually, routing changes the system from:

**Application → Model**

to:

**Application → Router → Appropriate Model**

The important idea for an executive isn't how the routing algorithm works.

It's that **the model becomes a choice rather than a permanent assumption.**

## Routing Alone Isn't Enough  

There is an important complication.

An AI implementation consists of much more than the model generating the answer.

Over time, organizations accumulate prompts, enterprise context, proprietary knowledge, business rules, tool integrations, workflows and other application logic.

If all of that becomes tightly coupled to the behavior or interface of a particular model, technically connecting to another model doesn't necessarily make the implementation portable.

You may have access to five different models and still find that changing from one to another requires substantial work.

This is where **abstraction** becomes important.

## What Does Abstraction Mean in AI?  

Abstraction is a familiar principle in technology: separate one layer of a system from another so that components can change without requiring everything around them to change.

Applied to AI, the objective is to separate as much of the organization's context, prompts and application logic as practical from the particular inference engine being used.

Think of it as separating:

**What your organization knows and what it wants done**

from:

**Which AI model happens to do the work.**

Conceptually, an implementation might look something like this:

**Application / Workflow**  
↓  
**Enterprise Context + Prompts + Business Logic**  
↓  
**Abstraction / Routing Layer**  
↓  
**Model A | Model B | Model C | Specialized Model**

The precise implementation will vary considerably by organization and use case.

Nor does abstraction mean pretending that every model is interchangeable. Different models and providers have unique capabilities, and sometimes taking advantage of those capabilities will create dependencies.

The objective isn't perfect model independence.

It is to avoid **unnecessary coupling** that makes changing models more difficult than it needs to be.

## Why Would an Enterprise Want Multiple AI Models?  

For decades, enterprise technology organizations have tried to reduce complexity, consolidate vendors and establish technology standards.

So standardizing on one AI model can seem like the obvious approach.

But AI presents a different problem: there may not be one model that is best at everything.

Different models can have different strengths in reasoning, coding, multimodal tasks, latency, cost, context handling and specialized domains.

A model that produces the highest-quality answer may not be necessary for a simple task performed millions of times. The economics of that workload may favor a smaller, faster model.

Another workload may require capabilities available from a particular frontier model.

A specialized model may outperform a much larger general-purpose model within a narrow domain.

And those relative advantages will continue to change.

That means the relevant enterprise question may not be:

**Which AI model should our company use?**

It may be:

**Which model should we use for this particular job?**

Routing makes that choice possible.

Abstraction makes changing the choice easier.

## Flexibility Without Evals Isn't Enough  

There is one more concept executives should understand: **evals**, short for evaluations.

Suppose a new model is released next month. It is faster, costs substantially less and performs better on published benchmarks.

Should you switch?

Industry benchmarks can't fully answer that question for your organization.

What matters is how the model performs on **your tasks, with your context, against your requirements.**

That's why an organization needs its own evaluation framework.

Representative tasks can be tested against multiple models and evaluated against defined measures of acceptable performance. Depending on the use case, those measures might include accuracy, response quality, latency, cost, reliability, safety or other business specific requirements.

This creates a disciplined process:

**Evaluate → Compare → Route**

rather than:

**New model released → Adopt it**

Routing gives you the ability to change models.

Abstraction reduces the friction involved in changing them.

Evals help you determine **whether you should change at all.**

## What Questions Should an Executive Ask?  

Executives don't need to understand the mechanics of an AI router.

But they should understand enough to ask the people designing or selling an AI implementation the right questions.

Before approving one, I would ask:

**1\. Can we change the underlying model without rebuilding the application?**

If changing inference engines requires significant redevelopment, understand what creates that dependency.

**2\. Can different tasks use different models?**

Determine whether the implementation assumes a single model or allows different workloads to use the model best suited to them.

**3\. Are our prompts and enterprise context portable across models?**

Understand where the organization's accumulated context and application logic live and how tightly they are coupled to a particular provider.

**4\. Can requests be routed based on our requirements?**

Those requirements might include quality, cost, latency, security, specialization, availability or the nature of the task.

**5\. How will we know when another model is better?**

Make sure the organization has an evaluation methodology based on its own use cases rather than relying solely on benchmarks or vendor claims.

And then I would ask one final question:

**What would it take to move away from our preferred model provider tomorrow?**

The answer will tell you a great deal about the flexibility of the implementation you're approving today.

## Optionality Is a Strategic Asset  

Models will improve. Prices will change. New specialized models will emerge. Providers will introduce new capabilities. Security and regulatory requirements will evolve. And your organization's requirements will change.

The model that makes sense for a particular workload today may not make sense a year from now.

That doesn't mean enterprises should constantly switch models or avoid making technology commitments.

It means those commitments should be made deliberately.

A sustainable AI implementation should allow the organization to benefit from innovation at the model layer without repeatedly rebuilding everything above it.

That's why routing, abstraction and evals matter.

The goal isn't to correctly predict which AI model will win.

**It's to preserve your ability to choose.**