Vendor-Neutral Routing

Vendor-neutral routing is the practice of putting one governed API in front of the provider adapters active in a deployment. The AP00006 codebase includes OpenAI, Anthropic, Azure OpenAI, and AWS Bedrock adapters, but all four are disabled by default. Applications target an operator-assigned model ID; the broker selects among registered, healthy candidates. Re-pointing an existing model across already-supported adapters can be a configuration change. Adding an unsupported transport still requires implementation, and the signed ProviderSpec extension path is inert until an operator explicitly configures and trusts it.

The category framing mirrors AWS. Just as AWS gives you one IAM-governed control surface over heterogeneous services, model-broker gives you one governed control surface over heterogeneous AI vendors: a single internal contract (ModelBrokerService) over the enabled adapter set, with policy, request-count quota, PII controls, audit, and cost attribution applied at one control point. Hard money enforcement and durable lineage have separate gates described below; they are not unconditional properties of every response.

Why it matters

Three properties make a single governed brokerage worth the architecture:

• No vendor lock-in at the code level. Your application speaks one provider-neutral contract. Moving a workload between shipped, validated adapters does not touch application code when both adapters and the required request/response semantics are enabled. An unsupported transport or normalizer still requires implementation and tests.
• Governance is enforced once, not per vendor. Authorization, atomic budget reservation for the active request-count quota, fail-closed response-PII masking, audit, and per-request cost attribution are applied at the single control point. The default-off hard-cost gate is a separate control.
• Failures look the same everywhere. Each provider SDK fails in its own dialect; the broker normalizes them all into one stable OSY-PROV-* error family, so clients branch on one taxonomy instead of four.

One governed API in front of enabled providers

model-broker contains four provider adapters — OpenAI, Anthropic, AWS Bedrock, and Azure OpenAI — each implementing a single Service Provider Interface (ModelProvider). Only adapters enabled and successfully registered by the deployment are candidates. The broker refuses normal serving startup with zero providers unless a non-serving environment explicitly opts out of that guard.

                  one governed API
   client ──▶ AP00001 EDGE ──▶ AP00006 model-broker
                                 admission pipeline
                                 (authorize → budget → route → dispatch
                                  → PII-mask → audit + cost record)
                                       │
              ┌────────────┬───────────┼───────────┬────────────┐
              ▼            ▼            ▼            ▼            ▼
          OpenAI      Anthropic     Bedrock    Azure OpenAI   (adapter SPI)
          adapter      adapter      adapter      adapter

Every adapter maps the provider's native request/response shape onto one provider-neutral ModelRequest / ModelResponse, so the governance pipeline and your client both see a single contract no matter who serves.

Routing is by model name

You select a model the way you already do — by its name. The broker resolves that name to a registered provider and dispatches the call. This is what makes the drop-in posture possible: an existing OpenAI-style client keeps its model-selection logic and only re-points its base URL at Osyra.

import os
 
from openai import OpenAI
 
client = OpenAI(
    api_key=os.environ["OSYRA_API_KEY"],
    base_url="https://api.osyra.ai/v1",
)
 
# The deployment operator assigns a chat-capable ID. The catalog confirms
# deployment registration, not workspace policy or operation capabilities.
model = os.environ["OSYRA_MODEL"]
available_ids = {entry.id for entry in client.models.list().data}
if model not in available_ids:
    raise RuntimeError(f"{model!r} is not registered in this deployment")
 
response = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Hello!"}]
)

A catalog hit is only a candidate: policy, workspace authorization, provider health, and capacity are still enforced on the inference request. A successful authorized operation—not list order—is the proof that the configured model is usable for that caller.

The native Osyra SDKs send an API key in X-Api-Key. The upstream OpenAI SDK in this example cannot choose that header, so Edge accepts an exact IAM-issued osy_(sandbox|test|live)_<43 base64url characters> key as Bearer only on GET /v1/models and POST /v1/chat/completions, POST /v1/chat/completions/stream, and POST /v1/embeddings. Native routes, including GraphQL, continue to require X-Api-Key for API-key authentication. Never send both an API key and an OAuth bearer JWT.

API-key lifecycle and verification are active only in deployments configured with IAM's API-key crypto backend and KMS HMAC key. Deployments without that wiring fail closed; the code example does not imply that every installation has API keys enabled.

When moving providers is configuration

Each provider adapter is wired behind an explicit per-provider enable flag. A deployment can turn an existing adapter on or off—or swap which enabled adapter backs an operator-assigned model—without changing application code.

| You want to… | What you change | What you do not change | |---|---|---| | Activate a built-in provider | Enable its adapter flag + supply credentials | Application code | | Retire a provider | Disable its adapter flag | Application code | | Re-point a model to another vendor | Routing/registry configuration | The client's model="…" name | | Add a ProviderSpec-supported provider | Supply a valid signed bundle and arm its trusted registry source | Application code; broker code only if the transport/normalizer is already supported | | Add an unsupported transport or normalizer | Implement and review it | This is not configuration-only |

One error taxonomy: OSY-PROV-*

Whatever a vendor SDK throws — an OpenAI 5xx, an Anthropic timeout, an Azure auth rejection, a Bedrock throttle — the broker normalizes it into the platform's provider error family, OSY-PROV-* (band 5000–5099, defined once in osyra-response-common). Clients branch on a single, stable taxonomy instead of four vendor-specific shapes, and no internal vendor detail leaks across the boundary.

| Code | Meaning | |---|---| | OSY-PROV-5001 | Provider unavailable | | OSY-PROV-5002 | Provider error (generic upstream failure) | | OSY-PROV-5003 | Provider timeout | | OSY-PROV-5020 | Provider circuit open (breaker tripped) | | OSY-PROV-5024 | Provider rate-limited (upstream) | | OSY-PROV-5029 | Invalid model for the selected provider |

The full set is the canonical OSY-<CATEGORY>-<NNNN> registry documented in the Error Code Catalog. Resilience is built in around this taxonomy: each provider is wrapped in its own circuit breaker and rate limiter, and a family-diverse fallback never re-routes a provider's safety refusal to a more-permissive provider.

Cost optimization and hard money budgets are gated

How Vendor-Neutral Routing relates to the rest of Osyra

The edge gateway (AP00001) is the customer-facing policy-decision point: it authenticates the caller, makes its own authorization decision, and forwards a workspace-scoped context to the broker over gRPC. The broker applies the shared admission pipeline and dispatches to the selected vendor adapter. Per-request cost is attributed back to Billing (AP00003).

Derivation-claim emission is conditional. The Java property defaults to enabled, but the current declared GitOps base explicitly sets OSYRA_BROKER_CLAIM_EMISSION_ENABLED: "false" while the AP00011 least-privilege identity and production signer/HSM acceptance gates remain open. That declared deployment emits no broker derivation claim.

If a deployment later clears those gates and enables the feature, production startup requires real AP00006→AP00011 mTLS material, and AP00011 accepts verified-memory emission only for an authoritative PRO or ENTERPRISE tier. The default emitter mode is asynchronous and fail-open for inference: the broker can return a synchronously minted claim ID before SignAndInsert persists, and the emitClaimForFresh seam returns null when no emitter is wired or setup fails. An asynchronous OME failure records warning/metric/audit evidence while the completion still succeeds. Therefore, a response is provenance-bound only when a durable OME claim was actually persisted; a minted ID alone is not proof of that outcome.

Scope and roadmap

The four provider adapters behind explicit enable flags, model-name routing with direct selection plus ordered fallback, the provider registry, per-provider circuit breakers and rate limiters, atomic request-count budget reservation, fail-closed response-PII masking, per-request cost attribution, and OSY-PROV-* error normalization are built in V1 — with the caveat that providers are opt-in per deployment, the signed ProviderSpec source is inert by default, and the service registry status is In Progress, not yet Production. Broker derivation-claim emission is also explicitly disabled in the current declared GitOps base.

Cost- and latency-optimized routing strategies (the V2 cost-optimization engine, ENG-1715, behind claim-substantiation gate ENG-1741), per-tenant bring-your-own-key credential isolation (wired end-to-end but disabled by default), and the semantic cache (present, off by default) are not V1 defaults. Hard cost-denominated enforcement is implemented behind a default-off gate and must not be described as active until a deployment arms and proves its real Billing/database/outbox chain.