@happyvertical/smrt-inference
Fallback for inference tiers: ordered model chains, recorded substitutions, fusions, user and tenant bindings, a settings panel, and a hosted spend gate.
What it gives you
Fallback for inference tiers: ordered model chains, recorded substitutions, fusions, user and tenant bindings, a settings panel, and a hosted spend gate.
- 01An alias binds a tier to an ordered chain of models; a failed call moves to the next candidate
- 02The usage row of the answering call records the resolved tier and each substitution
- 03smrt doctor --ai checks providers, key variables, models, aliases, and cycles before a call needs them
- 04Importing the root entry once at startup registers the package with smrt-core
Failures are classified before fallback
The executor classifies a failed candidate as quota, credential, missing-model, transient, refusal, or missing-capability. It retries a transient failure once, then tries the next candidate. When every candidate of a tier fails, the call moves to the next cheaper tier. Set inference.degrade to false, or pass minimumTier on the call, to stop that move. Another model never continues text that the caller already received.
Every move is recorded
The usage row of the answering call carries tierResolved and substitutions. A call that no candidate answers writes a zero-token usage row with the reason and throws InferenceExhaustedError. A fusion is recorded as a fusion run with one usage row for each seat call.
Fusions keep the tier interface
defineFusion declares a single, pair, or handoff fusion over smrt-prompts keys. A tier can bind to that fusion, and callers continue to request the tier.
Bindings for users and tenants
The resolver entry resolves each tier from the stored configuration of the user, then of the tenant, before the application configuration. A tenant can bind a tier only to targets on the application allow-list. Keys stay in smrt-secrets. InferenceSettingsPanel renders rows and callbacks; it fetches nothing and makes no authorization decision.
Hosted inference and assistant engines
The node entry provisions gateway keys for each user, and the gate entry supplies the spend gate and kill switch. The server entry adds Node-only adapters for a ChatGPT connection and for local command-line engines. The ChatGPT sign-in is an opt-in preview. It is never a default and never a fallback.
Package UI
Components in this package
Import these public components directly. When the package also exports playground entries, the next tab renders its real components with package-owned example data.
Package previews
No playground module yet
This package does not currently export a ./playground module. Its component names and import path remain available in the Components tab.
Intentionally not registered
Generated interface
REST
The exact route set comes from each model's @smrt({ api }) policy
and the application manifest—not from a second controller definition.
The InferenceOverride model has no generated REST, CLI, or MCP surface. Hosts change bindings through their own permission-checked actions.
Generated interface
MCP
AI tools are generated from the methods you explicitly include. Their inputs are validated against the same model metadata used by REST and CLI.
The InferenceOverride model has no generated REST, CLI, or MCP surface. Hosts change bindings through their own permission-checked actions.
Browser-native agent interface
WebMCP
The generated web definitions carry the same tool names, descriptions, and JSON Schema
inputs as MCP. smrt-web registers them with document.modelContext.
The InferenceOverride model has no generated REST, CLI, or MCP surface. Hosts change bindings through their own permission-checked actions.
Generated interface
CLI
The app CLI turns manifest resources into predictable operator commands while keeping package-specific actions close to their models.
The InferenceOverride model has no generated REST, CLI, or MCP surface. Hosts change bindings through their own permission-checked actions.