Custom Inference Provider | Alephant Documentation

Custom Inference Provider connects OpenCore to a model service that uses a LiteLLM-compatible provider definition. The provider configuration and its models are saved together; a provider with no valid model name cannot be enabled.

Compatibility

Use this flow when the service appears in the Provider selector or has a LiteLLM provider ID that the selector accepts. The configured endpoint must be reachable from the OpenCore backend, and the provider must support the request shape for each model capability you declare.

Custom inference works in Lite and Standard because language-model requests do not require OpenSearch. Standard is still required for connector-backed knowledge and indexing.

Add the provider

Open Models & Capabilities, choose Language Models, and add a Custom provider. Complete the provider fields before saving:

Field	Required	Meaning
Provider	Yes	LiteLLM provider ID used to route requests; it cannot be changed after the saved provider is opened for editing
API Key	Only when the provider requires it	Credential used by the provider endpoint
API Base URL	No	Base endpoint for a custom or self-hosted service
API Version	No	Provider-specific API version when its adapter requires one
Environment Variables	No	Additional key-value properties passed to the LiteLLM `completion()` call
Display Name	Yes in the normal Admin form	Name used to identify this configured provider in OpenCore

The Provider and Display Name are validated in the normal Admin flow. Add at least one model before submitting the form.

Optional fields

Field	When to set it
API Key	Set it only when the endpoint authenticates with a provider key. Store the real value only in the protected form.
API Base URL	Set it when the provider does not use its adapter’s default endpoint. From a container, `host.docker.internal` addresses the host in supported Docker environments; container `localhost` addresses the container itself.
API Version	Set the exact version required by the provider adapter; otherwise leave it empty.
Environment Variables	Add only documented provider properties. Blank keys are ignored, and later duplicate keys replace earlier values when the form is converted to a map.

Do not place provider credentials in Agent instructions, chat messages, model names, environment-variable examples, or source control. Use placeholder values in shared troubleshooting material.

Add models

Model field	Behavior
Model Name	Required provider-facing model identifier. At least one non-empty name is required.
Display Name	Optional user-facing label; OpenCore falls back to Model Name when it is empty.
Input Type	Choose Text Only or Text & Image. This declares image-input support but does not add support that the provider lacks.
Max Tokens	Optional maximum input-token value for this model. Leave it empty when no override is needed.

Use Add Model to configure more models under the same provider. Keep each model name exactly aligned with the provider’s accepted identifier and declare Text & Image only after verifying the endpoint accepts image input.

Access

Access choice	Result
All Users & Agents	Makes the configured provider models public inside the deployment.
Named Groups & Agents	Restricts use to selected groups and Agents. Group selection appears only when the deployment tier enables business group controls.
Admin	Admin access is always shared, including when named access is selected.

Access controls are shown in the normal Admin configuration flow, not the shortened onboarding form. Review access again whenever a model is added or its intended audience changes.

Verify

Save the provider and confirm that OpenCore reports it as enabled.
Confirm that every configured model appears on Language Models for an identity that should have access.
Send a representative text request. For a Text & Image model, also test an image request and confirm the provider actually accepts it.
If the request fails, recheck Provider, API Base URL, API Version, provider-specific Environment Variables, and the protected credential described in Model Credentials.