AI Security

Alephant protects AI traffic across authentication, policy evaluation, provider routing, Agent tool execution, and observability. These controls do not all run in one component: the Dashboard stores configuration, the Gateway enforces data-plane decisions, policy-service evaluates supported policies, and logs provide evidence of the result.

This page describes the security behavior that is visible in the current Gateway and Dashboard. A control shown in the Dashboard is not automatically proof that every deployed Gateway enforces it.

Security model

An authenticated model request follows this security path:

Client
  → Gateway authentication and Virtual Key context
  → Gateway model and provider checks
  → policy-service evaluation when enabled
  → selected upstream provider
  → request, response, usage, and policy logs

The main trust boundaries are:

Treat configuration, enforcement, and observability as separate checks. Saving a policy confirms control-plane state; a request result and its logs confirm runtime behavior.

Authentication and credentials

When Alephant authentication is enabled, the Gateway:

• Requires a Virtual Key for normal Gateway traffic.
• Resolves the key by its hash and rejects unknown or expired keys.
• Associates the request with its workspace, Virtual Key, Master Key, department, member, or Agent context when available.
• Uses the resolved Provider Key only for the selected upstream request.

Provider credentials stored as Master Keys are encrypted with AES-256-GCM. Secret values are masked when serialized or written through the Gateway’s secret wrapper.

Private deployments must not use an authentication-disabled Gateway configuration on an untrusted network. Disabling Alephant authentication allows callers who can reach the Gateway to use the configured upstream Provider Keys.

Authentication protects the Gateway entry point. It does not replace network controls, TLS, secret management, or role-based access to the Dashboard and logs.

Request policy evaluation

When policy evaluation is enabled, Virtual Key requests can be evaluated before they reach the upstream model. The Gateway supplies policy-service with trusted request context such as:

• Workspace, Virtual Key, and department identifiers
• Request identifier, model, and selected provider
• Client IP and request headers
• Estimated input tokens and cost when available

Request headers can contain sensitive information, so policy-service must be treated as a trusted internal dependency and isolated accordingly.

Body inspection and policy actions

The request body is attached to policy evaluation only when body inspection is enabled for the workspace. The current Gateway limits the attached body to 1 MiB.

policy-service can return one of the following request decisions:

When body inspection requires a rewritten body, an allow decision without a replacement body is rejected instead of forwarding an ambiguous request.

A model selected by policy becomes authoritative for the remainder of that request. Keep the policy-service replacement set aligned with workspace model and provider rules.

Policy availability

The Gateway supports two behaviors when policy-service is unavailable:

Managed and private deployments can use different settings. Choose the failure mode explicitly based on the workload: sensitive or regulated traffic normally requires deny, while a lower-risk workload may choose availability with allow.

Model and provider access

The Gateway applies model and provider rules at different scopes:

• A Virtual Key request that matches blocked_models is rejected.
• A model that matches allowed_models is allowed.
• In the current Gateway contract, a model that matches neither list is also allowed.
• Managed Cloud applies the workspace Provider allowlist before dispatching to an upstream provider.

Because an unmatched model is currently allowed, do not treat the Virtual Key allowed_models field by itself as a strict allowlist. If the Dashboard displays Model Allowlist, verify the effective behavior in the target deployment before relying on it as a security boundary.

Fallback, cost routing, and policy-driven model changes must use providers and models that remain valid for the workspace. Review Provider Routing together with the access policy.

Sensitive data and request logs

The Dashboard can store PII policy configuration for email addresses, phone numbers, SSNs, credit-card numbers, IP addresses, and custom regular expressions. Supported actions include block, redact, and alert.

These fields describe desired policy state. Detection coverage, input or output scanning, language support, timeout behavior, and notification delivery depend on the deployed policy-service. Test representative requests before relying on PII policy in production.

Request and response bodies

When observability is enabled, request logs can contain the full request and response bodies:

• When both bodies are below 1 MiB, the current Cloud logging path stores both inline.
• When either body reaches 1 MiB, the current Cloud logging path stores both in object storage and records temporary read URLs.
• If object-storage upload fails, the current Gateway can fall back to inline log storage.
• Users who can open Request Logs can view and copy the available request and response JSON.

PII redaction of the body forwarded to a model does not by itself guarantee that every operational log is redacted. Avoid sending unnecessary secrets, restrict log access, and set a retention period appropriate for the data.

Agent and tool security

Agent tool execution is a separate security boundary from a model request. When Agent tools are enabled, the Gateway can:

• Limit visible and callable tools by workspace, Virtual Key, and authenticated Agent identity.
• Ignore an untrusted Agent identifier reported only in the request body when an authenticated Agent identity is available.
• Validate tool arguments against the configured JSON Schema before dispatch.
• Limit concurrent tool calls per workspace and block calls above the configured per-call cost.
• Run policy preflight for OpenAPI, MCP Streamable HTTP, and MCP SSE targets.
• Block high- or critical-risk tool calls when the required audit event cannot be recorded.
• Limit request size, response size, and execution time.

The default egress policy requires HTTPS and blocks literal loopback, link-local, metadata-service, and private-network IP targets. Domain names are not currently revalidated after DNS resolution, so DNS rebinding and resolved private addresses require additional network-layer protection.

Keep Agent tools disabled unless the deployment needs them. When enabled, combine the Gateway rules with DNS controls, an egress proxy or firewall, narrowly scoped service credentials, and upstream authorization.

Failure behavior

Applications should handle machine-readable status and error fields rather than depend on a fixed human-readable message.

Record and alert on authentication failures, policy unavailability, policy denials, unexpected model changes, tool blocks, and unusual access to request bodies.

Private deployment checklist

Before exposing a private Gateway to production traffic:

1. Enable Alephant authentication and verify that unauthenticated requests are rejected.
2. Terminate TLS at the Gateway or a trusted Ingress and restrict direct access to the internal listener.
3. Restrict CORS at the trusted edge for browser-accessible deployments.
4. Configure trusted proxy CIDRs so client-IP policies and rate limits do not trust arbitrary forwarded headers.
5. Keep PostgreSQL, Redis, policy-service, log storage, and object storage on protected networks.
6. Inject Provider Keys, the Master Key encryption key, storage credentials, and service tokens through the deployment’s secret manager.
7. Disable request and response body debug logging in production.
8. Select the policy-service unavailable mode deliberately and test it during rollout.
9. Set log retention and Request Logs access according to the sensitivity of prompts and responses.
10. Enable Agent tools only with explicit tool scopes, egress controls, audit delivery, and upstream authorization.

The Gateway currently accepts permissive CORS settings in its application layer, so private deployments should apply a restrictive CORS policy at the trusted ingress or reverse proxy until the runtime configuration is explicitly narrowed.

Capability status

Verify enforcement

For every security policy that matters to the workload:

1. Save the policy in Control → LLM Policy or Control → Agent Policy.
2. Send one request that should be allowed and one that should be denied through the target Gateway Base URL.
3. Confirm the returned status, model, provider, and policy result.
4. Open Request Logs and confirm the workspace, Virtual Key, model, provider, status, and available policy metadata.
5. For Agent tools, inspect Agent Runs and confirm requested, blocked, approval, and completion events.
6. Repeat the test after restarting the Gateway and after making policy-service unavailable in a controlled environment.

This verification proves the behavior of the deployed version. It is stronger evidence than a saved form or a feature label in the Dashboard.

Related pages

• Guardrails
• Provider Routing
• Session
• Policies & Rules
• Audit & Logs