> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://developers.alephant.io/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://developers.alephant.io/_mcp/server. # Tracing > Inspect request, agent execution, and tool-call chains to diagnose quality, permission, and cost issues. Tracing reconstructs the key steps of a request from entry point through model calls, knowledge retrieval, tool calls, and response delivery. It helps administrators diagnose issues without exposing complete sensitive content to unrelated users. In supported deployments, the AIvis admin route is `/admin/tracing`. The page shows the tracing capability entry, product screenshots, and a `Contact Sales` action. Provider wiring is implemented for Braintrust and Langfuse; some deployments expose the provider setup flow to administrators, while hosted deployments may require an operator or Alephant contact to enable it. ![AIvis tracing analytics dashboard showing spend, tokens, success rate, latency, team budgets, and model costs](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/alephantai.docs.buildwithfern.com/f7a0db509b2279f7f227c46e29a509f506e282f44bfe35fc5b2a8564b4f78689/assets/aivis/tracing/tracing-analytics-overview.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260805T125948Z&X-Amz-Expires=604800&X-Amz-Signature=30b5e4da22c37a65269f222dccd3afb1adc84e674450c02c1b15e6a498eb0810&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject) ## What You Can Inspect | Question | Where to look | | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | How much did model usage cost this month? | Use the analytics view to compare spend, tokens, success rate, latency, team budgets, and model costs. | | Which team, Agent, model, or entity caused a spike? | Filter request logs by time range, department, entity, model, and status. | | Why did a request fail? | Open the request record and compare status, error reason, provider, token counts, latency, and trace-provider output. | | Are budgets or policies taking effect? | Check the budget, alert, rate-limit, and model fallback policy views before and after a test request. | | Can an external trace be opened? | Confirm Braintrust or Langfuse is configured, then search the external project by time, model, flow, or request metadata. | ## Management Boundary | Area | Guidance | | ----------- | ------------------------------------------------------------------------------------ | | Visibility | Allow only operators, admins, or authorized auditors to view traces. | | Content | Prefer facts, status, and metadata over long-term storage of full sensitive context. | | Correlation | Link request ID, user, workspace, agent, and model identifiers. | | Retention | Set retention according to audit and privacy requirements. | ## Before Configuration * Confirm who can open the AIvis admin tracing page and who can manage tracing-provider credentials. * Decide whether prompts, responses, tool arguments, or retrieved snippets require redaction, sampling, or shorter retention before they leave the private deployment. * Prepare at least one model call path for testing, such as a chat request, Agent run, bot message, voice call, image generation, or video generation. * If you need external traces, prepare a Braintrust or Langfuse project and decide whether the credential is managed by environment variables or the admin provider setup flow. * Correlate tracing with usage analytics, query history, request logs, budgets, and alerts. ## Open the Tracing Entry Open the AIvis admin console and go to the tracing page. In supported deployments this corresponds to `/admin/tracing`. The current page is a capability entry. It explains the observability surface and shows three expected views: usage analytics, budget and policy controls, and request logs. If your deployment only shows `Contact Sales`, the self-service provider setup is not enabled for that environment. Ask the deployment owner or Alephant contact to enable tracing, or configure the provider through backend environment variables. ## Configure a Tracing Provider AIvis supports two tracing providers: | Provider | Required values | Optional values | Where to get them | | ---------- | -------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Braintrust | `API Key` | `Project Name`, `API URL` | Create or copy an API key from [Braintrust](https://www.braintrust.dev/app). Use the default API URL unless you are on another region or self-hosted endpoint. | | Langfuse | `Secret Key`, `Public Key` | `API Base URL` | Create or copy keys from [Langfuse](https://cloud.langfuse.com). Use your own host when using a different region or self-hosted Langfuse. | When provider setup is exposed in the admin page, select the provider, fill the fields, and use the built-in test before saving. The backend validates the credential first, then saves the provider as enabled. For operator-managed deployments, configure the same values through environment variables: | Provider | Environment variables | | ---------- | ---------------------------------------------------------------------------------- | | Braintrust | `BRAINTRUST_API_KEY`, optional `BRAINTRUST_PROJECT`, optional `BRAINTRUST_API_URL` | | Langfuse | `LANGFUSE_SECRET_KEY`, `LANGFUSE_PUBLIC_KEY`, optional `LANGFUSE_HOST` | In multi-tenant hosted deployments, AIvis resolves tracing providers from environment variables only. In single-tenant deployments, an enabled database provider row can override the environment fallback for the same provider. ![AIvis tracing policy controls showing budgets, alerts, rate limits, and model fallback settings](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/alephantai.docs.buildwithfern.com/1ff0a52041fd9bb5a8789348d25b6a4f8524073898c4924739d640b18a3a661e/assets/aivis/tracing/tracing-policy-controls.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260805T125948Z&X-Amz-Expires=604800&X-Amz-Signature=f2456608a1f9e6a9defb9dc038efcfdd283b8b32b47f9b5768cb64b28d640633&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject) ## Generate a Test Trace 1. Save or enable the provider configuration. 2. Send one controlled AIvis request through a known path, such as a chat with a test Agent. 3. Record the timestamp, user, workspace, Agent, model, and expected result. 4. Open the request logs or external tracing provider and filter by the same time range and model. 5. Confirm that the record includes status, latency, token usage, cost metadata, and the correct flow name. AIvis traces many LLM paths through explicit flow names, including chat responses, chat summarization, query expansion, document filtering, embeddings, reranking, image generation, video generation, speech-to-text, text-to-speech, privacy gateway calls, and Agent automation calls. A missing trace usually means the provider is not active, the request path is not instrumented, or the request failed before the model call started. ![AIvis request log showing filters for time, department, entity, model, status, tokens, and cost](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/alephantai.docs.buildwithfern.com/ec5e2d4c3a029965837f15a653de3a4fbe0a109ab79f9149d250972f05b2da03/assets/aivis/tracing/tracing-request-logs.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260805T125948Z&X-Amz-Expires=604800&X-Amz-Signature=e234d78a71d50514212559b03fb047725ec9705fcb69ea3577991348816fe1da&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject) ## Verification * The AIvis tracing page can be opened only by authorized operators, admins, or auditors. * A test request appears in request logs with the expected time range, model, status, token count, cost, and latency. * If Braintrust or Langfuse is enabled, the same request can be found in the external project. * Failed or denied requests include a usable reason without exposing unrelated sensitive content. * Budget, alert, rate-limit, or fallback changes are visible in later request behavior. ## Troubleshooting | Symptom | Priority checks | | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The page only shows `Contact Sales` | Self-service tracing setup is not enabled in this deployment. Ask an operator or Alephant contact to enable tracing or configure provider environment variables. | | Provider test fails | Confirm the API key pair belongs to the selected provider, the host or API URL is reachable from the backend, and no extra spaces were copied. | | Langfuse rejects credentials | Confirm both `Secret Key` and `Public Key` are from the same Langfuse project. | | Braintrust traces go to the wrong project | Check `Project Name` or `BRAINTRUST_PROJECT`; leave it at the deployment default only if that project is intended. | | No trace appears after a test request | Confirm the provider is enabled, backend workers picked up the latest config, and the tested flow is instrumented. | | Request log is empty | Broaden the time range, remove department/entity/model/status filters, and confirm the request reached AIvis rather than failing at an upstream bot or gateway. | | Tokens or cost are missing | Confirm the model provider returns usage metadata and that the request completed far enough for the usage processor to record it. | | Users can see too many traces | Restrict tracing access to admin, operator, or auditor roles and avoid giving general workspace users access to raw traces. | ## Security and Maintenance * Store provider API keys only in protected configuration or the admin credential store. * Do not put API keys, raw prompts, private responses, access tokens, or tool secrets into documentation, tickets, screenshots, or Agent instructions. * Rotate Braintrust and Langfuse credentials when administrators change or a key may have leaked. * Review tracing access regularly because traces can reveal model names, user IDs, workspace IDs, document references, errors, and cost patterns. * Keep raw-content retention shorter than audit metadata retention when privacy requirements are strict. ## Related Pages * [Model Credentials](../models/model-credentials) explains where model-provider credentials are configured. * [Language Models](../models/language-models) explains how model routing affects traced calls. * [Generation and Chat](../models/generation-and-chat) explains the user-facing chat path that can be used for test traces. * [Hooks](./hooks) explains controlled outbound callbacks that can be correlated with traces.