File

View as Markdown

What it indexes

The File connector uploads one or more files into OpenCore-managed storage and indexes each recognized file as a document. It is a load-state connector with no scheduled refresh or source system to poll.

ContentIndexing behavior
Text and document filesSupported extensions are .txt, .md, .mdx, .conf, .log, .json, .csv, .tsv, .xml, .yml, .yaml, .sql, .pdf, .docx, .pptx, .eml, .epub, .html, .xlsx, and .xlsm. Unrecognized extensions can be stored by the upload handler but are skipped during connector processing.
Tabular files.csv, .tsv, .xlsx, and .xlsm become tabular sections and retain the stored file ID for code-interpreter staging. If raw-file staging is unavailable, tabular files are skipped.
Images and embedded images.png, .jpg, .jpeg, and .webp become image sections. DOCX and PPTX actively extract embedded images. PDF embedded images are extracted only when the deployment’s image-extraction-and-analysis setting is enabled. The current EML, EPUB, HTML, XLSX, and XLSM paths do not emit embedded images.
ZIP uploadOne ZIP may be expanded by an upload request. Directories and path components beginning with . are skipped; .onyx_metadata.json is stored separately as metadata. A second ZIP is rejected. Files extracted from the archive still pass the extension checks above during indexing.

The normal Admin flow accepts multiple files and does not apply a file-extension filter in the browser. Uploading bytes successfully therefore does not guarantee that the connector will emit an indexed document.

Metadata timing affects document IDs. An id supplied by .onyx_metadata.json is processed before the document ID is chosen and can set it. First-line inline ONYX_METADATA is extracted later, after the document ID has already been fixed, so an inline id does not replace the ZIP-provided or generated document ID.

Prerequisites

  • Prepare files with one of the supported extensions and validate that their contents match the extension.
  • For encrypted PDFs, note that the normal Admin File flow creates an empty internal credential and exposes no PDF-password field. Use an unencrypted copy unless your deployment has a separate supported ingestion path.
  • For ZIP metadata, put exactly one .onyx_metadata.json at the archive root and use JSON keyed by filename or a list whose entries contain filename.
  • Ensure the file-store and indexing workers can hold and process the upload. The File upload handler does not enforce a connector-specific byte limit. The current MAX_FILE_SIZE_BYTES indexing check only logs when a constructed document exceeds the configured value; it does not reject the upload or skip that document.
  • Apply deployment ingress, storage, memory, and worker limits appropriate to your environment, then test representative large PDFs, spreadsheets, images, and archives before broad use.

Credentials

MethodOpenCore behavior
Admin File uploadNo external credential is required. After storing the files and creating a load-state connector, the UI creates an empty internal credential so the connector-credential pair has an owner and can use OpenCore access controls.
PDF passwordThe backend connector can read a pdf_password value from a credential, but the normal Admin File form does not expose or populate it. Do not assume encrypted PDFs are supported by this UI flow.

Do not upload access tokens, private keys, or source-system credentials as ordinary content. A File connector indexes content; it does not use a file as an authentication credential.

Configure in OpenCore

  1. In the Admin panel, open Connectors, select File, and enter a descriptive connector name.
  2. Select one or more files. A ZIP is expanded by default; include at most one ZIP in the upload.
  3. If metadata is required, use either the first-line ONYX_METADATA protocol marker in a supported plain-text file or an archive-root .onyx_metadata.json file. ONYX_METADATA is a legacy protocol field, not the product brand, and must retain this exact spelling for compatibility.
#ONYX_METADATA={"title":"Example policy","link":"https://kb.example.com/policies/example","primary_owners":["<OWNER_NAME>"],"department":"People"}
Document text begins here.

Parsed protocol fields are id, connector_type, link, file_display_name, title, primary_owners, secondary_owners, and doc_updated_at. If a stable document ID is required, put id in .onyx_metadata.json; an inline id is parsed too late to change the document ID.

ZIP metadata is applied first. For source type, owners, update time, display name, title, and link, only non-empty inline values replace earlier values because the connector uses fallback (or) selection; an empty inline value cannot clear a ZIP value. Inline custom tags update same-named ZIP custom tags.

Custom tags contain keys outside the reserved exclusion list. The reserved fields document_id, time_updated, doc_updated_at, link, primary_owners, secondary_owners, filename, file_display_name, title, connector_type, pdf_password, and mime_type are excluded from custom tags. document_id and time_updated are not parsed aliases for id and doc_updated_at. Under the current compatibility parser, id itself is not excluded from custom tags, even though ZIP-level id can also set the document ID.

  1. Select OpenCore Private groups for restricted files. Choose Public only when every OpenCore account may search every file in the connector.
  2. Create the connector. The UI uploads the files, creates the connector and internal empty credential, links them with the selected access, and starts one indexing run.
  3. To revise an existing File connector, add or remove files in its management view. At least one file must remain; additions trigger update indexing and removals trigger pruning.

Permissions

File contents and ONYX_METADATA owner fields do not create source ACLs. primary_owners and secondary_owners are document metadata only. File is not an Auto Sync Permissions source in the current Admin access selector.

Search access is controlled only by the connector’s OpenCore Private groups or Public mode. Every user with connector access can search every emitted document in that File connector. Split files into separate connectors when audiences differ.

File-management authorization is separate from search access. All three File management endpoints first require a curator or admin session. Upload proceeds after that role gate. Listing then checks connector access; updating then checks editable access. A global curator has a special update allowance for a Public File connector, but not for an unrelated Private connector. An ordinary connector user cannot list or update files through these Admin endpoints merely because the user can search the connector. These management rules do not create per-document search permissions.

Verify

  1. After creation, confirm the upload returns stored file names and the indexing attempt completes.
  2. Search a distinctive title and sentence from each supported file type in the batch.
  3. Confirm an unsupported-extension test file is absent even if upload storage accepted it.
  4. For CSV, TSV, XLSX, or XLSM, inspect the extracted table and test raw-file analysis if code-interpreter staging is part of your deployment.
  5. For a ZIP, compare extracted file names with the archive. Confirm hidden path entries and .onyx_metadata.json are not indexed as documents.
  6. With ZIP metadata, verify that id sets the document ID. Then test a file with only an inline id: its document ID must remain generated, while the current parser can retain id as a custom tag. Also verify title, link, owner metadata, doc_updated_at, one non-reserved custom tag, and the absence of reserved fields from custom tags. Use non-sensitive values.
  7. Test every assigned OpenCore group and a user outside those groups. Metadata owners must not change the result.
  8. Add one file and remove one file from an existing connector, then verify the new document appears and the removed document is pruned.

Troubleshooting

SymptomWhat to check
Upload succeeds but no document appearsConfirm the filename has a supported extension, the file record exists, and extraction produced text, a table, or an image section. Upload storage itself does not enforce the indexing extension allowlist.
A second ZIP is rejectedThis is expected. Use one archive for the connector upload, combine the archives before uploading, or use a separately supported ingestion API.
ZIP metadata is ignoredConfirm the exact archive-root filename is .onyx_metadata.json, the JSON is valid, and each key or filename matches the stored basename. Hidden-path filtering removes metadata files placed inside a dot-prefixed directory.
Inline metadata appears in searchable textThe ONYX_METADATA marker must be on the first line of a supported plain-text file and contain valid JSON. Keep the exact legacy field name.
Inline id does not change the document IDThis is expected because the document ID is chosen before inline metadata is extracted. Put id in .onyx_metadata.json when it must set the document ID. The current parser may still retain inline id as a custom tag.
document_id, time_updated, or another reserved field disappearsReserved fields are excluded from custom tags. document_id and time_updated are not aliases; use ZIP-level id and doc_updated_at for those behaviors. Non-empty inline recognized fields can override earlier values, but empty values cannot clear them.
A large upload exhausts resourcesThe connector upload handler has no connector-specific byte rejection, and MAX_FILE_SIZE_BYTES currently warns rather than blocks. Enforce deployment-level request and storage limits, reduce the batch, and test extraction memory separately.
A tabular file is skippedConfirm raw-file staging is configured. CSV, TSV, XLSX, and XLSM require the staging callback before tabular sections are emitted.
Users can search files outside their source responsibilityFile has no source ACL sync, and metadata owners are not authorization. Restrict the connector with OpenCore Private groups or split files by audience.