Security model
A language model can be tricked: it processes untrusted text and it can act. That is why AI Workforce OS bounds damage with deterministic controls in the backend (Go), not with prompt instructions. This page is for anyone evaluating or operating the platform: it describes what is implemented and, at the end, what is not.
On this page
- Design principles
- Credentials vault
- Human approvals
- Least privilege for connections
- Spending caps, hours and anomalies
- Kill switch and read-only mode
- Audit log
- Tenant isolation and identity
- Untrusted content and the code sandbox
- What is not covered
Design principles#
Agents only produce tool requests; the backend decides whether they run. The Tool Gateway checks every request in this order:
- Controls: kill switch, agent pause, read-only mode and tools or actions switched off.
- The connection grant for that agent.
- Validation: connection status, granted capability, the grant's maximum risk and usage limits.
- A scan for secrets in the outgoing arguments.
- Approval reasons: the policy engine collects every verdict and keeps the strictest (deny beats approval, and approval beats allow).
Note
The agent runtime never receives connection credentials. If an organization stores its own model key, that key does travel to the runtime on each call, because the runtime is what calls the provider.
Tip
There is no promise that the model will never be influenced by malicious text. The promise is that the blast radius stays limited to what the backend authorizes.
Credentials vault#
Connection credentials (OAuth tokens, API keys) and each organization's own model keys are stored encrypted with AES-256-GCM in an envelope scheme. A key-encryption key (KEK) held outside the database wraps a versioned per-organization data key (DEK), and the DEK encrypts each secret with a random nonce. The additional authenticated data includes organization, record, version and kind, so a ciphertext copied to another row or organization does not decrypt.
Without a KEK (CONNECTIONS_KEK, 32 bytes in base64) the vault fails closed: it neither stores nor reads live credentials. It can load several KEKs at once for rotation (CONNECTIONS_KEK_PREVIOUS), and each credential has active and retired versions.
- The API returns only metadata and, for secrets of 20 bytes or more, the last 4 characters as a hint.
- Revoking a connection suspends its use, calls the provider's revocation endpoint and destroys the ciphertext. If the provider does not answer, the ciphertext is kept until a retry succeeds.
- The exact values of secrets in use are registered with a redactor that masks them in tool outputs. The audit log also masks anything shaped like a credential.
KEK (outside the DB, CONNECTIONS_KEK)
└─ wraps → per-organization DEK (versioned)
└─ AES-256-GCM(secret, random nonce, AAD = org|record|version|kind)
Warning
Keep the KEK apart from the database backups. If you lose it, the encrypted credentials cannot be recovered; if it leaks together with a backup, they are exposed.
Human approvals#
An agent can request an action, never approve it. People with the admin or owner role decide, and the role comes from the session token, not from the request body. Each approval is bound by hash to the exact arguments approved, is consumed once and is revalidated before execution, because rules may have changed while it waited.
An approval for an interactive request expires after 30 minutes (APPROVAL_TIMEOUT), and an expiry counts as a rejection. Project approvals do not expire by default: the system sends an hourly reminder while they are pending.
There is a floor no organization rule can weaken: declared high risk, actions on the mandatory-approval list (APPROVAL_ACTIONS) and agents at the suggest or approve_each levels. On top of it, an organization can add rules: amount thresholds, forbidden actions, new recipients, sensitive categories, required role by amount or action, and dual approval.
Dual approval requires two distinct people holding the required role, and the person who made the request cannot approve. A single rejection from an authorized person is final. If the action runs after a server restart, the approval is claimed atomically so that it executes at most once.
| Autonomy level | Behavior |
|---|---|
suggest | Every action with an external effect waits for approval; once approved, it runs. Today it behaves exactly like approve_each. |
approve_each | Every action with an external effect waits for approval. |
rules | Runs without approval unless the floor or an organization rule requires one. Today it behaves exactly like autonomous. |
autonomous | Runs without approval except for the floor, organization rules, capabilities marked as always requiring approval and tasks flagged for suspected injection. |
Note
Each role template or onboarding pack sets the initial level of its agents.
Least privilege for connections#
An action through an external connection passes three filters in series: the capability the provider declares, the grant the organization gives to a specific agent, and the policy and approval of the action itself. A grant can only narrow what the connection allows. The recommended pattern is to keep a read-only connection separate from a write one.
Limits are computed from the usage table in Postgres, so they survive restarts. An agent cannot change its own autonomy, permissions or rules, and any deny rule beats an allow rule.
| Limit | Connection | Grant |
|---|---|---|
| Calls per minute, hour and day | Yes | Yes |
| Writes per day | Yes | Yes |
| Bytes per day | Yes | No |
| Monthly budget (last 30 days) | Yes | No |
Spending caps, hours and anomalies#
Before every runtime call, the maximum estimated cost is reserved against the request, the agent, the department and the organization. Afterwards it is reconciled using the higher of the cost the runtime reports and the one the backend recomputes. When a request, agent or department cap is reached, the work pauses and a person decides whether to raise it; the organization cap makes the call fail.
Structural limits also guard against loops: delegation depth (5), retries per task, time per task, plan and project size, and a per-organization queue (8 concurrent calls by default).
Operating hours are a weekly schedule with a time zone that only prevents starting new work outside open hours; it does not block approvals or work already admitted. Anomaly detection applies four explainable rules, each with a 15-minute cooldown between alerts, and records every finding in the audit log. On its own it never blocks: automatic freeze is optional and ships disabled.
| Rule | Fires when |
|---|---|
spend_spike | Spend in the current hour is at least 4 times the hourly average of the previous 24 h and at least 2 USD (needs 3 h of history). |
tool_request_burst | An agent requests 40 or more tools within 60 seconds. |
rejected_approvals | 5 or more approvals are rejected within 1 hour. |
new_recipient_domains | 3 or more never-seen recipient domains appear within 1 hour (counted once 5 domains are known). |
Kill switch and read-only mode#
The kill switch has two levels. Any admin or owner can activate it; lifting it requires the owner role and a written reason. It can also be imposed with the KILL_SWITCH environment variable (freeze or lockdown), which no API call can lift: the variable must be removed and the process reloaded.
There are finer controls: pausing an agent (it denies the agent's new tool calls and new work), read-only mode for the organization or an agent (it keeps reading and proposing, but effects are denied) and switching off a tool or a single action, for example email or email.send. Held actions return to the approval queue and must be approved again: nothing runs by inertia when a control is lifted.
| Level | Effect | Who lifts it |
|---|---|---|
none | Normal operation. | Not applicable |
freeze | No new work starts, the Tool Gateway denies all connection use (reads included) and held actions return to the approval queue. | owner, with a written reason |
lockdown | Everything in freeze, plus suspending every connection. | owner, with a written reason; connections are resumed afterwards |
Audit log#
The audit log is a per-organization hash chain: each entry includes the hash of the previous one (SHA-256 over a canonical representation with sequence, organization, timestamp, actor, action, entity and details). Changing, deleting, reordering or inserting an entry breaks the chain from that point, and truncating the tail is detected against the chain head.
Before being chained, every entry goes through a scrubber that keeps metadata only: it replaces content fields (bodies, arguments, subjects, summaries) and credential-like keys with [redacted], masks embedded secrets and cuts strings to 300 characters. Audited events include authentication, policy decisions, approvals, tool executions, and changes to controls and budgets.
GET /api/v1/audit # query
GET /api/v1/audit/export?format=jsonl # or format=csv
GET /api/v1/audit/verify # recomputes the hash chain
Tip
Every so often, store the chain head's sequence and hash off the server. GET /api/v1/audit/verify accepts anchor_seq and anchor_hash to check the chain against that anchor.
Tenant isolation and identity#
Tables holding organization data use row-level security (RLS) in Postgres. Each transaction sets the organization with a transaction-scoped set_config, the policies filter by it and the application connects with a role without BYPASSRLS. An integration test covers isolation between organizations.
- Access tokens last 15 minutes and refresh tokens 30 days.
- After 5 failed attempts the account is locked for 15 minutes. Sign-in also allows 10 attempts per 15 minutes per email and 20 per IP (in Redis when available, in memory otherwise).
- For nonexistent accounts a password hash is still computed, so response time does not reveal whether the user exists.
- There is single sign-on with OIDC (with PKCE), verified domains, a sign-in policy that can require MFA or SSO, TOTP MFA and SCIM 2.0 provisioning.
Untrusted content and the code sandbox#
Everything that comes from outside (emails, API responses, files) goes through a deterministic pipeline before reaching the model: text is normalized (NFKC, invisible characters removed), credentials, one-time codes and card numbers are masked, markdown images and links are neutralized, the text is cut (4,000 characters by default), scored with injection heuristics and wrapped as data in a delimited block. A high score flags the task and forces approval of its actions. The score is a signal, not a barrier: the barrier is permissions, approvals and limits.
Code an agent wants to run is shown in full to a person, and the backend only sends it to the runner after approval. Changing the code or an input requires a new approval, and the kill switch aborts a run in progress.
- Container without network (
--network none) and with a read-only root filesystem. - Unprivileged user, all capabilities dropped and
no-new-privileges. - CPU, memory and process limits, a hard timeout and output caps, with an extension allow-list for generated files.
- None of the runner's environment variables and no direct access to the Drive or the backend.
<untrusted_data id="msg-18c2f" source="tool:email.read" from="client@example.com" trust="external_untrusted">
subject: ...
body: ...sanitized content...
</untrusted_data>
Warning
Desk mode is optional and ships disabled. If an organization turns it on (which requires an explicit risk acknowledgement), code confined to the agent's own files runs without approval, under smaller caps.
What is not covered#
There are no certifications (neither SOC 2 nor ISO) and the project is not ready for regulated enterprise environments. These are the known limits:
- Single sign-on is OIDC only: SAML is not implemented.
- Audit export is an unsigned JSONL or CSV dump, and nothing anchors the chain head off the server automatically.
- Recovery after a restart assumes a single instance: with several replicas, the per-organization queue, budget reservations and some counters are per process or last-writer-wins.
- A call already sent to the runtime finishes even if the kill switch is activated.
- There is no per-risk approval permission: deciding equals being admin or owner. The ban on self-approval ships disabled (except with dual approval) for compatibility with one-person teams.
- The HTTP rate limits on authentication let requests through if their store fails.
- The production
Caddyfiledoes not set aContent-Security-Policy. - External actions cannot be undone. There is a 60 to 300 second window to cancel an email, and Drive file versions can be restored.
- There is no spending cap unified across model and connections, and no per-tool cap.
Tip
If you are evaluating the project for production, treat this page as the starting point of your own review, not as a guarantee.
Warning
The sandbox-runner service (the sandbox profile of the development compose file) mounts the Docker socket, which is equivalent to root access on the host. For anything beyond a single trusted host, use a dedicated rootless or remote daemon and a runtime such as gVisor (SANDBOX_RUNTIME=runsc).