Trust realms

One process, several logical identity services. A trust realm has its own configuration, its own signing key, and its own sessions, authorization codes, access and refresh tokens, credential offers, SAML request state, artifacts, statistics and audit log. Every realm answers on the same ports as every other, and they are told apart by a segment at the front of the path.

https://localhost:8081/oauth2/token                the DEFAULT realm
https://localhost:8081/realm/acme/oauth2/token     the realm `acme`

The point is what a client sees. Two realms are two authorization servers with two issuer identifiers and two JWKS documents, so a token minted in one does not verify in the other. That is the property somebody defines a second realm to get: a test that a client is checking the issuer it was configured with, a test that a wallet refuses a credential from an issuer it did not ask, a staging identity provider beside a production-shaped one on the same laptop.

If you are not using realms, nothing here has changed. The default realm has an empty prefix. A service with no realms defined strips nothing, rewrites no URL and grows no control — every path this service published before realms existed is a path in the default realm.

Defining one

curl -k -X POST https://localhost:8081/admin-api/realms/create \
     -H 'content-type: application/json' \
     -d '{"id":"acme","name":"Acme Corporation","domain":"acme.example.com"}'

Or on /admin/realms in the console, which is also where a realm’s settings, its four discovery URLs and the identifier of its signing key are.

A realm survives a restart when persistence.realms has a store under it: its name, its description, its per-realm settings and its own directory all come back. In the default persistence.mode=memory it does not, so whatever starts your stack should create its realms — which is why the API call above exists rather than a config-file section. A realm’s signing keys follow the mode, exactly like the default realm’s: in development mode they are regenerated on every start, so a token minted in a realm today verifies against nothing tomorrow; in product mode they are kept, sealed, in the store.

The id becomes a path segment: lower-case letters, digits and hyphens, starting with a letter or a digit, at most 31 characters. It may not be default, and it may not be the first segment of a path this service already serves — GET /admin-api/realms lists those in reserved, read off the live router, so the list cannot go stale. Nor may it be an EST label — a certificate profile id such as tls-server or kdc — because a realm is also reached at /.well-known/est/<id>/…, the path position EST gives its labels (see EST, Reaching a realm through the label).

Its domain

A realm has a DNS domain — iyasec.io, dev.iyasec.io, craptastic.net — set when it is created and fixed from then on. It is the root of every name the realm makes up:

It is not where the realm is reached. The OAuth issuer, did:web, the WebAuthn RP ID and every URL still come from the host a request arrived on, because a domain you typed says nothing about which hosts this service answers at.

• Leave it empty and the realm gets <id>.<global.domain> — acme.example.com — whose tree dc=acme,dc=example,dc=com is the one every realm had before realms had domains.
• No two realms may share a domain, and the default realm’s is global.domain.
• A domain inside another realm’s is allowed: dev.iyasec.io beside iyasec.io is a separate tree, and a search from dc=iyasec,dc=io does not see dc=dev,dc=iyasec,dc=io’s entries.
• To change it, remove the realm and create it again. The domain is inside every DN in the realm’s directory, every SPIFFE ID it issued and every key its KDC holds, so an update that names a different one is refused.

The names in the table are ordinary settings on the realm, visible on its page and changeable like any other; pass overrides on the create to choose your own.

Finding one

GET /realms is the directory, and it needs no credential. The prefix segment is a setting and the ids are whatever somebody typed, so a client being pointed at a realm cannot build a single URL without it.

{
  "pathSegment": "realm",
  "enabled": true,
  "active": true,
  "current": "default",
  "realms": [
    { "id": "default", "pathPrefix": "",            "baseUrl": "https://localhost:8081" },
    { "id": "acme",    "pathPrefix": "/realm/acme", "baseUrl": "https://localhost:8081/realm/acme" }
  ],
  "support": [ "…which families a realm separates, and which are shared…" ]
}

Each realm but the default also carries estLabelUrl, https://localhost:8081/.well-known/est/acme: the address of its EST server for a client that cannot put a path in front of a well-known URI.

Everything follows from baseUrl. Point a client at https://localhost:8081/realm/acme as its issuer and its discovery, token, authorization, userinfo, JWKS, SAML and OpenID4VCI endpoints all fall out of the metadata that base URL publishes — with no per-endpoint configuration on your side and none on this service’s.

enabled is the realms.enabled setting; active is whether any prefix is actually answering, which is false when the setting is on and nobody has defined a realm yet. They are two flags because “switched off” and “none defined yet” send you looking for different problems.

What a realm separates — and what it does not

A realm separates what this service ISSUES, not who it knows. Read this before you build a test on it.

Separated, completely

Separated — the embedded directory, one per realm

Each realm has a directory of its own behind the one socket, rooted at its domain — a naming context of its own, published in the root DSE:

dc=example,dc=com                 the DEFAULT realm  (global.domain)
dc=acme,dc=example,dc=com         the realm `acme`, created with no domain
dc=iyasec,dc=io                   a realm whose domain is iyasec.io

with its own ou=users, ou=groups, ou=applications, ou=federations and SPIFFE containers under each. So:

• the same name signing in to two realms is two entries, one per realm;
• an OAuth client registered under one realm is unknown to every other;
• a SAML service provider entry belongs to the realm it was created in;
• the SPIFFE registry is per realm, and so is the X.509 signing authority — each realm has a SPIFFE Issuing CA of its own on /admin/pki — and so is the trust domain itself: a realm is created with its domain as its spiffe.trustDomain, so acme issues spiffe://acme.example.com/…. See SPIFFE below, which is no longer on the not-separated list;
• and a realm is reachable over LDAP: ldapsearch -b "dc=acme,dc=example,dc=com".

That last point is why the realm is in the DN rather than in a partition of its own. LDAP answers on a socket with no path to put a segment in — a search arrives carrying a base DN and nothing else — so a name is the only thing a client could ever use to say which realm it means.

Every operation is answered from the directory its DN names. ldapsearch -b "dc=example,dc=com" is the default realm’s directory; ldapsearch -b "dc=acme,dc=example,dc=com" is acme’s; an entry belonging to another realm is filtered out of a search based above it, and the number filtered is logged rather than dropped silently. The root DSE publishes one namingContexts value per realm, which is how a client discovers that the others are there.

An operation that names one DN — an add, a modify, a delete, a compare, or a base-scope search of a single entry — is answered in that DN’s realm. Spelling out …,dc=acme,dc=example,dc=com is how a client says which realm it means on a socket that has nowhere else to put one, so refusing it would make a realm unreachable rather than isolated.

The one operation that carries two DNs is a rename, and a rename may not cross a realm: modifyDN from one realm’s subtree into another is refused with LDAP_AFFECTS_MULTIPLE_DSAS (71), which is what a directory answers when a rename would move an entry out of the server holding it. Two realms here are two directories, so that is the truth rather than a borrowed error code.

With no realms defined there is one naming context and one container, and every byte of every answer is what it was before realms existed.

Separated, and confined — the admin console roles

Every realm has two administrator rosters that matter to it:

• Its own. The realm’s cn=admin-read and cn=admin-write, in its own ou=groups. A new realm is seeded with an admin account holding both, which must change its password at its first sign-in; in product mode creating the realm shows that password once. In development mode, until that account signs in to the realm’s console, anybody who signs in through the realm holds both of its roles. Product mode never opens it that way: until that account has signed in there with its password, nobody else may use the realm’s console, and a sign-in as admin by any other method holds nothing.
• The service’s. The default realm’s two groups. Their members administer every realm, as they always did.

A realm’s own administrators are confined to their realm. Everything about the whole process is hidden from them and refused if asked for: the persistence store, the database, encryption, the secret store, the TLS certificate and client truststore, the LDAP service page, the embedded debugger and the API explorer. (Kerberos is not on that list: a realm has a KDC of its own, so its principals and keytabs are its administrator’s — what stays service-wide is the two Kerberos sockets and the development-mode trust.) So are creating or removing a realm, reading or editing another realm, replacing the service Root, exporting the TLS certificate’s key, and every setting that belongs to the process. That confinement is what makes a per-realm roster safe: creating a realm makes somebody an administrator of that realm and of nothing else.

The console asks the roster of the realm a person signed in through. Sign in at /realm/acme/admin and acme’s roster decides; admin in acme and admin in the default realm are different people. A realm administrator who opens another realm’s console is told they administer another realm, with a link back.

Choosing a realm. When realms are defined, the plain /admin and /portal ask which realm you belong to before signing you in — a list of realms in development mode, a box for the realm’s id in product mode. Add ?realm=<id> to skip the question, or ?realm=default to sign in to the default realm. A link to any page below /admin or /portal never asks.

The management API. /admin-api accepts a token from the default realm’s sts-management-api client everywhere. Under /realm/<id>/admin-api it also accepts a token that realm’s own sts-management-api client was issued, with the realm’s issuer and audience, and refuses that token the same service-wide operations the console refuses the realm’s administrators.

Separated — SPIFFE, by ADDRESS

SPIFFE is separated per realm, and the discriminator is neither a path nor a name but the endpoint address.

A realm is created with SPIFFE off and with a trust domain of its own — its domain, acme.example.com — and with Unix socket paths of its own. Turning spiffe.enabled on for that realm builds its authorities and binds a Workload API and a SPIRE Server API of its own on its Unix sockets. Nothing restarts: writing the setting is what binds the sockets.

A realm is created with its TCP listeners OFF (spiffe.workloadPort and spiffe.serverPort seeded to 0) and with no administrators (spiffe.adminIds seeded empty). The ports it would otherwise inherit are the default realm’s, already bound on 0.0.0.0, so turning SPIFFE on used to mean two refused binds; and nothing here can know which addresses a host has. To serve a realm over TCP, set spiffe.grpcHost on the realm to an address of its own and the two ports back to 8092 / 8181 — a client configured for those ports then reaches every realm where it expects to.

The address is the only thing a SPIFFE client can name a tenant with. gRPC has a path and it is the method name — /SpiffeWorkloadAPI/FetchX509SVID is fixed by the Workload API specification — so a realm segment there would be a method no conforming client calls. It is also what a real deployment looks like: one SPIRE server is one trust domain, and several trust domains are several endpoints.

So a container running several realms’ SPIFFE needs several addresses, and the compose files give it four: a static one for the service and three more added to its own interface on the way up (STS_EXTRA_IPS, which needs NET_ADMIN). Join tokens are per realm too — a join token is a credential for joining a trust domain.

What is still shared is the default realm’s own four sockets, which are bound when the process starts and stay bound with spiffe.enabled off (they answer Unavailable; a socket that vanished would read as a service that had stopped). Federated bundles are a realm’s own, and no realm may register one under a trust domain any realm of this service serves.

Separated — Kerberos, by REALM NAME

Kerberos is separated per realm — a KDC, a principal database and a krb5.realm each — and the discriminator is neither a path nor an address but the Kerberos realm name inside every request — which the protocol has always carried, because Kerberos has realms of its own.

Port 88 is still one socket. An AS-REQ or TGS-REQ names the realm it is for, and the KDC answers it out of that trust realm’s principal database.

A realm is created with Kerberos off, and named by its domain. krb5.enabled is seeded false and krb5.realm is the domain in capitals — ACME.EXAMPLE.COM for acme.example.com — because two realms answering to one name is a request nothing can route, and a unique domain is a unique name. So turning it on is one setting:

curl -X POST .../admin-api/realms/set \
  -d '{"realm":"acme","key":"krb5.enabled","value":"true"}'

Set krb5.realm on the realm first to answer to a different name.

Turning it on builds that realm’s principal database from its own settings: its own krbtgt, its own service account, its own fixture accounts in development mode, every key salted with its own realm name. Every name in it follows the realm’s own domain — ACME.EXAMPLE.COM gives HTTP/web.acme.example.com and a service account to match, and a realm called CORP.BANK.EXAMPLE looks like itself throughout, with nothing inherited from the service’s own domain. Set krb5.servicePrincipal on the realm to name its acceptor outright. Its people are the people in its own directory subtree, and their Kerberos keys (product mode) are derived onto their own entries there. Nothing restarts.

Three refusals keep the routing honest, and each is a code in error codes:

• Turning Kerberos on without a krb5.realm of its own — STS-KRB-0123.
• A name another realm already answers to — STS-KRB-0124: another realm’s, the default realm’s, or krb5.trustedRealm. Compared without regard to case, because two realms that differ only in case are two realms nobody can tell apart in a krb5.conf.
• Renaming or clearing it while Kerberos is on — STS-KRB-0125. Every key in that realm’s database is salted with the name. Turn it off, rename, turn it on.

A client reaches a realm by naming it, which is what a krb5.conf already does:

[realms]
  EXAMPLE.COM      = { kdc = sts-host:88 }
  ACME.EXAMPLE.COM = { kdc = sts-host:88 }   # the same port

Over MS-KKDCP there are two doors. A bare /KdcProxy routes by the name, like the socket. A realm’s own /realm/acme/KdcProxy is pinned to that realm and refuses another realm’s name with KDC_ERR_WRONG_REALM (STS-KRB-0122) — the prefix is an address somebody chose. The acceptor on krb5.servicePort and SPNEGO follow the same two rules: a ticket is accepted in the realm that issued it, and a ticket from another realm presented under a realm’s prefix is refused (STS-KRB-0126).

Trust realms do not trust each other’s Kerberos. A realm’s KDC holds no inter-realm key for another realm, so a ticket from one is not a referral to another — it is unknown there. The cross-realm referral this service does serve is the development-mode second realm (krb5.trustedRealm, PARTNER.COM), which belongs to the default realm alone.

What is still the process’s: the two sockets (krb5.kdcPort, krb5.servicePort), so a realm cannot move or take a port of its own, and that development trust. A realm administrator manages their realm’s Kerberos — principals, keytabs and the settings the database is built from — and those five settings stay service-wide.

Optionally separated — a front-end listener of its own (#99)

A realm can be reached at a host name and load balancer of its own, on a port of its own on every node. The /realm/<id> prefix still decides the realm; what changes is the scheme, host and port every URL the realm builds is on — https://acme.example.com/realm/acme/... instead of https://idp.example.com/realm/acme/.... Five settings, set on the realm (POST /admin-api/realms/set, or the TLS page read inside the realm); none can be set for the default realm or the whole process:

What the realm’s listener does:

• it is built like the main port — it asks for a client certificate and requires none, under the same truststore, TLS policy and PROXY protocol;
• it answers only that realm’s paths: the default realm’s and every other realm’s are 404 there (STS-TLS-0041), so a realm’s host fronts that realm alone. The main port still serves the realm under its prefix;
• it is bound, rebound and closed at once when the realm is created, changed or removed, on every node; a listener that cannot bind is recorded with its reason (STS-TLS-0039, STS-TLS-0040) and shown in GET /admin-api/realms (listener) — the service keeps running;
• the realm’s console and portal sign-in callback on its base is registered on their clients in every mode, because an administrator configured it.

The load balancer and DNS record in front of it are the deployment’s: on AWS, var.realm_listeners in deploy/aws/environment makes a network load balancer per realm (realm_listeners.tf), and its realm_listener_settings output prints the /admin-api calls that match. Not yet: a realm listener in a service running as several cells (STS-CORE-0148), and LDAPS, the KDC or SPIFFE ports per realm by this mechanism.

Not separated — the TLS certificate and client certificates

The certificate a handshake presents, and the client certificate it carries. A socket has no path in it, so neither can name a realm; a session started by GET /tls/sign-in goes in the realm of the authority that signed the certificate. LDAP’s 389 and 636 are not on this list — the sockets are shared, but what they serve is told apart by DN — nor are SPIFFE’s sockets, of which each realm has its own, nor Kerberos, which routes on the realm name in the request.

Not separated — the key-encryption key

A realm has keys of its own at rest, but not an independent boundary. Each realm has its own signing keys and its own branch of the certificate authority — that is the table above — and since #391 its own DATA ENCRYPTION KEYS: everything sealed for a realm is encrypted under keys no other realm’s values use. Those data keys are all wrapped under a single key-encryption key for the whole deployment, read once at startup from keys.kekProvider.

So anybody who can read that key can unwrap every realm’s data keys, and rotating it re-wraps every realm’s at once. Encryption at rest argues it, and says what a key-encryption key per realm would cost.

GET /realms and /admin/realms both publish this list family by family, so it is something the service tells you rather than something to remember.

The console

Every page shows one realm — the one whose prefix it was reached under — and carries a Trust realm chooser at the top of the sidebar, inside the same card as the sections, that moves to the same page in another realm, carrying the filter and the page you were on. It is a <select> and a button rather than a select that navigates on change, because the console runs no script at all (script-src 'none') and an inline handler is the one thing that policy forbids. It submits to GET /admin/realm-switch, which builds the target from the realm registry and a path it has checked is rooted and single-slashed — never from the query string as given.

Every root-relative link in an HTML response is rewritten on the way out to carry the current realm’s prefix, which is what makes the console work inside a realm without a link being edited. The chooser is the one control whose job is to leave the realm, so the links it builds are absolute and are left alone.

/admin-api is realm-scoped by the same prefix, so /realm/acme/admin-api/config is that realm’s configuration and every one of its operations works per realm. The five operations under /admin-api/realms manage the registry itself, which is process-wide: there is one list of realms in a process, and remove refuses to remove the realm the call arrived in — the caller would be left talking to a prefix that had stopped existing.

Three settings

The first two cannot be set on a realm: a realm that could switch realms off would be doing it from inside the request that found it, and a realm that could move its own prefix would be changing the prefix already used to find it.

Removing one

Removing a realm takes everything it held with it — its sessions, authorization codes, tokens, offers, service provider state, statistics, audit log and signing key. That is deliberate rather than thorough: a realm re-created with the same id inheriting the last one’s sessions would be the most surprising thing a re-created realm could do.

The realm’s directory subtree goes too — its people, groups, applications, federation relationships and SPIFFE registrations — so dc=acme,dc=example,dc=com answers NoSuchObject afterwards and a realm re-created under that id starts with a fresh seeded tree. The default realm cannot be removed at all.

Before it goes, everybody who was relying on it is told (#232). Removing a realm from /admin/realms or POST /admin-api/realms/remove:

1. stops anything NEW from starting in it (#262): from this moment, on every node, a sign-in there is refused, and so is every issuance — a token of any grant, an authorization code, a SAML or WS-Federation assertion, a WS-Trust token, a Kerberos ticket, a GNAP grant, an OpenID4VCI credential, an ACME, EST or SCEP certificate and a SPIFFE SVID (STS-CORE-0121; the token endpoint answers invalid_grant). A sign-out, a revocation, introspection, UserInfo, metadata and a Shared Signals poll still work, so a receiver can collect what the removal sends it;
2. ends every session in it the way a sign-out does — a CAEP session-revoked (initiating_entity: admin) for each, and the back-channel Logout Tokens of the relying parties on it;
3. reports every person in its directory purged — RISC account-purged;
4. tells every Shared Signals stream stream-updated with status disabled;
5. waits, at most realms.removalDeliveryTimeoutS seconds, for all of that to be delivered, and only then removes the realm.

What had not been delivered when the wait ran out — a push receiver that did not answer, a Logout Token still being retried, a SET on a poll stream that nobody collected — is logged (STS-CORE-0120), listed in the answer’s retirement member, and goes with the realm. A poll receiver can collect only while the wait lasts; the removal does not wait for somebody to come and poll. On a cluster the node that received the removal does this once: the other nodes remove the realm when they learn of it, and say nothing again.

The first step is written to the store before anything else happens, so every node refuses from then on. If the service stops half way through a removal, the realm stays in that state — refusing sign-ins — until you remove it again, which finishes the job.

A removal that stopped half way is visible. Once a realm has been in that state for longer than a removal can take (realms.removalDeliveryTimeoutS plus 30 seconds), it is shown as removal interrupted:

• on Trust realms (/admin/realms), the row is marked, and a block at the top says when the removal began and what is refused, with a Finish removing button;
• on the realm’s own page there, the same block;
• on every console page inside the realm, a banner;
• on the sign-in screen, the refusal says the removal was interrupted;
• in GET /admin-api/realms, each row has a retiring member (null when the realm is not being removed) with since, sinceIso, inProgress, interrupted, refusing, why and finish. The service also logs STS-CORE-0123 when it starts with such a realm.

To finish, remove the realm again from another realm: the button, or POST /admin-api/realms/remove with {"id": "<realm>"}. It ends whatever is still live, reports the realm’s people purged and removes it. A receiver may be told account-purged a second time about somebody the interrupted removal had already reported; the service cannot know who that was, and telling somebody twice is better than not at all. While a removal is still in progress, removing the realm again is refused (STS-CORE-0122), so a second click does not announce everything twice.