Case study 01 / Systems
Building recoverable Matrix administration for Hypha
How Zenith moved homeserver authority out of the desktop client, made every mutation traceable, and tested the contract against an exact Synapse release.
508 + 20
Hub tests and subtests
89
Focused broker and deployment tests
1.157.2
Exact Synapse version exercised
0
Fixable High or Critical image findings
The brief
Make homeserver administration feel like a product workflow—without turning the product into the homeserver's root of trust.
Hypha needed a safe way to create accounts, promote room operators, issue password resets, end sessions, and remove rooms. The hard part was not drawing an admin screen. It was deciding what should happen when a request collides, a connection disappears, or an operator is about to remove the last path back into the system.
Why the obvious path failed
An admin panel is a distributed system.
Authority in the wrong place
A desktop app should not retain a permanent Synapse administrator token or the registration secret that can create local accounts.
Creation that could overwrite
The administrative user endpoint behaves like an upsert. A repeated request could change an existing account instead of failing safely.
Unknown outcomes
When a connection drops mid-request, retrying blindly can duplicate—or reverse—an operation whose result the interface never received.
The system
Split conversation state from operator authority.
Synapse remains responsible for rooms, events, accounts, and devices. A small administration broker becomes the only production mutation path. It runs beside the homeserver as a separate service, presents a same-origin contract to Hypha, and keeps every powerful credential on the server side.
User surface
Hypha
Reads state, negotiates capabilities, and holds one operation ID per action.
Same-origin gateway
One public surface
Synapse
Rooms / messages / sync
Admin broker
Mutations / policy / receipts
Matrix database
Conversation state
Operation journal
Redacted receipts
Contract before controls
Every click becomes a recoverable operation.
The client first asks what the broker supports. Mutations stay disabled when required safety capabilities are absent. When an operator acts, Hypha creates one UUID and keeps it through the operation's entire lifecycle.
pending
Accepted and still being reconciled
applied
Verified against Synapse state
rejected
Safely refused with a typed reason
outcome unknown
Not retried; requires operator review
Safety mechanics
Fail visibly. Recover deliberately.
Create means create
Account creation uses Synapse's nonce and HMAC registration flow. A collision returns user_exists—without changing the existing password, role, profile, sessions, or devices.
Journal before mutation
The broker records a redacted pending receipt before calling Synapse. Reusing the same key returns the same receipt; conflicting reuse fails with a typed error.
Protect recovery
Service accounts are protected, and the broker rejects a demotion or deactivation that would leave no active human recovery administrator.
Preserve uncertainty
A transport failure never triggers a blind repeat. Reads reconcile the result where possible; otherwise the receipt remains outcome unknown for an operator to resolve.
Exact-version testing
The integration test changed the implementation.
Synapse rejects service localparts beginning with an underscore.
Current admin reads no longer expose raw password hashes.
Logout-all now uses the v2 bulk device-deletion endpoint.
Asynchronous purge IDs are opaque 16-character tokens—not UUIDs.
Operator experience
The interface tells the truth about the system.
- Read-only administration still works against an older broker; unsafe mutations do not.
- The last successful snapshot remains visible—and is explicitly labelled stale—during an outage.
- Promotion and demotion are separate actions, not side effects hidden inside an edit form.
- The active Matrix account is visible and cannot log out all of its own sessions.
- The entered administrator secret is cleared when the short-lived session expires or is revoked.
- Password-reset state comes from the durable server journal, never local memory.
The honest status
Verified release candidate. Not a rollout announcement.
At publication, the broker and client branches are implemented and the isolated exact-version lifecycle is green. Production still waits on four non-substitutable gates:
- 01A clean current-SDK Hypha build after the Git LFS budget is restored.
- 02Infrastructure validation with the repository-pinned Terraform 1.14.0.
- 03A non-federating rehearsal restored from the production backup.
- 04Broker-first merge and deployment before any live Hypha mutation is enabled.
“Authority belongs behind a narrow, observable boundary. The interface should reveal that boundary—not pretend it isn't there.”
Building an operator surface where security, recovery, and usability have to agree?
Talk to Zenith