A manufacturer with operations across several countries was maintaining and sharing controlled documents by email and shared drive. Files lived in scattered places, and getting the right document to the right client or team member was slow and entirely manual.
The obvious build — upload, folders, share link — takes a few weeks and works beautifully in a demo. It also stops working somewhere around ten thousand documents, and by then it is load-bearing. So the interesting part of this project was never the feature list.
Access is the feature, not folders
The system is built around need-to-know rather than folder permissions. Each client or employee sees exactly the documents scoped to them; staff share or email documents to clients directly from the system rather than re-attaching files. Role-based access is enforced at the query layer, not in the UI, so a client cannot reach another client's document by changing an id in the URL.
One decision is deliberately unusual and is now locked: the archive never auto-deletes. Retention rules that silently remove documents are a good idea right up until an auditor asks for something from four years ago. Storage is cheap; a missing controlled document is not.
Finding the failures that only appear at scale
We seeded the database to production volume and measured instead of assuming. The problems that surfaced were the ordinary, expensive kind — invisible on a developer laptop with 200 documents:
| What we found | Why it only breaks at scale |
|---|---|
| Workspace route returned all documents, unbounded | Fine at 200 rows. Fatal at 30,000 |
| Dashboard stats ran five full-table scans on every load | Cost grows linearly with the table, on the most-visited screen |
| Share creation did an N+1 insert loop | One share is fast; a fifty-recipient share is fifty round trips |
| Folder tree walked one query per level | Deep hierarchies multiply the cost invisibly |
Keyword search used a LIKE scan | Scans the whole column; no index can help it |
| Document catalogue was unbounded and sorted in memory | Loads the entire catalogue to return a page of it |
The fixes were unglamorous and permanent: server-side pagination, composite indexes matched to the actual access patterns, a single-query folder walk instead of a walk-per-level, batched share inserts, a FULLTEXT index to replace the LIKE scan, and a short cache on the stats endpoint.
Workspace listing: 126ms across 30,000 documents → 2ms.
We also wrapped every multi-write path in a database transaction after a near-miss where an email failure could leave a committed record without its side effects. The document-create path now persists first and sends afterwards, so a mail outage can never fail a record that is already saved.
The part that keeps it fixed
Performance work decays unless something defends it. The system ships with 182 automated tests that run on every push through GitHub Actions:
- 121 API assertions — permissions, atomicity, regression
- 56 Playwright browser tests — every page, desktop and mobile at 375 / 390 / 768 / 1280, plus an RBAC matrix that walks every role against every route
- 5 scale tests that fail the build if a query regresses at volume
- axe accessibility checks on every page
That suite is not decoration. It caught two real accessibility defects before any user did — an icon button with no accessible name, and a navigation label below contrast minimums — both fixed and redeployed the same day.
Reading documents that are not text
Much of what arrives is a scan rather than a text PDF. Extraction runs as a cascade, cheapest reliable path first: native text extraction, then a language model, then OCR, then a human. No single engine is a single point of failure, and the expensive path only runs when the cheap one genuinely cannot answer.
Where it runs
The platform moved from shared hosting to a dedicated cloud instance with MariaDB and object storage for files, behind an automatic-TLS reverse proxy with nightly database backups. A companion courier-tracking system runs alongside it on the same estate.
The client owns the source code, the repository and the hosting accounts. The test suite ships with it — which means the next team to touch this system finds out within minutes if they have broken something, rather than finding out from the manufacturer.