2024 · Kray.AI search and document assistant
One model for everyone, or one per customer
- Situation
- A document assistant for a fleet operator needed an architecture, and one model trained on every vessel's documents was the assumed default.
- Task
- Lay out the real trade-offs of cost, latency and access control instead of just picking one.
- Action
- I wrote both options out for client and team: one shared model versus one model per vessel, with pros, cons and the replica arithmetic.
- Result
- The partitioned design went forward, with the loss of cross-fleet questions accepted openly.