Start with the model
A language model is a large set of model weights plus software that runs them. A hosted chatbot loads those weights in somebody else’s data centre, usually in Virginia or Dublin. A private AI server loads them on a machine in your office or another environment you control.
The word server describes the job, not the shape. Ours is usually a Mac Studio sitting under a desk, about the size of a sandwich box. Staff reach it through the office network while the machine does the model work for everyone.
What sits around it
A model on its own is not much use. What turns it into something an office can run on:
- A model runtime loads the chosen model, manages memory and produces each response.
- A browser workspace gives staff accounts, conversations, files and controls without a separate desktop app.
- A document index finds relevant passages in approved files and returns them with the answer for checking.
- A speech service can transcribe supported recordings locally.
- Administration covers people, permissions, network rules, updates, records and health checks.
LocallyAI calls its browser workspace the Harness. The full feature list lives on the Harness page.
What “local” means
In the base offline configuration, your prompts, your documents, your recordings and the model work itself all stay on the appliance. The model does not need to send a prompt to a cloud AI provider to write an answer.
That does not mean the machine is sealed shut. Web research, Harness or model updates, an external integration, remote support and off-site fine-tuning are separate features or services that can move data or require a connection. Your quote names every exception you switch on, in writing, before anything is installed.
What it changes
You can see where the data goes. The everyday path runs between a staff browser and the machine down the hall — and that is the whole of it. Any outside path can be treated as an exception and reviewed on its own.
Cost moves from seats to capacity. The practical limit is memory, storage and simultaneous work rather than a licence for every prompt. Electricity, backups, IT support and any connected service still cost money.
More responsibility stays with the office. Accounts, physical access, disk protection, backups, recovery and network design matter because the machine holds the working copy.
What it does not fix
A local model can still be wrong, incomplete or biased. A citation can point to the right passage while the answer draws the wrong conclusion. Material work still needs source checking and qualified review.
Hardware also sets a real ceiling. A machine can run only the models and context that fit with enough working headroom. Model size, format and runtime affect speed, and a model that loads is not automatically suitable for the intended work.
Local processing does not by itself satisfy privacy, professional or record-keeping law. Consent, purpose, access, retention, security and incident response still apply where relevant.
Questions to ask before buying
- Model
- Which exact model, version, licence and format will be supplied?
- Capacity
- Which normal and heavy jobs were tested on this machine?
- Sources
- Can staff open the passage behind a document answer?
- Network
- Which destinations are blocked, and which exceptions are approved?
- Accounts
- Are permissions enforced by the server, including direct requests?
- Recovery
- Who owns backup, restore testing, keys and support access?
- Updates
- How is a temporary update connection opened, checked and closed?
Next reads: trace patient data through a transcription service, or compare local and subscription costs.