Sizing hardware for local LLM inference: laptop, shared node, or private cluster

Hardware for local LLM inference should be sized to the workflow, not chosen from a parts list. Who runs the system, how many of them at once, what they read, and whether the machine can be unplugged decide the shape. The model is fitted to that shape, not the other way round.

smolsi deploys in three shapes. This guide explains when each fits and what to have ready before a sizing conversation.

Laptop: one analyst, unplugged

A laptop deployment runs the full system for one person as a signed application. The engine, document index, and tools are local, and it works with the network unplugged.

It fits an analyst who works with sensitive documents on the move, in a client's building, or in a facility where nothing connects. It is the simplest shape to evaluate: install, index the documents, run the workflow.

Licensing: annual, per seat.

Shared node: a team, concurrently

A shared node is one machine serving a team. It supports concurrent sessions and background workers, so indexing, curation, and delegated analysis run while people keep working in the foreground.

It fits a department with a shared corpus: a legal team and its agreements, a finance group and its ledgers, a clinical team and its records. The node sits inside the organization's network boundary and can be isolated.

Licensing: annual, per node.

Private cluster: isolated environments at scale

A private cluster provides dedicated capacity for isolated environments, with deployment engineering. It is for organizations that need more than one node, separation between groups, or a controlled facility that has its own rules.

Private clusters are in early access. For defense companies, smolsi 1 is a dedicated hardware device in development as an integrated NVIDIA-based system, with controlled facilities first.

Licensing: enterprise deployments are quoted.

What drives the sizing

Five inputs decide the configuration:

1. Concurrency. One analyst or twenty. Shared inference changes the hardware class.
2. Corpus. How many documents, spreadsheets, and images, and how often they change. The document index runs locally and selects relevant material into the model's context; the corpus size shapes storage and indexing work.
3. Workflow shape. Long reads over large documents cost more than short questions. Host-executed analysis with tables and charts adds compute on the host side.
4. Isolation. Air-gapped operation changes how models and updates arrive and how the machine is administered.
5. Model family. Qwen is certified for the current release. Llama, DeepSeek, Mistral, Gemma, Kimi, and gpt-oss are available for requested deployments subject to certification, and the family affects memory requirements.

The engine is tuned to its hardware. One team evaluates the workflow, configures the system, and supports the deployment, and hardware selection and procurement are available through that team.

Three shapes at a glance

	Laptop	Shared node	Private cluster
Who	One analyst	A team	Isolated environments
Runs disconnected	Yes	Yes, inside the boundary	Yes, with deployment engineering
Background workers	Yes	Yes, concurrent sessions	Yes
Availability	Available	Available	Early access
License	Per seat, annual	Per node, annual	Quoted

Before the sizing call

Have these ready and the call is short:

• The workflow you would run first.
• Roughly how many people would use it, and how many at once.
• The size and kind of the corpus.
• The hardware your team already has.
• Whether the deployment must run disconnected.

Request an evaluation or pick a time.