Setting up a local inference stack raises a simple question: is buying a GPU actually cheaper than paying for API access? The honest answer: almost never on cost alone. The real calculus is different.
The AI industry has a default answer for every problem: send your data to an API, pay per token, get results. For most people, that’s the right choice. But the calculus differs for firms processing sensitive data — financial statements, client portfolios, internal documents — and sending that overseas introduces problems that per-token pricing doesn’t solve.
Three of them, specifically:
Data residency. Your documents leave your control. Even if the provider promises not to train on your data, the data crosses borders and enters someone else’s infrastructure. In Singapore, MAS guidelines on data localisation aren’t optional for financial services[1] — they’re the floor, not the ceiling.
Cost predictability. Per-token pricing looks cheap until you’re processing at scale. At high volume, the annual bill compounds into a significant operating expense with no ceiling. You’re paying rent that never becomes equity.
Latency predictability. API responses depend on network conditions and provider load. For workflows where an LLM is one step in a pipeline — ingestion, extraction, review — consistent latency matters more than peak speed.
The counterargument is straightforward: local inference means buying hardware, maintaining software, and accepting lower capability than frontier models. The question is whether the trade-off is worth it at today’s hardware prices.
Settling that requires a concrete setup rather than speculation. The results follow.
Test environment (July 2026):[M]
| Component | Specification |
|---|---|
| CPU | AMD Ryzen 9 9950X3D |
| GPU | NVIDIA RTX 5090, 32GB GDDR7 VRAM |
| RAM | 64GB DDR5 |
| OS | Windows 10 |
| Inference server | LM Studio v0.4.x |
| Model | Qwen3.6-27B (Q4_K_M quantisation) |
Qwen3.6-27B was the choice for a specific reason: it’s large enough for meaningful document processing and reasoning tasks but small enough to run on consumer hardware with 4-bit quantisation. The Q4_K_M quantisation was a deliberate trade-off — higher precision would be slower, lower precision would degrade output. Q8 sits in the middle ground.
The test generated a 1,764-word financial analysis document — a realistic workload for document summarisation and report drafting. The measurements were tokens per second, total generation time, and the cost against equivalent API calls. All tests ran on this machine with no other GPU workloads.
Inference speed: 35 tokens per second, generating 1,764 words (2,048 tokens) in 50.6 seconds.[M]
Now, GPT-5.4 streaming through OpenAI’s API typically returns the same output in 15–20 seconds. The local model is 2.5–3x slower. But what matters is this: “slower” depends on the use case. For a document processing pipeline where the LLM is one step among many, adding 30 extra seconds per document is acceptable. For interactive chat where a user is waiting for a response, it’s not. The question isn’t “is it fast?” — it’s “is it fast enough for the job?”
Output quality: The generated financial analysis was coherent, numerically accurate, and structurally sound. For document processing tasks — extracting fields, categorising entries, summarising sections — Qwen3.6-27B at Q4 quantisation produces usable output. For creative writing or nuanced reasoning, the gap vs frontier models is more noticeable. Honesty matters here: if you need the best possible output for complex reasoning, models like GPT-5.5 or Claude Opus 4.6 still win. No amount of local hardware changes that.
Memory footprint: The model loaded into 18.2GB of the available 32GB VRAM, leaving headroom for concurrent tasks. An 8B parameter orchestrator model ran alongside it without issues.[M]
| Metric | Qwen3.6-27B (local) | GPT-5.4 (API) |
|---|---|---|
| Tokens/sec | 35 | ~100–150 |
| Latency (2048 tokens) | 50.6s | ~15-20s |
| Cost per run | $0 (after hardware) | ~$6.50 |
| Data leaves premises | No | Yes |
| VRAM required | 18.2GB | None |
This is where the industry gets it wrong. A lot of articles claim local inference “pays for itself” quickly. The actual numbers are less flattering for the hardware side than you’d expect.
API pricing as of July 2026 (converted to SGD at 1.35 USD/SGD):[2][3]
| Model | Input (per M tokens) | Output (per M tokens) |
|---|---|---|
| GPT-5.4 | S$3.38 | S$13.50 |
| Claude Sonnet 4.6 | S$4.05 | S$20.25 |
| Qwen3.6-27B (local) | S$0 | S$0 |
The hardware cost for this setup is approximately S$8,000–10,000 (the RTX 5090 alone is ~S$5,000–6,000 in Singapore). Here’s when that investment actually pays for itself against GPT-5.4 API costs at a 60/40 input/output split:
Source: OpenAI — “API Pricing”. openai.com
| Monthly Volume | Annual API Cost | Break-even |
|---|---|---|
| 1M tokens | S$89 | 101 years |
| 10M tokens | S$891 | 10.1 years |
| 34M tokens | S$3,000 | 3 years |
| 50M tokens | S$4,455 | 2 years |
| 100M tokens | S$8,910 | 1 year |
That last row — 100M tokens/month — is an enormous volume. For context, a typical document processing pipeline at a mid-sized firm might process 5–10M tokens/month. At that level, API pricing wins on pure cost by a wide margin.
The honest takeaway: local inference does not pay for itself on cost alone unless you’re processing at enterprise scale. The economic case for local hardware is about cost predictability at sustained high volume, not about saving money at moderate volumes. If you’re processing less than 20M tokens/month, API is cheaper, — the cost math says so plainly.
Given the cost math above, local inference only makes sense for reasons other than economics. The reasons are:
Data sovereignty. The primary driver. Financial documents, client portfolios, internal analysis — keeping that data on-premise isn’t a luxury, it’s a compliance requirement in regulated industries. You can’t outsource data residency to a terms-of-service clause.
Unmetered compute. Once you own the hardware, there’s no per-token meter. That means you can run experiments, iterate on prompts, and process documents without the anxiety of a ticking bill. The marginal cost of your 1,000th document is the same as your 1st — electricity.
Capability floor. Qwen3.6-27B won’t beat GPT-5.5 on complex reasoning. But for structured tasks — field extraction, document classification, summarisation with a template — it’s genuinely good. The task is finding numbers in PDFs, not writing poetry. For that, it’s more than capable.
The hybrid model. The practical split: run sensitive workloads (document processing, data extraction, internal queries) locally, and use API for tasks where quality matters more than data residency (market research, content generation, non-sensitive analysis). This captures the benefits of both.
Being honest about limitations matters more than hype, so the limits are worth stating:
Cutting-edge reasoning. If you need the best possible output for complex reasoning, mathematical proof, or code generation, frontier models (GPT-5.5, Claude Opus 4.6) still outperform open-weight models at 27B parameters. No amount of local hardware changes that.
Low-volume use. If you’re sending 10 prompts a week, buying a GPU is throwing money at a problem API pricing already solved.
Interactive chat. 35 tokens/sec is acceptable for batch processing. For a real-time chatbot where users expect instant responses, it feels sluggish. Models under 10B parameters run faster but sacrifice quality.
Maintenance overhead. Running inference locally means you maintain the software stack, update models, handle driver updates, and troubleshoot when things break. An API is someone else’s problem. That’s a real benefit, not a trivial one.
For financial services in Singapore, the local vs API decision isn’t just about cost — it’s about compliance and risk. MAS Technology Risk Management guidelines (2022) require firms to assess risks from third-party dependencies, including data localisation concerns.
The hardware isn’t expensive by enterprise standards. The RTX 5090 at S$5,000–6,000 is cheaper than most companies spend on a single API subscription in a year. And it does other things — GPU compute for data analysis, local AI experimentation, even rendering. It’s an asset, not a subscription.
This exact setup powers Kapital’s internal document processing pipeline. The benchmark data above is from production use, not a lab test. The document processing benchmarks are covered in a separate memo.
This analysis is based on publicly available data as of 2026-07-03. For related coverage, see Local LLM Document Processing: Measured Benchmarks for Financial PDFs and Family Office Technology.
Claims marked [M] are Kapital measurements from Kapital’s own benchmark runs, not third-party data, and carry no public URL.