When we set up our local inference stack, we asked a simple question: is buying a GPU actually cheaper than paying for API access? The honest answer: almost never on cost alone. The real calculus is different.
The AI industry has a default answer for every problem: send your data to an API, pay per token, get results. For most people, that's the right choice. But we've been thinking about this differently, because the businesses we work with process sensitive data — financial statements, client portfolios, internal documents — and sending that overseas introduces problems that per-token pricing doesn't solve.
Three of them, specifically:
Data residency. Your documents leave your control. Even if the provider promises not to train on your data, the data crosses borders and enters someone else's infrastructure. In Singapore, MAS guidelines on data localization aren't optional for financial services — they're the floor, not the ceiling.
Cost predictability. Per-token pricing looks cheap until you're processing at scale. At high volume, the annual bill compounds into a significant operating expense with no ceiling. You're paying rent that never becomes equity.
Latency predictability. API responses depend on network conditions and provider load. For workflows where an LLM is one step in a pipeline — ingestion, extraction, review — consistent latency matters more than peak speed.
The counterargument is straightforward: local inference means buying hardware, maintaining software, and accepting lower capability than frontier models. The question is whether the trade-off is worth it at today's hardware prices.
We decided to test this with a concrete setup rather than speculate. Here's what we found.
Test environment (July 2026):
| Component | Specification |
|---|---|
| CPU | AMD Ryzen 9 9950X3D |
| GPU | NVIDIA RTX 5090, 32GB GDDR7 VRAM |
| RAM | 64GB DDR5 |
| OS | Windows 10 |
| Inference server | LM Studio v0.4.x |
| Model | Qwen3.6-27B (Q4_K_M quantization) |
We chose Qwen3.6-27B for a specific reason: it's large enough for meaningful document processing and reasoning tasks but small enough to run on consumer hardware with 4-bit quantization. The Q4_K_M quantization was a deliberate trade-off — higher precision would be slower, lower precision would degrade output. We wanted the middle ground.
For the test, we generated a 1,764-word financial analysis document — a realistic workload for document summarization and report drafting. We measured tokens per second, total generation time, and compared the cost against equivalent API calls. All tests ran on this machine with no other GPU workloads.
Inference speed: 35 tokens per second, generating 1,764 words (2,048 tokens) in 50.6 seconds.
Now, GPT-5.4 streaming through OpenAI's API typically returns the same output in 15–20 seconds. The local model is 2.5–3x slower. But here's what we think matters: "slower" depends on your use case. For a document processing pipeline where the LLM is one step among many, adding 30 extra seconds per document is acceptable. For interactive chat where a user is waiting for a response, it's not. The question isn't "is it fast?" — it's "is it fast enough for the job?"
Output quality: The generated financial analysis was coherent, numerically accurate, and structurally sound. For document processing tasks — extracting fields, categorizing entries, summarizing sections — Qwen3.6-27B at Q4 quantization produces usable output. For creative writing or nuanced reasoning, the gap vs frontier models is more noticeable. We should be honest about this: if you need the best possible output for complex reasoning, models like GPT-5.5 or Claude Opus 4.6 still win. No amount of local hardware changes that.
Memory footprint: The model loaded into 18.2GB of the available 32GB VRAM, leaving headroom for concurrent tasks. We ran an 8B parameter orchestrator model simultaneously without issues.
| Metric | Qwen3.6-27B (local) | GPT-5.4 (API) |
|---|---|---|
| Tokens/sec | 35 | ~100-150 |
| Latency (2048 tokens) | 50.6s | ~15-20s |
| Cost per run | $0 (after hardware) | ~$6.50 |
| Data leaves premises | No | Yes |
| VRAM required | 18.2GB | None |
This is where the industry gets it wrong. A lot of articles claim local inference "pays for itself" quickly. We ran the actual numbers and the result is less flattering for the hardware side than you'd expect.
API pricing as of July 2026 (converted to SGD at 1.35 USD/SGD):
| Model | Input (per M tokens) | Output (per M tokens) |
|---|---|---|
| GPT-5.4 | SGD 3.38 | SGD 13.50 |
| Claude Sonnet 4.6 | SGD 4.05 | SGD 20.25 |
| Qwen3.6-27B (local) | SGD 0 | SGD 0 |
The hardware cost for this setup is approximately SGD 8,000–10,000 (the RTX 5090 alone is ~SGD 5,000–6,000 in Singapore). Here's when that investment actually pays for itself against GPT-5.4 API costs at a 60/40 input/output split:
| Monthly Volume | Annual API Cost | Break-even |
|---|---|---|
| 1M tokens | SGD 89 | 101 years |
| 10M tokens | SGD 891 | 10.1 years |
| 34M tokens | SGD 3,000 | 3 years |
| 50M tokens | SGD 4,455 | 2 years |
| 100M tokens | SGD 8,910 | 1 year |
That last row — 100M tokens/month — is an enormous volume. For context, a typical document processing pipeline at a mid-sized firm might process 5–10M tokens/month. At that level, API pricing wins on pure cost by a wide margin.
The honest takeaway: local inference does not pay for itself on cost alone unless you're processing at enterprise scale. The economic case for local hardware is about cost predictability at sustained high volume, not about saving money at moderate volumes. If you're processing less than 20M tokens/month, API is cheaper, and we'd be the first to tell you so.
Given the cost math above, local inference only makes sense for reasons other than economics. For us, those reasons are:
Data sovereignty. The primary driver. Financial documents, client portfolios, internal analysis — keeping that data on-premise isn't a luxury, it's a compliance requirement in regulated industries. You can't outsource data residency to a terms-of-service clause.
Unmetered compute. Once you own the hardware, there's no per-token meter. That means you can run experiments, iterate on prompts, and process documents without the anxiety of a ticking bill. The marginal cost of your 1,000th document is the same as your 1st — electricity.
Capability floor. Qwen3.6-27B won't beat GPT-5.5 on complex reasoning. But for structured tasks — field extraction, document classification, summarization with a template — it's genuinely good. We're not asking it to write poetry; we're asking it to find numbers in PDFs. For that, it's more than capable.
The hybrid model. The practical approach we've settled on: run sensitive workloads (document processing, data extraction, internal queries) locally, and use API for tasks where quality matters more than data residency (market research, content generation, non-sensitive analysis). This captures the benefits of both.
Being honest about limitations matters more than hype, so let us be clear about when local inference is the wrong choice:
Cutting-edge reasoning. If you need the best possible output for complex reasoning, mathematical proof, or code generation, frontier models (GPT-5.5, Claude Opus 4.6) still outperform open-weight models at 27B parameters. No amount of local hardware changes that.
Low-volume use. If you're sending 10 prompts a week, buying a GPU is throwing money at a problem API pricing already solved.
Interactive chat. 35 tokens/sec is acceptable for batch processing. For a real-time chatbot where users expect instant responses, it feels sluggish. Models under 10B parameters run faster but sacrifice quality.
Maintenance overhead. Running inference locally means you maintain the software stack, update models, handle driver updates, and troubleshoot when things break. An API is someone else's problem. That's a real benefit, not a trivial one.
For financial services in Singapore, the local vs API decision isn't just about cost — it's about compliance and risk. MAS Technology Risk Management guidelines (2022) require firms to assess risks from third-party dependencies, including data localization concerns.
The hardware isn't expensive by enterprise standards. The RTX 5090 at SGD 5,000–6,000 is cheaper than most companies spend on a single API subscription in a year. And it does other things — GPU compute for data analysis, local AI experimentation, even rendering. It's an asset, not a subscription.
We use this exact setup to power our internal document processing pipeline. The benchmark data above is from production use, not a lab test. We wrote about the document processing benchmarks in a separate memo.