An r/LocalLLM thread asked the quiet question: "What are you doing with your local LLMs that justifies the hardware cost?" Most replies fell into three categories: privacy-required workloads, high-volume inference that would cost more via API, and hobbyist experimentation. The honest answer depends on your query volume.
When local LLMs pay for themselves
A 3090 costs ~$900 used. Claude Sonnet at $3/M input tokens processes ~300M tokens before the GPU pays for itself. If you run fewer than 300M tokens/month, cloud APIs are cheaper. If you run more, local wins on pure compute cost. The crossover is lower for smaller models (Qwen 7B, Phi-3) where inference is faster.
The missing piece: search grounding
Local models hallucinate on current facts. The cheapest fix is injecting search results into the prompt. This adds ~$0.005/query via Scavio but saves the model from generating (and the user from correcting) fabricated answers. For factual workloads, search grounding is the difference between "usable" and "toy."
# Add search to any Ollama model via MCP
# Works with Claude Code, Cursor, or any MCP-compatible runtime
claude mcp add scavio https://mcp.scavio.dev/mcp \\
--header "x-api-key: $SCAVIO_API_KEY"Use cases that justify the cost
- Privacy-sensitive document processing (legal, medical, HR)
- High-volume classification or extraction (10K+ docs/day)
- Offline or air-gapped environments
- Agent loops where each step costs tokens (local = flat cost)
- Experimentation and fine-tuning on proprietary data
Use cases that do not
- Occasional Q&A (cheaper via API)
- Tasks requiring frontier reasoning (GPT-4o, Claude Opus)
- Production systems needing SLA guarantees (cloud APIs have them)
The honest take
Most hobbyists will not recoup the hardware cost in saved API fees. The real justification is privacy, experimentation, or volume. If you run a local model for factual tasks, search grounding is non-optional. A grounded 7B model beats an ungrounded 70B model on factual accuracy every time.