Estimates · inference only

LLM VRAM Calculator

Open-weights model dropped this week? Estimate what it takes to run it — weights + KV cache + overhead — and which GPUs clear the bar.

—
HardwareMemoryFits?

How the estimate works

Weights ≈ params × bits ÷ 8 (a 27B model at Q4 ≈ 13.5 GB). KV cache scales with context length (estimated per-token from model size; GQA models use less). We add ~15% runtime overhead (CUDA buffers, activations, framework). Real usage varies by architecture and runtime (llama.cpp vs vLLM) — treat results as a planning estimate, not a guarantee. Training and fine-tuning need far more.

New open models get covered every Friday on This Week in AIOps.

Get the field note by email

A short email when the calculators get new numbers, the directory gets new entries, and the week's episode lands. No hype — unsubscribe in one click.

Your address is used to send the field note and nothing else.

Who builds these tools

AIOps Field Notes is written by James Son, a network, security, and automation engineer, and everything here is tested in a live multi-vendor lab. That same lab work is behind two paid things for teams who run the infrastructure underneath the AI:

DriftWatch → Config-drift monitoring that explains what changed in plain English, with CIS/PCI checks. Read-only, never touches your gear. 14 days free, then from $29/mo. Config-drift & compliance audit → A one-time, flat-fee review of your network configs: what drifted, what is undocumented, and a CIS/PCI gap list — delivered as a client-ready report.