New guides and project ideas on X: Follow @3DeltaD

VRAM calculator: will your AI model fit on a GPU?

VRAM is the memory on a GPU, and an AI model has to fit in it to run at full speed. Enter a model, a quantization (how much the model is compressed) and how you plan to run it. 3DeltaD Builder estimates the VRAM you need and lists the RunPod GPUs that can hold it, with hourly and monthly cost side by side.

Uses real RunPod pricing, fetched from the RunPod API within the last 30 days. Free to use, no sign-up.

Some links here are referral links: we may earn credits if you sign up and use RunPod.

GPU calculator

Your workload

1K to 128K tokens. A token is a piece of a word. Context is how much text the model keeps in view at once.

How many requests run at the same time. Each one needs its own cache memory. Use 1 if it is just you.

Workload type: run a model (inference) or train it further (fine-tuning)
Cloud type

Estimated VRAM

0 GB total, approximate

    Recommended GPUs (Secure Cloud, prices checked within the last 30 days)

    GPUs that can hold the estimated VRAM, sorted by hourly cost
    GPU VRAM Fit $/hr 10 h/mo 50 h/mo 100 h/mo 730 h/mo

    Fits = at or below 90% of GPU memory. Tight = 90-100%, little headroom for longer prompts or framework differences. Monthly figures assume the pod runs for the stated hours; 730 h is a full month. Prices are RunPod on-demand rates as of the last 30 days and may change; check RunPod for current pricing.

    VRAM estimates are approximate. Actual requirements may vary based on software framework, batch processing, and other factors.

    Picked a GPU? Plan what one session costs, and what a stopped or forgotten pod keeps billing.

    Ready to try it on real hardware?

    Start a pod (a rented machine) with the GPU you picked and check the real memory use against this estimate.

    Referral link: we may earn credits if you sign up.

    Create a RunPod account

    How VRAM is calculated

    The calculator adds four parts together. All of it is arithmetic on published model dimensions; nothing here is a benchmark. The calculator has never met your GPU.

    1. Model weights = parameters (billions) x bytes per parameter. FP16/BF16 uses 2 bytes, INT8 uses 1, and 4-bit formats use about 0.56 once group-quantization scales are counted. GGUF variants range from 0.60 (Q4_K_M) to 1.06 (Q8_0).
    2. KV cache = 2 x layers x KV heads x head dimension x 2 bytes per token (FP16 cache), multiplied by context length and batch size. Models with grouped-query attention (such as Llama 3.1 with 8 KV heads) need far less cache than older full-attention designs.
    3. Activations: about 10% of the weight size for inference.
    4. Overhead: 1.5 GB per GPU for CUDA kernels and the serving framework.

    Fine-tuning roughly doubles the weight, cache and activation figures as a rough allowance for gradients and optimizer state. LoRA or QLoRA can use less; full fine-tuning with Adam can use more.

    When a model does not fit on one GPU, the calculator tries 1, 2, 4 and then 8 identical GPUs (up to the per-pod maximum RunPod offers) and picks the smallest of those whose combined memory holds the model, adding the per-GPU overhead each time. It does not model communication slowdowns between GPUs.

    total = weights + kv_cache + activations + overhead
    weights = params_B x bytes_per_param
    kv_cache = 2 x layers x kv_heads x head_dim x 2 B x context x batch / 1024^3

    Popular models at common quantizations

    Estimated total VRAM at 4,096 tokens of context, batch size 1, inference. Generated by the same code as the calculator.

    Estimated VRAM by model and quantization
    ModelParamsFP16INT8INT4

    FAQ

    What is VRAM and why does it matter?

    VRAM is the memory on the GPU. A model's weights, its attention cache, and working buffers must all sit in VRAM to run at full speed. If they do not fit, the job fails with an out-of-memory error or has to offload to slower system memory.

    Why does quantization matter?

    Quantization stores weights in fewer bits. Going from FP16 to 4-bit cuts weight memory by roughly 3.5x, which can move a model from multi-GPU to a single card. It can reduce output quality to varying degrees depending on the method and model, so test on your own task.

    Why does context length change the result?

    Each token in the context stores key and value vectors for every layer. The cache grows linearly with context length and with the number of concurrent requests, which is why long contexts and high concurrency can dominate memory for smaller models.

    RunPod vs a local GPU: which is cheaper?

    It depends on how many hours you use. Renting suits occasional or bursty work, large GPUs you could not otherwise afford, and trying hardware before you buy. A GPU you own can be cheaper if you run it most of the day for a long time. Use the monthly columns above to compare against the price of a card you are considering, and remember to include electricity.

    If you want to try RunPod, here is our referral link. Referral link: we may earn credits if you sign up.

    What is the difference between Secure Cloud and Community Cloud?

    On RunPod, Secure Cloud runs in vetted data centers and is priced higher. Community Cloud uses third-party hosts at lower prices. Availability varies by GPU. This tool lets you switch pricing between the two.

    How accurate are the estimates?

    They are approximations. Serving frameworks differ in how they allocate memory (some pre-reserve a fraction of VRAM for cache), and options like speculative decoding, LoRA adapters, or larger vision encoders add more. Leave headroom and verify on real hardware.

    About

    3DeltaD Builder is a free set of calculators and guides for makers. No sign-up.