- A model's memory is roughly the number of parameters times bytes per parameter, plus memory for context and overhead.
- Quantization shrinks the size at some cost to quality, and longer context needs extra memory.
- Check on your own examples whether a smaller version still suffices.
The question “will this model run on my machine” has a simple answer if you know three numbers: how many parameters the model has, how many bits each takes and how long a context you want to handle. The calculation below is an arithmetic estimate, so treat results as an order of magnitude, not an exact measurement. If you are only choosing a tool to run it, first read the comparison of Ollama, LM Studio and llama.cpp.
Weights: parameters times bytes
A model is stored as billions of numbers, that is parameters (weights). Each takes as much memory as the number of bits allotted to storing it. At 16-bit precision that is two bytes per parameter, at 8-bit one, at 4-bit half a byte. A model with seven billion parameters therefore takes about 14 GB at 16-bit precision, about 7 GB at 8-bit and about 3.5 GB at 4-bit. A model with seventy billion parameters needs ten times more: about 140 GB, 70 GB and 35 GB. On top of that comes the overhead of the program and the system, which you need to add to the result.
| 16 bits | 8 bits | 4 bits | |
|---|---|---|---|
| 7B parameter model | about 14 GB | about 7 GB | about 3.5 GB |
| 30B parameter model | about 60 GB | about 30 GB | about 15 GB |
| 70B parameter model | about 140 GB | about 70 GB | about 35 GB |
Quantization: a smaller file, slightly worse quality
Quantization means storing weights with fewer bits. The file shrinks, and the model loads faster and fits in less memory. The price is usually some drop in quality, which depends on the model and the task. In practice 8-bit versions are usually close to the original, and heavier shrinking needs checking on your own examples. Tools such as llama.cpp and LM Studio use the GGUF format, in which you will meet several quantization variants of the same model; on the model’s page look for which variant the author recommends.
The decision rule is simple. Start with the variant that fits in memory with room to spare, run it on your cases, and only if the result is not enough try a larger variant or another model. Reaching too early for the largest variant simply because it “is better” often ends in slow work with no visible benefit.
Context takes memory too
Weights are not everything. The longer the context, the more memory is needed to handle it, and the Ollama documentation says so plainly: increasing the context length increases the memory requirement. By default Ollama picks the length according to the available graphics memory: below 24 GiB it is 4 thousand tokens, from 24 to 48 GiB 32 thousand, and from 48 GiB 256 thousand. Tasks that need large context, such as web search, agents and coding tools, according to the documentation should be set to at least 64 thousand tokens. This means a model that “fits” with the default context may not fit after raising it to the size an agent needs.
How to check what is really happening
The ollama ps command shows the size of the loaded model, the context length and a processor column split between GPU and CPU. The documentation explains that 100% GPU means it was loaded entirely into the card’s memory, and a split, for example 48% CPU and 52% GPU, means the model is partly in system memory, which slows work down. For best performance it advises avoiding offloading the model to the CPU.
- 01ParametersCheck the parameter count on the model card.
- 02BitsPick a quantization variant.
- 03WeightsMultiply parameters by bytes per parameter.
- 04ContextAdd memory for the planned length.
- 05MarginLeave room for the system and other programs.
- 06MeasureRun it and check ollama ps or equivalent.
A final practical tip: before you buy hardware for a model, run a test on what you already have with the smallest sensible variant. It often turns out that a smaller model suffices for your task, or on the contrary that using a cloud API is better than buying an expensive card. The choice between them depends on costs, which I cover in the article Task cost in tokens, and on whether you may send data off your computer. You will find all materials in the Local AI hub.
Last checked: 20 October 2026. Default values and command names change with program versions.