Quick answer
For local AI, the graphics card's VRAM matters more than processor power. Models with 7 to 8 billion parameters need 8 GB, 13 to 14 billion need 12 to 16 GB, and for models with 30 billion and more, 24 GB or more is realistic. If you only use it occasionally, 16 GB is a reasonable compromise.
How to recognize the problem
- The model loads slowly or does not fit into graphics memory at all
- The computer starts to stutter as soon as the AI model begins responding
- Fans run at full speed and responses are several seconds late
- One application works while another reports an out-of-memory error
- You bought an expensive graphics card and the model still runs from system memory
Most common causes
The model is larger than the VRAM
When the model does not fit into graphics memory, part of it is moved to system RAM or disk, so speed can drop tenfold.
Wrong format and quantization
The same model in full precision requires several times more memory than a quantized version, so the choice of file often matters more than the card itself.
Graphics card vendor and support
Some cards work with local AI tools with additional configuration, while others are supported out of the box, which affects how much VRAM can realistically be used.
Other programs hold memory
A browser with many tabs, games and video editing applications take up part of the VRAM before the model even starts loading.
Not enough system RAM
If the model is larger than the VRAM, the computer uses system memory, so 16 GB total quickly becomes a bottleneck.
Step-by-step fix
- 1
Define what you will run
First determine the purpose: chat and translation, image generation, audio transcription or working with documents. For conversation and text, a smaller model is enough, while image and video generation require more memory. Write down two or three specific models you want to use, because the card is chosen based on them, not the other way around.
- 2
Calculate the required VRAM
A rough calculation: the number of model parameters in billions multiplied by the size per parameter. At about 4-bit quantization, that is roughly 0.6 GB per billion parameters, so an 8 billion model needs about 5 GB, and a 14 billion model about 9 GB. Add 2 to 3 GB for context and application operation. At 16-bit precision, count on about 2 GB per billion parameters.
- 3
Choose the VRAM class
For models up to 8 billion parameters, 8 GB is enough. For 13 to 14 billion with a longer context, 12 to 16 GB is realistic. For models with 30 billion and more, as well as for more demanding image and video generation, 24 GB or more is recommended. If you occasionally use larger models, 16 GB is the best compromise between price and usefulness.
- 4
Check support for your system
On a Windows computer, check in
Task Manager > Performance > GPUhow much memory the card has and how much is currently used. For NVIDIA cards, install the current driver from the manufacturer's official website, because local AI tools most often work best with CUDA support. AMD and Intel cards are supported in newer tools, but with more configuration. - 5
Match the rest of the configuration
Along with the graphics card, provide at least 32 GB of system RAM if you plan to use models larger than the VRAM, because part of the model is then kept in system memory. The power supply must handle the card's peak load, and the case must have sufficient airflow. An SSD of at least 1 TB is practical because models take up several tens of GB each.
- 6
Test before buying
If you have a computer with any newer graphics card, run the desired model and monitor in
Task Manager > Performance > GPUhow much memory it uses and whether it spills into system memory. That gives you a realistic picture of how much VRAM you need. If the card does not have enough memory, a tool that supports spilling will run significantly slower, and some tools will report an error. - 7
Decide between a card and a prebuilt computer
A desktop card is installed in an existing case and is easy to replace later, but check the card length, the number of power pins and the power supply wattage. Laptops with a strong graphics card are more expensive and harder to upgrade, and a card with the same name in a laptop often has less VRAM than the desktop version. For constant work at home or in the office, a desktop is more cost-effective.
- 8
Install the tool and check operation
Install a tool for running models locally from the official website and load a model that matches your memory. Monitor the time to first response and the generation speed; if everything is fine, the model stays in VRAM. If the tool reports an out-of-memory error, reduce the model size or context length before thinking about a new card.
When it is better to go to the cloud or share one computer
Local AI makes sense when data must not leave the computer or when you use the model daily. If you run the model a few times a month, pay-as-you-go on a cloud service is often cheaper than a 24 GB card. In a company with multiple users, consider one stronger computer that shares the model over the local network, instead of everyone buying a strong graphics card. For experiments, a 12 GB card with smaller models is enough, and it can be upgraded later. Do not buy the strongest card in advance: first check which models you actually use.
How to prevent it
- Before buying, check the exact card model and the amount of VRAM, not just the series name
- Keep graphics drivers up to date from the manufacturer's official website
- Close unnecessary programs before running a larger model
- Provide at least 32 GB of system RAM if you use models larger than the VRAM
When to call a professional
Contact IT support if you are not sure which models you will use, if you are choosing a computer for multiple users in a company, or if the tool reports memory errors even after reducing the model.
IT infrastructure maintenance · Servers, networks, backup, security and helpdeskFAQ
Is 8 GB of VRAM enough for local AI?
For models up to 8 billion parameters in a quantized version, yes, with a shorter context. For larger models or longer conversations, more memory will be needed, otherwise the model runs significantly slower.
Is the graphics card or the processor more important?
For local AI, the graphics card and the amount of VRAM are crucial. The processor and disk speed affect model loading, but not the response speed when the model fits into the card's memory.
Can I use integrated graphics?
You can for smaller models, but speeds are significantly lower, and integrated graphics share memory with the system. For more serious work, a dedicated card with its own VRAM is recommended.