How to Fix 'Out of Memory' Errors When Running Local LLMs in Ollama and LM Studio

Quick Answer

To resolve "Out of Memory" (OOM) errors in Ollama and LM Studio, you must reduce the model's VRAM footprint by using a higher quantization level (such as Q4_K_M instead of Q8 or FP16), lowering the context window size, or adjusting GPU offloading ratios to split the layers between VRAM and system RAM. This prevents the LLM from overflowing your GPU's physical memory boundaries and causing application crashes or extreme slowdowns.

Systematic Solutions for Local LLM Memory Exhaustion

Local Large Language Models (LLMs) push consumer hardware to its absolute limits. Unlike standard software, an LLM must load its entire weight matrix directly into fast memory—ideally Video RAM (VRAM) on your graphics card—to run at acceptable speeds. When the combined weight of the model, the context window, and system overhead exceeds your available VRAM, the system crashes with a CUDA Out of Memory error, or drops back to agonizingly slow system RAM (CPU offloading).

The primary culprit is usually attempting to run a model that is too large or using a quantization level that is too high. Quantization compresses the model's weights from 16-bit floating-point (FP16) down to smaller bit-widths like 8-bit (Q8) or 4-bit (Q4) with minimal loss in intelligence. Here is how quantization levels affect memory footprint and system requirements for a typical 8-billion parameter model (such as Llama 3.1 8B):

Pricing above reflects publicly listed rates as of August 2026. Subscription pricing changes often — confirm current rates on the provider's own pricing page before subscribing.

If you are experiencing crashes, follow this sequence of concrete troubleshooting steps to diagnose and resolve the issue in both Ollama and LM Studio.

1. Reduce the Model Quantization

If you are trying to run a 7B, 8B, or 14B model at Q8_0 or FP16 and hitting OOM limits, download the Q4_K_M version instead. In LM Studio, you can filter search results by quantization level. In Ollama, you can specify the tag directly in your terminal (for example, ollama run llama3.1:8b-instruct-q4_K_M instead of the default or higher-quantized tags). Quantization reduces the memory footprint by up to 60% with almost no noticeable drop in conversational quality.

2. Adjust GPU Offload Ratios in LM Studio

LM Studio allows granular control over how many layers of the model are loaded onto your GPU versus your CPU (system RAM). If you experience a hard crash, your GPU is likely being asked to store more layers than its VRAM can handle. Locate the Hardware Settings panel on the right side of LM Studio and find the GPU Offload slider. Decrease the number of offloaded layers by 5 to 10 layers, then reload the model. This forces your system memory (RAM) to handle the overflow, preventing a crash at the expense of generation speed.

3. Restrict the Context Window Size

The context window (the history of your conversation plus the current prompt) consumes VRAM dynamically as your chat gets longer. A model that runs fine on the first prompt can trigger an OOM error fifty messages deep because the KV Cache (Key-Value Cache) has grown too large. In LM Studio, navigate to Context Length under the model settings and lower it from 8192 or 16384 tokens down to 4096 or 2048 tokens. For Ollama, you can set the context limit in a custom Modelfile using the num_ctx parameter (e.g., PARAMETER num_ctx 4096).

4. Terminate Background VRAM Hogs

Operating systems and background applications consume a surprising amount of VRAM. Web browsers with hardware acceleration enabled, video editing suites, and games can tie up several gigabytes of GPU memory before you even launch Ollama or LM Studio. If you are running on Windows, you may need to implement system optimization strategies. You can review how to speed up a slow Windows laptop to identify and disable background startup processes that drain system RAM and GPU resources. Additionally, check your Task Manager (Performance tab -> GPU) to ensure other software isn't silently reserving your VRAM.

5. Manage Ollama Concurrency and Multi-Model Settings

By default, Ollama may attempt to keep multiple models loaded in memory simultaneously or handle parallel requests, which rapidly exhausts VRAM. You can control this behavior using environment variables. Set OLLAMA_NUM_PARALLEL=1 to prevent Ollama from processing multiple requests concurrently on different VRAM chunks. You should also set OLLAMA_MAX_LOADED_MODELS=1 to ensure that loading a new model immediately evicts the old one from your GPU's memory. On Windows, you set these in the System Environment Variables panel; on macOS or Linux, export them in your terminal profile or systemd configuration.

6. Understand VRAM vs. Shared System Memory Allocations

Windows has a feature that automatically spills excess VRAM usage over into your system RAM (known as Shared GPU Memory). While this prevents hard crashes, it slows token generation down to a crawl—often less than one token per second. This bottleneck is highly similar to issues found in other hardware-heavy local AI processes, such as when an AI image generator keeps failing or stalling due to memory misallocation. If you notice your generation speeds drop off a cliff, your system has likely run out of dedicated VRAM and is paging memory over PCIe to your system RAM. The fix is to reduce the model size or the context length until your active memory usage stays entirely within your dedicated VRAM limit.

Hardware-Driven Model Selection: What You Should Run

To avoid trial-and-error OOM crashes entirely, you must align your model selection with your exact hardware specifications. Do not attempt to run models that require more memory than your dedicated VRAM can provide, unless you are willing to accept incredibly slow CPU-bound speeds.

• If you have 8 GB VRAM (e.g., RTX 3060/4060, Apple M-Series Base): Your absolute limit for comfortable local execution is an 8B parameter model at Q4_K_M quantization (such as Llama 3.1 8B). Do not attempt to run 14B or larger models. Keep your context length capped at 4096 tokens to prevent VRAM overflow during extended sessions.
• If you have 12 GB to 16 GB VRAM (e.g., RTX 4070/4080, Apple M-Series Pro): You are in the sweet spot for local AI. You can comfortably run 8B models at Q8_0 or FP16, or step up to 14B or 15B models at Q4_K_M (such as Qwen 2.5 14B). You can also expand your context window to 8192 tokens without risking OOM errors.
• If you have 24 GB VRAM or Unified Memory (e.g., RTX 3090/4090, Apple Mac Studio with 64GB+ RAM): You can execute 32B models at Q4_K_M or run 70B models at highly compressed quantizations (like IQ2_XXS or Q3_K_S). This hardware class allows for extensive context windows up to 32,000 tokens on smaller models.

If your local hardware cannot reliably run these models without throwing memory errors—or if you need advanced reasoning for complex coding operations—relying entirely on local hardware might not be the most productive choice. In those instances, offloading intensive tasks to dedicated developer environments makes sense. Many professional engineers pivot to cloud-powered setups by choosing among the best AI coding assistants, which leverage state-of-the-art hosted models like Claude Opus 5 or GPT-5.6 Sol without taxing local VRAM. However, if offline privacy is your non-negotiable priority, sticking to a strictly bounded 8B Q4 model on your local machine remains the most reliable path forward.

Information accurate as of August 2026 — hardware requirements, model architectures, and local runner software capabilities are subject to frequent updates, so verify current technical specifications on official documentation channels before making dedicated hardware upgrades.