Run and Speed Up Qwen3.8-27B in LM Studio: Real RTX 2070 8 GB Test

LM Studio lists Qwen3.8-27B as a dense 27B vision-language model for coding, professional work, research and long-horizon agentic tasks. It supports vision, tool use, configurable reasoning and a native 262K context window.

This guide documents a real Windows test rather than a synthetic benchmark: the initial engine-startup error, model loading, GPU-offload experiments, RTX 2070 8 GB VRAM limits and Think/Reasoning behavior are shown with the original screenshots.

The test machine used an Intel Core i7-10875H, 63.84 GB RAM and NVIDIA GeForce RTX 2070 8 GB CUDA GPU. All speed numbers are specific to this hardware.

Immediately after the download, the first Load Model attempt showed “Model failed to load” and “Engine protocol startup was aborted”. The message alone does not prove a single cause such as CUDA failure or insufficient RAM; it means the inference-engine startup was interrupted before the model became ready.

Confirm the download completed, verify the model appears correctly, check the runtime and logs, then retry. If needed, reduce context length and GPU offload and inspect memory/CUDA/Vulkan errors.

In this test the same model loaded successfully after waiting and using Load Model again, so deleting the model was not the first action needed.

The initial profile used context 8192, GPU offload 21, CPU thread pool 6, evaluation batch 2048, physical batch 512 and four concurrent predictions.

A simple greeting produced about 2.74 tokens/sec, but the UI also showed 13.66 seconds of thinking. Perceived latency therefore came from both generation speed and reasoning before the visible answer.

Keep prompts and chat history consistent when comparing load settings.

We reduced context to 4096, raised GPU offload from 21 to 23, set evaluation batch to 1024 and reduced Max Concurrent Predictions to 1 for a single-user test.

After reloading, the model reached about 2.89 tokens/sec. This was a small but measurable improvement over the 21-offload result while remaining stable.

CPU thread pool stayed at 6. More threads are not automatically faster; benchmark fixed values on your own CPU.

Context Length: 4096
GPU Offload: 23
CPU Thread Pool: 6
Evaluation Batch: 1024
Physical Batch: 512
Max Concurrent Predictions: 1
Unified KV Cache: ON

Moving the slider to 65 sounds attractive, but a 27B model and its runtime buffers cannot fully fit in the RTX 2070's 8 GB dedicated VRAM.

With the dedicated-memory limiter disabled, the 65-offload experiment dropped to about 1.30 tokens/sec, far below the 2.89 tokens/sec seen at 23 offload.

Shared GPU memory can become a bottleneck on Windows. LM Studio provides a dedicated-memory limiter specifically to keep model weights away from shared GPU memory.

The Hardware page showed the test machine's i7-10875H, 63.84 GB RAM and RTX 2070 8 GB VRAM. “Limit Model Offload to Dedicated GPU Memory” was initially disabled.

LM Studio states that enabling this setting limits model weights to dedicated GPU memory; when they do not fit, LM Studio reduces GPU offload and keeps the remainder in system RAM.

KV cache GPU offload remained enabled in this test. Benchmark it separately if VRAM is tight.

Windows shortcut: Ctrl + Shift + H
RTX 2070: ON
Limit Model Offload to Dedicated GPU Memory: ON
Offload KV Cache to GPU Memory: ON

After enabling the limiter, the load dialog explicitly said that actual offloaded layers may differ from the requested number.

The limiter-enabled 65 request produced about 2.45 tokens/sec: much better than 1.30 without the limiter, but still below the 2.89 result from the manual 23-layer profile.

The best setting therefore still needs to be benchmarked on each machine.

The screenshots show 13.66, 26.42 and 38.52 seconds of thinking on very short greetings. Much of the perceived delay was reasoning before the final answer.

Turning Think off makes simple chat respond directly. Re-enable reasoning for coding, mathematics, planning or complex analysis where depth matters more than latency.

The first Think-off test was still inside an older conversation, so it was not a clean benchmark.

Fast everyday profile: Think = OFF
Complex analysis: Think = ON + Reasoning Low/Medium

Chat history increases the number of input tokens that must be processed, so a fresh zero-token chat is useful for comparisons.

In the fresh conversation, Think was disabled and the short greeting finished at about 2.82 tokens/sec in 4.35 seconds.

For a reliable benchmark use a fixed 150-300 token answer prompt and compare tokens/sec, total time and first-token latency.

Benchmark prompt:
Explain how to build a secure user login system with PHP PDO in about 300 words.

A sensible starting point from this test is 4096 context, six CPU threads, evaluation batch 1024, physical batch 512, one concurrent prediction and the dedicated-memory limiter enabled.

Keep Think off for everyday chat. Speculative Decoding remained active in the screenshots with roughly 46-59% draft-token acceptance, so benchmark it separately on and off.

Start with GPU offload around 23 and compare against a limiter-enabled maximum request instead of assuming one value is universal.

Context: 4096
CPU Threads: 6
Evaluation Batch: 1024
Physical Batch: 512
Concurrent Predictions: 1
Dedicated GPU Memory Limit: ON
KV Cache GPU Offload: ON
Think: OFF for everyday use
GPU Offload: start near 23 and benchmark

Qwen3.8-27B can run on an 8 GB card with enough system RAM, but partial RAM/CPU execution limits generation speed. Long context, vision, concurrent requests and higher tokens/sec targets increase the value of more VRAM.

LM Studio's general Windows recommendation is at least 16 GB RAM and 4 GB dedicated VRAM, but a 27B model must also be sized for its actual quantization and context requirements.

For hosted local-AI workloads, size CPU, RAM and GPU from the model format, context size, concurrent users and target throughput rather than parameter count alone.