Local LLMs on an $800 mini PC: benchmarking the Ryzen 7 255 and Radeon 780M

Prefill and decode numbers for Qwen3-30B-A3B and gpt-oss-20b on a Reatan X7 (Ryzen 7 255, Radeon 780M) with llama.cpp and Vulkan on Linux, and why one missing RAM stick matters more than the GPU.

8 min read

Before spending Strix Halo money on a laptop, I wanted a baseline: how well does the mini PC already on my desk run modern open-weight models? The machine is a Reatan X7 with a Ryzen 7 255 and its integrated Radeon 780M, running Omarchy (Arch). I ran two mixture-of-experts models fully on the iGPU through llama.cpp’s Vulkan backend.

The short version: both models are usable on the 780M, at roughly 15–20 tokens/s for generation. But the number that explains the results isn’t anything about the GPU. It’s that this box shipped with a single 48 GB stick of RAM.

The setup

CPUAMD Ryzen 7 255 (8 Zen 4 cores)
GPURadeon 780M (RDNA 3, 12 CUs), RADV / Mesa 26.2.5
Memory1 × 48 GB DDR5-5600 (one slot empty)
GPU-usable RAM (GTT)22 GiB (the default: half of system RAM)
OSOmarchy, kernel 7.2.8
Inferencellama.cpp b12032, Vulkan build, all layers on the GPU (-ngl 99)
Power profileperformance
Price$800 on sale during Prime Day ($1,200 list) for the 48 GB config

Each test ran 3 times. pp is prefill (reading the prompt), measured at 512 and 4096 tokens. tg is decode (generating 128 new tokens).

Results

ModelSizepp512 (tok/s)pp4096 (tok/s)tg128 (tok/s)
Qwen3-30B-A3B Q4_K_M17.3 GiB397 ± 14379 ± 119.4
gpt-oss-20b MXFP411.3 GiB521 ± 4506 ± 0.314.7

A few things stand out:

  • Prefill holds up at long contexts. Going from a 512- to a 4096-token prompt costs only about 3–5% of throughput. A 4k-token prompt is ingested in about 8 seconds with gpt-oss and 11 with Qwen.
  • Decode speed is fine for chat. 19 tok/s is faster than most people read. It’s slow for agentic loops that generate thousands of tokens per step.
  • gpt-oss-20b decodes slower than the bigger Qwen. That’s surprising, since gpt-oss is smaller on disk and activates a similar number of parameters per token (~3.6B vs ~3.3B). One plausible cause: llama.cpp reports fp4: 0 for the 780M, so the MXFP4 weights can’t use native FP4 math and have to be converted on the fly. I haven’t verified that this is the cause.

Why one RAM stick matters

Token generation on an iGPU is limited by memory bandwidth. To produce each token, the GPU has to read every active weight from RAM. Compute barely matters.

One DDR5-5600 DIMM gives ~44.8 GB/s of theoretical bandwidth. Qwen3-30B-A3B activates ~3.3B parameters per token, which at Q4_K_M (~4.85 bits/weight) is about 2 GB read per token. That puts the ceiling at roughly 22 tok/s. I measured 19.4, close to 90% of the theoretical limit. The GPU isn’t the bottleneck. The memory bus is.

With the second slot empty, the 780M is running on half the memory bandwidth the platform supports. Adding a matching second 48 GB stick should roughly double decode speed (toward ~35–40 tok/s for Qwen3-30B-A3B), and it would also double the GTT pool, leaving more room for context or bigger models.

Compared with Strix Halo

The reason for this test was the Strix Halo decision (Ryzen AI Max+ 395, as in the ROG Flow Z13). For reference, a Strix Halo mini PC (GMKtec EVO-X2) on Linux + Vulkan runs the same Qwen model at:

Qwen3-30B-A3B Q4_K_Mpp512tg128
Reatan X7 (780M, 1 stick)39719.4
Strix Halo (8060S, 256-bit LPDDR5X)74986
Strix Halo advantage1.9×4.4×

The prefill gap is modest. Strix Halo has more than three times as many GPU compute units, but prefill on the 780M is already efficient. The decode gap is the bandwidth gap: ~256 GB/s vs ~45 GB/s. Even with a second DIMM, the X7 would still be ~2.4× behind on generation speed.

So the deciding question isn’t “can I run local models?” (yes, today, on a mini PC). It’s whether 4× faster generation and up to 128 GB of unified memory justify the price. For interactive chat with 20–30B MoE models, a properly dual-channel mini PC is good enough. For agent loops and 70B+ models, it isn’t.

Trying Strata instead of llama.cpp

Strata is an inference engine built around Qwen3.8-Flash-Next. It streams the model’s experts between RAM, the GPU and the SSD, so it can run a much bigger model than fits in GPU memory. It has an experimental path for the 780M (gfx1103), and someone has already measured it on a Ryzen 7 255. So I tried its coding model, Qwen3.8-Flash-Next Coder (IQ1_M, a 58 GB download), as a drop-in replacement for llama.cpp.

Setup worked better than I expected. Strata 0.2.10’s check said this PC could run the model. Then it pulled AMD’s ROCm into a Python virtualenv (no sudo, no system ROCm) and compiled the engine for gfx1103. The only warning was that my hipBLASLt version (100200) didn’t match the 780M tuning table (100401), so prompt processing would fall back to plain hipBLAS.

Running it didn’t work. I tried three configurations, and each one died on the first real prompt (about 600 tokens):

TryConfigurationResult
1Default: ~30 GB of experts in RAM plus a 12.6 GB expert cache on the GPUGPU reset mid-prompt. The system was 12 GB into swap
24 GB of GPU memory reservedWouldn’t start: after loading the experts, the OS had 1.7 GB left, so there was no room for a GPU cache
3Low-RAM mode (experts memory-mapped from the SSD)Loaded with 114 MB of GPU memory free, then the same GPU reset

The kernel log was the same each time:

amdgpu: failed to remove hardware queue from MES
amdgpu: MES might be in unrecoverable state, issue a GPU reset
amdgpu: Failed to evict process queues
amdgpu: GPU reset(1) succeeded!

Strata’s documentation describes this exact failure. When the kernel moves memory the GPU has registered, the 780M’s queue manager firmware (MES) can stop responding. The engine process dies and the request fails. The documented 780M run used a machine with 60 GB of RAM, 50 GiB available to the GPU and the amdgpu.cwsr_enable=0 kernel option, and didn’t hit it. With 48 GB, the experts and the GPU cache don’t fit together, so the memory runs near full, and the kernel ends up evicting GPU memory, which sets off the bug.

The only request that completed was a 55-token warm-up: 12.6 tok/s for the prompt and 6.3 tok/s for generation, on a cold cache. That’s too small to compare with the numbers above. I didn’t try cwsr_enable=0, because it needs a kernel parameter and a reboot. On a 48 GB machine, I’d still expect memory to be the limit.

Reproducing this

I used a short bash script that installs the Vulkan bits, fetches a llama.cpp Vulkan build and the GGUFs, then runs llama-bench:

llama-bench -m model.gguf -ngl 99 -p 512,4096 -n 128 -r 3 -o md

Two things broke since I first wrote the script, so watch for them:

  • llama.cpp’s “latest” GitHub release no longer carries binaries. The Vulkan builds ship as llama-bXXXXX-bin-ubuntu-vulkan-x64.tar.gz in the numbered pre-releases.
  • The gpt-oss GGUF on Hugging Face is now gpt-oss-20b-MXFP4.gguf (uppercase), so the old lowercase URL returns 404.

Also check how your RAM is populated before benchmarking. udevadm info -q property /sys/devices/virtual/dmi/id | grep MEMORY_DEVICE shows it without root.

Written by

Farez Vadsaria

Senior engineer on AI voice services at Medsender, previously identity & auth at Warner Bros and email infrastructure at Amazon. Builds free AI side projects with coding agents, and teaches how they work.

Get the next post when it ships.