Part of what I want from a local model is work I can queue up and leave: small fixes that run overnight with nobody watching. This test is closer to agent work than my animation scene test. It’s one of four posts from my local LLMs page, which also explains the stack and the model types if any of the terms here are new.
The short version: Qwen3.8-Flash-Next and Qwen3.8 27B fixed all three projects. Qwen3-30B-A3B is fast but unreliable.
The setup
Everything ran on the laptop: an ASUS ROG Flow Z13 with a Ryzen AI Max+ 395 (Strix Halo) and its Radeon 8060S integrated GPU, with 96 GB of its 128 GB set aside for the GPU, on Windows 11 with llama.cpp b12387 and the Vulkan backend. How fast each model runs there is in Strix Halo laptop vs RTX 3090.
Each run gives the model a small broken JavaScript project, a short task and a few file tools (list, read, write, edit, run tests), then lets it loop with nobody to answer questions. Afterwards the harness re-runs the original tests and a set of hidden checks the model never saw. A pass means every visible test and every hidden check passed.
Results
All on the Z13, thinking capped at 4,000 tokens:
| Project (visible tests / hidden checks) | Qwen3.8-Flash-Next | Qwen3.8 27B + MTP | Qwen3-30B-A3B |
|---|---|---|---|
| Sales summary (6 / 17) | pass, 99 s | pass, 102 s | fail (4/6, 16/17), 403 s |
| Interval merge (4 / 14) | pass, 41 s | pass, 50 s | pass, 130 s |
| Dispatch queue (8 / 32) | pass, 105 s | pass, 193 s | fail (0/8, 0/32), 140 s |
- Flash-Next and Qwen3.8 27B passed everything, in 41 to 193 seconds each.
- Qwen3-30B-A3B is fast but unreliable. On the dispatch queue it declared the job done with a syntax error in the file, so nothing ran.
Tool count can cost time
I reran one project with 91 tool definitions in the prompt instead of 5 (86 of them irrelevant: git, browser, email and so on). Flash-Next still passed but took twice as long (88 s vs 41 s). A3B’s time didn’t change (131 s vs 130 s).
Next
I ran the same three projects on AMD’s new Loom engine, with Qwen3.8 27B and two Ornith 1.5 models: see Loom vs llama.cpp on Strix Halo.
Caveats
- Single runs. Every result above is one attempt. A different seed could pass or fail differently, especially on the borderline ones.
- Windows only, so far. Linux, with ROCm, is next.
- Model settings differ between columns where noted (MTP on for Qwen3.8 27B).