I make an explainer-video series, engineering.fm, whose animations are written as code (Remotion scenes in React). I wanted to know whether open-weight models running on my own machines could take on real parts of that work, well enough to queue it up and let it run overnight. This is one of four posts from my local LLMs page, which also explains the stack and the model types if any of the terms here are new.
The short version: yes, for the right model. Qwen3.8-Flash-Next on the laptop and Qwen3.8 27B on the desktop both rewrote a full 44-second scene from a detailed spec and came out close to the original (scores of 0.94–0.97 out of 1). But capping the model’s thinking mattered more than I expected, the fast models weren’t good enough, and the spec is the ceiling.
The machines
| Laptop | Desktop | |
|---|---|---|
| Machine | ASUS ROG Flow Z13 | Desktop PC |
| CPU | Ryzen AI Max+ 395 (Strix Halo) | Ryzen 9 5900X |
| GPU | Radeon 8060S (integrated) | RTX 3090 |
| Memory for the model | 96 GB of the 128 GB, set aside for the GPU | 24 GB on the card |
| Backend | llama.cpp b12387, Vulkan | llama.cpp b12387, CUDA 12.4 |
| OS | Windows 11 | Windows 11 |
How fast each model runs on each machine is in Strix Halo laptop vs RTX 3090.
The test
The test takes a real scene out of my video project, gives a model a written spec plus the project’s shared components, and asks it to write the scene file back. A script then checks the result: does it typecheck, does it render, and how close are its frames to the original’s? The score runs from 0 (no closer than a blank frame) to 1 (identical). If the code doesn’t compile, errors go back to the model, up to three rounds.
Results
The hard task is a whole 44-second recap scene, from a detailed build plan with the layout numbers and timings:
| Machine | Model | Thinking | Result | Model time |
|---|---|---|---|---|
| Z13 | Qwen3.8-Flash-Next | capped at 4,000 tokens | 0.943, after 2 fix rounds | 21 min |
| Z13 | Qwen3.8-Flash-Next | uncapped | 0.972, after 2 fix rounds | 32 min |
| Z13 | Qwen3.8 27B + MTP | uncapped | no code: thought until the budget ran out | — |
| Z13 | Nemotron 3.5 Lightning | uncapped | compiled, scored 0.103 | — |
| Z13 | Qwen3-30B-A3B | uncapped | didn’t compile after 3 fix rounds | — |
| 3090 | Qwen3.8 27B + MTP | capped at 4,000 tokens | 0.948, after 2 fix rounds | under 10 min |
| 3090 | Nemotron 3.5 Lightning | capped at 4,000 tokens | didn’t compile after 3 fix rounds | — |
| 3090 | Qwen3-30B-A3B | capped at 4,000 tokens | didn’t compile after 3 fix rounds | — |
What I took from it:
- Cap the thinking. These models reason before they answer, and without a limit (llama.cpp’s
--reasoning-budget) Qwen3.8 27B spent every token planning. Capped, the same model on the 3090 wrote one of the best scenes of the lot. On Flash-Next, the cap cost a little quality (0.943 vs 0.972) and saved a third of the time. - The 3090 was fastest when the model fit. Qwen3.8 27B with MTP and a capped budget produced a 0.948 scene in under 10 minutes.
- The fast mixture-of-experts models aren’t there yet for whole scenes. They mostly did fine on small pieces (an end card, a single animated bar), but on the full scene they either didn’t compile or compiled something unlike the original.
The spec is the ceiling
Then a more honest test. For a different episode I wrote a spec from the narration script alone, the way I’d actually start a new scene, without looking at the existing one. Flash-Next on the Z13 wrote a scene that compiled and rendered on its first try, in about 6 minutes, but the layout had faults. The quality of the result tracked the quality of the spec. So the realistic split is that I, or a frontier model, do the design and write the plan, and the local model does the typing overnight. A local model saves me coding tokens. It doesn’t save me the design work.
A small side test writing a manim animation didn’t show anything the Remotion tests didn’t.
The same models also had a go at unattended code repair, which is closer to agent work.
Caveats
- Single runs. Every result above is one attempt. A different seed could pass or fail differently, especially on the borderline ones.
- Windows only, so far. Linux, with ROCm, is next.
- The scene score compares against one known-good design. It can’t tell whether a different design would also be good, and it doesn’t see motion between the sampled frames. I looked at the frames before trusting a number.
- Model settings differ between rows where noted (MTP on or off, thinking capped or not, and a smaller context for Qwen3-30B-A3B on the 3090).