Local AI lab · Research snapshot 01 · September 2026

How intelligent is local AI?

More than I expected. On one 128 GB mini PC, with no cloud, two open models solved 119.6 of 120 graded problems across six full runs.

17
model setups tested
25
full benchmark runs
20
graded scenarios each
4
coding agents compared

The verdict

Two models. Two jobs.

Not one winner — a team. A careful model plans and reviews; a fast one builds. This is a coding and planning choice; the personal-assistant seat is still open.

Plans · reviews

Gemma 4 31B QAT

20/ 20

in all three runs

  • Never dropped a point
  • 10.6 tokens/s — slow, deliberate
  • 6/6 in every coding agent

Builds · executes

gpt-oss-120b

19.88/ 20

mean of three runs

  • Only miss: part of the parser
  • ≈48 tokens/s — 4.6× faster
  • 6/6 in every coding agent, in minutes

Watching: GLM-5.3-Flash also scored 20/20 — in one run, after two GPU freezes on a different backend. It needs more runs before it earns a seat.

Quality × speed

Only one model is both fast and right.

Every setup on one picture. Up is smarter, right is faster. The corner you want is nearly empty.

Benchmark score versus generation speedgpt-oss-120b is the only configuration that is both above 19.5 out of 20 and faster than 30 tokens per second.Fast and right1012141618200102030405060Generation speed · tokens per second →Score / 20 →Gemma 4 31B QATgpt-oss-120bGLM-5.3-FlashGemma 4 26B-A4B QATMistral Small 4 119B
Dot = mean of full runs. Vertical whisker = spread across runs. Hover or tab through dots for detail.

Leaderboard

All seventeen, ranked.

Open any row for its profile: every scenario, every run, throughput, and whether its misses were wrong or unfinished.

  3. Only ever dropped points on the expression parser — and still passed most of its hidden cases. Near-perfect at 4–5× Gemma’s speed.

    Twenty scenarios · standard | hard

    3 scenario-runs below full marks — all genuine errors.

    Runs (3)
    19.96 · 19.72 · 19.96
    Best
    19.96 / 20
    Generation
    48.5 tok/s
    Prompt
    248 tok/s
    Load
    24 s
    Tested
    09/27, 09/28, 09/29
    Role
    Builder · daily driver

Means of recorded full runs; one score = one run. “Retired” rows are not claims about what is installed today. The two Qwen3.8-Flash-Next rows are separate quants with corrected sampling, not repeats. Qwen3.8-27B’s request settings are still under audit. Merged partial runs are excluded.

Where models break

The hard half separates them.

Every model, every scenario. The standard half is nearly solid; the hard half is where the field thins out — and scenario 13 is a wall.

Pass rate
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Gemma 4 31B QAT
GLM-5.3-Flash
gpt-oss-120b
Muse Glimmer 30B
Qwen3-Coder-Next
Nemotron 3 Super 120B
DeepSeek V4-Flash
KAT-Coder v2.5
Qwen3-Coder-Next Q6
Qwen3.8-Flash-Next (PLE)
Qwen3.8-Flash-Next
Tiel-Coder 35B
Devstral 2 24B
Qwen3.8-27B
Gemma 4 26B-A4B QAT
Gemma 4 12B QAT
Mistral Small 4 119B

Hover any cell to read it. Columns 1–10 are the standard half, 11–20 the hard half.

Solved Partly solved Failed Ran out of output budget
Scenario key
  1. 01Python algorithm (interval merge)95%
  2. 02Python bug fix (date arithmetic)88%
  3. 03Compound-interest math82%
  4. 04Tiered tax calculation100%
  5. 05JavaScript ratio stress test90%
  6. 06SQL: best value per group89%
  7. 07Single tool call100%
  8. 08Bilingual email → JSON100%
  9. 09Long context (~18k tokens)97%
  10. 10Predict program output83%
  11. 11Payment schedule with prepayments67%
  12. 12Two-formula penalty comparison100%
  13. 13Expression parser, no eval()33%
  14. 14JS async retry with backoff94%
  15. 15Exact set cover71%
  16. 16SQL window functions80%
  17. 17Multi-file bug hunt78%
  18. 18Multi-hop long context80%
  19. 19Multi-step tool use100%
  20. 20Logic puzzle64%

Coding agents

The agent changes the outcome.

Six real repository tasks, judged by hidden tests, run through four agent harnesses: Pi, OpenCode, DeepSeek Harness (dsh) and Hermes.

Coding agent scores out of six, with minutes to finish all six tasks
ModelPiOpenCodedshHermes
gpt-oss-120b6/64 min6/66 min6/66 min6/611 min
Gemma 4 31B QAT6/624 min6/634 min6/641 min6/626 min
Nemotron 3 Super 120B6/625 minnot re-run6/626 min6/635 min
Qwen3-Coder-Next Q45.83/64 minnot re-run5.83/66 min5.83/68 min
Gemma 4 12B QAT5.33/633 min · 1 timeout6/627 min6/634 min4.15/627 min · 1 timeout

One setting, opposite conclusion.

OpenCode’s first run blocked edits in headless mode. Every model scored 1.04/6. After the fix, the leaders scored 6/6.

Before fix1.04/6
After fix6/6

Same score, very different wait.

Minutes for gpt-oss-120b to finish all six tasks. Rounded run summaries, not controlled speed trials.

Pi4 min
OpenCode6 min
dsh6 min
Hermes11 min
The six tasks
  1. t1Feature: tax rebate + CLI (Python)
  2. t2Multi-file bug hunt (Python)
  3. t3Implement JS from a written spec
  4. t4Cross-file rename + deprecated alias
  5. t5Messy CSV → JSON script
  6. t6Debug a timezone / loop bug (Node)

Hermes’s coding time says nothing about its quality as an assistant — that is tested next. With the leaders at 6/6, this suite is saturated and now serves as a regression check.

From the workbench

How the team took shape.

  1. 01 · August

    Fit

    128 GB of unified memory made 100B+ models possible at home. Fitting was only the first question; speed and tool behaviour decided whether they were usable.

  2. 02 · September

    Fairness

    The test grew from ten to twenty scenarios. Chat templates, tool parsing, output budgets, sampling and permissions became part of the evaluation.

  3. 03 · September

    Agents

    Six repository tasks through four coding agents. Hidden tests exposed real completions — and showed the leaders have saturated this suite.

  4. 04 · October

    Team

    Planner → builder → reviewer passed a local self-test. Next: everyday assistant work, not just code.

The test bench

Every result, one machine.

A Strix Halo mini PC: CPU and GPU share one 128 GB pool of fast memory, so models far bigger than any consumer graphics card can run at home.

GMKtec EVO-X2 AI · AMD Ryzen AI Max+ 395 (Strix Halo)

128 GB
unified LPDDR5X-8000
256 GB/s
peak memory bandwidth
96 GB
addressable by the GPU
2 TB
PCIe 4.0 NVMe

Processor

Chip
AMD Ryzen AI Max+ 395 (Strix Halo)
CPU
16 Zen 5 cores · 32 threads
Clocks
3.0 GHz base · up to 5.1 GHz boost
Cache
16 MB L2 · 64 MB L3
Process
TSMC 4 nm
Power
Up to 140 W (GMKtec rating)AMD lists a 45–120 W configurable range; GMKtec rates this chassis at up to 140 W.

Graphics and AI

GPU
Radeon 8060S · 40 RDNA 3.5 compute units
GPU clock
Up to 2.9 GHz
Architecture ID
gfx1151
NPU
XDNA 2 · up to 50 TOPS
Total AI
Up to 126 TOPS (CPU + GPU + NPU)The NPU is not used by any result on this site; inference runs on the GPU.

Memory

Capacity
128 GB unified, soldered
Type
LPDDR5X-8000
Bus
256-bit
Bandwidth
256 GB/s theoretical peak8000 MT/s × 256 bits ÷ 8. Token generation on large models is mostly limited by this number.
GPU share
2 GB fixed carve-out + up to 96 GB shared (GTT)96 GB is the Linux default of three quarters of system memory.

Storage and I/O

Storage
2 TB M.2 PCIe 4.0 NVMe
USB
2 × USB4 (40 Gb/s) · 3 × USB 3.2 Gen 2 · 2 × USB 2.0
Display
HDMI 2.1 · DisplayPort 1.4
Network
2.5 GbE · Wi-Fi 7 · Bluetooth 5.4
Power supply
230 W

Software (September 2026 runs)

OS
Ubuntu 24.04.4 LTS, OEM kernel 7.0 · headless
Graphics stack
In-kernel amdgpu · Mesa 25.2.8 (RADV Vulkan)
Inference
llama.cpp b11238 (Vulkan) · ROCm for GLM-5.3-Flash
Also installed
Ollama 0.32
Agent harnesses
Pi 0.87.1 · OpenCode 1.18.33 · dsh 0.2.0-rc.1 · Hermes 0.21.5
Firmware
BIOS 1.12

Method and limits

How to read these numbers.

Honest about what one machine and one month can prove.

The setup

One GMKtec EVO-X2 AI (Ryzen AI Max+ 395, 128 GB) — full specs above. Twenty automatically graded scenarios: code, math, data, tools, context, reasoning. Scenarios 11–20 are the hard half.

The runs

Each score is a full run at temperature 0.2, transcribed from per-run summaries. Finalists got three runs; others got one. Quants and backends differ between rows.

What “unfinished” means

The model was still reasoning when it hit its output-token limit, so no answer was graded. It counts as a miss, but it is reported separately because a larger budget may fix it.

What this does not measure

Everyday assistant reliability, long-term memory, safety, or delegation. No OpenMuse evaluation has run yet. Selection favours correct, completed work over raw speed.

Read the testing protocolDownload the data (JSON)

Planned · no scores yet

What gets tested next.

Screen every viable model, then give finalists harder held-out work. Separate leaders for assistance, coding, planning and specialist jobs — no single crown from a single score.

01

Assistant work

The same synthetic scenarios in Hermes and a locally compatible OpenMuse: research with citations, documents, budgeting, memory and recovery, tool failures, approval boundaries, prompt injection.

02

A wider agent matrix

Retest coding in Pi, OpenCode, dsh, Hermes and Claude Code where a local endpoint works. llama.cpp and Ollama are compared as serving engines, not agents.

03

New candidates

Nex-N2.5-mini, Occamy 1.0, MiMo-V2.6-Distill-Qwen-9B and Ling-3.0-flash-Fin have integrity-checked downloads. Compatibility smoke tests first; none has a score yet.

04

Delegation

One agent versus bounded workers under equal token and tool budgets. More agents is not evidence of more intelligence — only finished, verified work is.

Every new round will date the cohort, model version, quant, backend, prompts, budgets, run counts and failures.