Local AI benchmarks: what runs on which machine, and how fast

What I have found so far

• My Sparks would run a little faster than the figures on this page. I cap their graphics chips at 2,200MHz, against about 2,400 uncapped, to cut their power use. When I tested the cap on one box in August, writing speed stayed within 1 per cent, reading was 2 to 4 per cent slower, and the chip's peak power fell from 55.5W to 43.3W.
• The four DGX Sparks wrote code at 104.1 tokens a second with GLM 5.3 Flash (NVFP4 at 4 bits, dense layers at 8), and read a document of about 16,000 tokens at 3,159 tokens a second. That is a wait of about 5 seconds before the first word (the size of the document divided by the speed).
• With GLM 5.3 Flash thinking before it answers, the first word of the code answer came after about 0.5 seconds on the four Sparks. With thinking off it came after 0.15 seconds.
• With Qwen 3.8 27B on all of them (each machine in the 4-bit format its own software uses), the RTX 4090 desktop wrote code at 62.4 tokens a second, two of my DGX Sparks at 44.7, one DGX Spark on its own at 26.7 and the HP ZBook Ultra G1a (a Strix Halo laptop with the Ryzen AI Max+ 395) at 23.5.
• Two of my DGX Sparks read a document of about 19,000 tokens at 2,034 tokens a second, the RTX 4090 desktop at 1,994, one DGX Spark on its own at 1,188 and the HP ZBook Ultra G1a (a Strix Halo laptop with the Ryzen AI Max+ 395) at 194. That is a wait of about 9 seconds on the two Sparks, 9 seconds on the desktop, 16 seconds on the single Spark and 97 seconds on the laptop before the first word (the size of the document divided by each speed).
• With Qwen 3.8 27B thinking before it answers, the first word of the code answer came after about 7 seconds on the desktop, 14 seconds on the two Sparks (one run), 17 seconds on the single Spark (one run) and 22 seconds on the laptop. With thinking off it came after 1.3 seconds at most.
• On the prose prompt with thinking on, Qwen 3.8 27B was still thinking when the 2,000 tokens ran out, which took 37 seconds on the desktop, 55 seconds on the two Sparks, 86 seconds on the single Spark and 104 seconds on the laptop.
• Given room for 8,000 tokens, the prose answer began after 52 seconds in one run on the two Sparks and after 271 seconds in one run on the single Spark. The thinking before it ran to about 1,800 and 7,400 tokens, so most of that gap is the model thinking for longer.
• A recipe change alone doubled GLM 5.3 Flash on four DGX Sparks. A short answer went from 28.2 to 60.4 tokens a second between 2 and 5 October, on the same boxes with the same script.
• On the same eight prompts in August, three models came within 6 tokens a second of each other (40.9 to 46.3), whether they ran on two Sparks or four.

The machines

The same test on each machine

Each figure is the middle of three runs with the model's thinking switched off, and the range is under it. First word is the wait after a short question. Reading is how fast the machine takes in a long document before it starts to answer, which other sites call prefill or prompt processing. The LM Studio rows were run with LM Studio's defaults, which include the model's own multi-token prediction. My Sparks would run a little faster than the figures on this page. I cap their graphics chips at 2,200MHz, against about 2,400 uncapped, to cut their power use. When I tested the cap on one box in August, writing speed stayed within 1 per cent, reading was 2 to 4 per cent slower, and the chip's peak power fell from 55.5W to 43.3W.

The same test with thinking switched on

Thinking is the model working through the problem before it answers. For GLM 5.3 Flash, thinking on means high, the middle of the model's three thinking levels, on every machine. For Qwen 3.8 27B, thinking on means medium, the middle of the model's three thinking levels, on every machine. The speeds in this table count the tokens it thinks with and the tokens of the answer. The wait is from sending the prompt to the first word of the answer, and it is the middle of three runs, because the model thinks for a different length each time. The cap of 2,000 tokens covers the thinking and the answer together. A wait marked "one run only" was timed in a single separate run with room for 8,000 tokens, because the three runs of that row were made before the script timed the answer. Where a wait begins with "over", the model was still thinking when the 2,000 tokens ran out. The speed beside it is then the speed of its thinking, and the real wait is longer than the one shown.

The models, exactly

• GLM 5.3 Flash on the four DGX Sparks. NVIDIA's own NVFP4 export of GLM 5.3 Flash (4-bit, 204.5GB on disk), with the dense layers re-encoded at 8 bits, which the recipe calls lossless8. A small draft model, DFlash2 converted to FP8, guesses several tokens ahead. It runs on vLLM split across all four boxes (tensor parallel 4) under another owner's published recipe, with a window of 1,048,576 tokens. The model is about 300 billion parameters with about 15 billion active for each token, which is my estimate from its config file. See NVIDIA's weights, the draft model and the recipe.
• Qwen 3.8 27B on two of my DGX Sparks. Unsloth's NVFP4 build of Qwen 3.8 27B (4-bit, 23.4GB on disk), on vLLM split across two boxes through the 200Gb/s switch (tensor parallel 2). The model's own multi-token prediction was on, guessing 3 tokens ahead. The window was 351,232 tokens, the engine kept its working memory of the conversation at 8 bits (FP8), and sampling was the model's own default, a temperature of 1.0. See the weights.
• Qwen 3.8 27B on one DGX Spark on its own. The same weights, engine and settings as the two-box row, on one box by itself (tensor parallel 1). See the weights.
• Qwen 3.8 27B on the RTX 4090 desktop. The Q4_K_M GGUF file of Qwen 3.8 27B from LM Studio's own catalogue (4-bit, 16.8GB), loaded whole onto the graphics card with a window of 20,480 tokens. LM Studio's settings were left as they come, so the model's own multi-token prediction was on. In this run it kept 78 per cent of its guesses. See the file.
• Qwen 3.8 27B on the HP ZBook Ultra G1a (a Strix Halo laptop with the Ryzen AI Max+ 395). The same Q4_K_M GGUF file as the RTX 4090 ran (4-bit, 16.8GB), loaded whole onto the graphics chip with a window of 20,480 tokens. LM Studio's settings were left as they come, so the model's own multi-token prediction was on. In this run it kept 69 per cent of its guesses. See the file.

The hardware, exactly

• The DGX Sparks (the same boxes, used one, two or four at a time). The boxes: four NVIDIA DGX Sparks in two makes, GIGABYTE AI TOP ATOM and ASUS GX10, which behave the same here. Chip: an NVIDIA GB10 in each box, which is a Blackwell graphics processor and a 20-core Arm processor (10 cores at about 3.9GHz and 10 at about 2.8GHz) on one package. Memory: 128GB in each box, shared between the processor and the graphics, with a bandwidth of 273GB a second. Network: two ConnectX-7 ports of 200Gb/s on each box, joined through a MikroTik CRS812 switch. System: NVIDIA DGX OS. Standing setting: graphics chips capped at 2,200MHz to cut power use.
• RTX 4090 desktop. Graphics card: NVIDIA GeForce RTX 4090 with 24GB of its own memory. Processor: AMD Ryzen 9 7950X, 16 cores and 32 threads. Memory: 96GB of DDR5 as two 48GB modules running at 6,000MT/s (Corsair CMK96GX5M2B6000Z30). Motherboard: ASRock X670E Taichi. System: Windows 11 Pro.
• HP ZBook Ultra G1a. Machine: a 14 inch laptop, timed on mains power. Processor: AMD Ryzen AI Max+ 395 (Strix Halo), 16 cores and 32 threads. Graphics: Radeon 8060S, built into the processor. Memory: 64GB of LPDDR5X soldered to the board, running at 8,000MT/s. Windows sees 47.8GB of it, because 16GB is set aside for the graphics. System: Windows 11 Pro.

GLM 5.3 Flash on four DGX Sparks, before and after a recipe change

The same script on both days. The only change was a recipe another owner published. The first word of an answer arrived after 0.26 seconds in a session recorded on 5 October. The change column is my own maths.

Three models, the same eight prompts

Measured in August 2026 with each model thinking at its highest setting, on different prompts from the test at the top of this page. These were also taken before October's recipe change, so they should not be set beside the other tables.

How I test

• One user at a time unless the row says otherwise. A total across several users is always labelled as a total.
• The machine is idle. The script waits for 8 quiet seconds before each run, and a run that had other work arriving part way through is thrown away.
• The same prompts and the same script on every machine, so two rows can be compared.
• Three runs, the middle one reported. The lowest and highest are under each figure.
• Thinking off, and thinking on. Where a model can think before it answers, I run the writing test both ways, and each way has its own table.
• A laptop is tested on mains power, and the row says which Windows power mode it was in.
• Every row has its date. Recipes and software move quickly, and an old figure stays labelled as old.
• Nothing here is a maker's claim. Every figure on this page is one I measured.
• If a figure looks wrong, tell me under the latest video and I will run it again.