Can a small desktop run a large language model without a recurring cloud API bill? Ryzen AI Halo makes that a practical question, but the answer depends on the checkpoint, numerical representation, conversation length and workload. Fitting a model and delivering useful performance are separate acceptance tests.

This original analysis is prompted by AMD’s November 12 technical report on Quark and Qwen3.8-Flash-Next. We also read the reader-supplied November 16 LinkedIn post by Matt Baskovc. Its clip again shows a 128 GB configuration; it is not presented here as a new launch. AMD described the newer PRO 495 Halo on October 12. We combine checked specifications, the published experiments’ limitations and our own illustrative calculations. SIExpo has not benchmarked the hardware.

Establish which generation you are looking at

The official Ryzen AI Max+ 395 configuration has 128 GB of LPDDR5x, 16 CPU cores and 32 threads, integrated Radeon 8060S graphics with 40 compute units, and 256 GB/s memory bandwidth. AMD lists a 2 TB SSD, 10 Gb Ethernet, Linux or Windows 11, and 120 W TDP. TDP is a platform specification, rather than a wall-power measurement for every workload.

The Ryzen AI Max+ PRO 495 configuration has 192 GB, Radeon 8065S graphics, again 16 cores/32 threads and 40 GPU compute units, with 273 GB/s bandwidth. Both systems are listed at 150 × 150 × 45.4 mm. The memory difference matters substantially: a picture of one generation must not be combined with the other generation’s specifications.

The CPU, GPU and the rest of the system draw on a shared memory pool. AMD states that up to 160 GB can be allocated to the PRO 495 GPU. That is not 192 GB of separate VRAM plus another 192 GB of RAM. We do not transfer this allocation limit to the 395: the actual device’s BIOS, driver and exposed allocation must be checked. More capacity does not eliminate compute or bandwidth limits.

Why 200B or 300B is an incomplete specification

AMD advertises capacity for models up to 200 billion parameters on the 128 GB version and 300B on the 192 GB version. Its earlier announcement explicitly associates the large configurations with four-bit quantization. These are capacity guides, not promises of equal accuracy, context length or speed across all models.

Our simplified arithmetic is useful here. Two hundred billion weights at four bits require 100 billion bytes, approximately 93.1 GiB, for ideally packed weights alone. Three hundred billion require 150 GB, about 139.7 GiB. Real formats add scales, metadata and sometimes higher-precision layers. Runtime allocation can differ from file size. Cache, buffers, the operating system and other applications still need space.

Quantization reduces numerical precision. Its effects need testing on the intended task, especially coding and tool use. In a mixture-of-experts model, the parameters activated per token differ from the total stored parameters. A small active fraction does not make every other weight disappear from memory.

The fresh Quark example exposes a genuine trade-off

AMD’s November 12 report describes reducing approximate Qwen checkpoint storage from 130 to 80 GB by replacing a large n-gram table with a compact alternative. The smaller configuration has run on Halo. This changes model information rather than providing free, lossless compression: accuracy effects depend on the task.

The accuracy comparisons in that report ran on Instinct systems. They are not Halo speed or quality benchmarks. The model card describes Linux and a modified vLLM stack. Its configured context ceiling is not evidence of a sustained desktop session at that length. The report and card reference different required changes; a deployable environment needs exact version pinning and its own validation.

Working context consumes memory too

The KV cache preserves intermediate representations of previously processed tokens. For conventional attention with grouped KV heads, a simplified size is two × layers × KV heads × head dimension × bytes × tokens × independent sequences. The following is a calculation example, not a Qwen specification or Halo measurement.

Assume 64 layers, eight KV heads, dimension 128 and two-byte storage. Each token then needs 262,144 bytes of cache. At 32,768 tokens, that is eight GiB for one sequence; at 131,072 tokens, it becomes 32 GiB. Four independent long conversations can radically alter the budget. Architectures with other cache designs, shared prefixes, quantized cache or restricted attention behave differently.

Context should therefore follow the job’s requirements. A short extraction task may not need an enormous history. An analysis spanning many files needs measurements of both input processing and subsequent generation. The llama.cpp documentation separates context size, KV storage types and parallel slots; a parameter-count headline cannot replace those settings.

Speed, concurrency and software

Time to first token, subsequent generation rate and completed-task latency measure different things. A long prompt may produce a slow start even when a short demonstration looks fluid. Multiple users can increase aggregate throughput while making each individual wait longer. CPU and GPU resources are also shared with browsers, indexing, compilation and other work.

NPU TOPS and GPU FP16 TFLOPS are not tokens per second. Not every library uses the NPU, and Windows/Linux availability does not imply identical backends, operators or packages. The archived ROCm 7.2.1 page itself points to newer unified documentation; we do not treat it as the definitive current PRO 495 support matrix. A validated combination needs the exact APU, OS, driver, runtime and checkpoint format.

Our proposed acceptance protocol starts with a small set of real tasks: short and long inputs, verifiable code, structured extraction and the expected number of concurrent requests. Record peak memory, first-token latency, generation speed, final correctness and wall power. This is a suggested evaluation method, not a SIExpo test result.

What stays local, and what still costs money

Local weights and a local runtime can operate without a cloud inference API. An agent can nevertheless use external search, tools, telemetry or a cloud fallback. Downloads and updates also require connectivity. Privacy follows the complete data route, permissions and configuration, rather than the word “local” on a product page.

Costs include hardware, taxes, delivery, electricity, maintenance and setup time. Our hypothetical example is straightforward: 150 W at the wall for eight hours daily equals 36 kWh over 30 days; multiply that by your actual tariff. Continuous operation would be 108 kWh. These are calculations, not measured Halo power consumption.

AMD publishes its own lower three-year-cost scenario for sustained use. It does not establish equal output quality to the cloud model or savings for every buyer. An API may be cheaper for infrequent use; predictable continuous workloads and sensitive data may strengthen the case for local hardware. Compare the cost of a correctly completed task, including retries and human review.

At verification, AMD’s official site specifies purchase and use in the United States. We have not confirmed Bulgarian availability, delivered price or warranty. Halo deserves attention for substantial shared memory in a compact desktop. A buying decision should begin with the exact model and workflow, rather than the largest number on a promotional slide.

Editorial verification: 2026-11-17T07:54:00.882419+00:00.