This review is built from published specifications, current retail pricing and third-party benchmarks rather than hands-on use. GPU pricing moved sharply through 2026, so verify before buying.
On the RX 9070 XT, Vulkan currently beats ROCm by 29% on local inference. That is not a typo, and it is not because Vulkan is a better compute API.
It happens because vLLM runs FP32 dequantization on every operation on RDNA 4 while llama-server uses Vulkan compute shaders properly. Until RDNA 4 FP8 kernel support lands in vLLM mainline, the officially supported AMD stack is the slower one on this card.
That single fact tells you most of what you need to know about running AI on AMD in 2026: the hardware is good, ROCm finally works, and the software is still two steps behind where the marketing says it is.
In this guide:
- What the RX 9070 XT Offers
- ROCm 7.2 Changed the Situation
- Real Benchmark Numbers
- The 16GB Cliff
- Vulkan vs ROCm: The Counterintuitive Result
- Where AMD Still Loses
- RX 9070 XT vs RTX 5070 Ti
- Who Should Buy What
- Frequently Asked Questions About the RX 9070 XT
- Verdict
- Sources and Further Reading
What the RX 9070 XT Offers
The RX 9070 XT is AMD’s RDNA 4 mid-range card, launched early 2026, with the specifications that matter for local AI:
- 16GB GDDR6
- 645 GB/s memory bandwidth
- 220W board power
- Roughly $550 to $599
- Architecture target
gfx1201for anyone compiling backends
The bandwidth figure is the interesting one. At 645 GB/s it sits well above what integrated solutions manage, and since local inference is bandwidth-bound once a model is resident, that number predicts throughput better than compute specs do.
For context on why that matters more than almost anything else on a spec sheet, our how much VRAM you need for local AI guide covers the sizing arithmetic, and the capacity-versus-bandwidth trade appears again in our Framework Desktop review, where AMD’s integrated platform makes the opposite choice.
ROCm 7.2 Changed the Situation
Any AMD-for-AI recommendation written before March 2026 is out of date, and this is why.
ROCm 7.2, released March 2026, is the first version with official RDNA 4 support. It ships as one installer covering both Windows and Linux, with out-of-the-box parity across Ollama, LM Studio, llama.cpp and vLLM. The current line is 7.2.x, with 7.2.4 the latest patch as of mid-2026.
Before that, running an RX 9070 XT meant WSL workarounds, manual library substitution and HSA_OVERRIDE_GFX_VERSION incantations. Ollama 0.16.1 shipped ROCm 6.1 and did not support the RX 9000 series at all, leaving owners copying gfx1201 libraries into place by hand.
Those workarounds are gone. gfx1201 is officially listed as supported. That is a genuine change rather than a marketing claim, and it is the single reason this card is now worth considering.
The broader point: the “AMD is painful for AI” objection has a shelf life, and it expired in March 2026 for this hardware generation. Whether it stays expired depends on AMD sustaining the effort.
Real Benchmark Numbers
Third-party RX 9070 XT measurements through Ollama on ROCm, gathered August 2026:
| Model | Throughput |
|---|---|
| gpt-oss:20b | ~92 tok/s |
| Qwen3.5 9B | ~58 tok/s |
| Qwen3 14B | ~52 tok/s |
| Llama 3.1 8B Q4 | ~92 tok/s |
| Dense 27B (spilled to RAM) | ~6.3 tok/s |
A published 43-question benchmark suite run on a 9070 XT in a Ryzen 7 9800X3D system, using the official ollama/ollama:rocm image at v0.18.2, produced the first four rows. For comparison, 92 tok/s on 8B models puts this card ahead of an RTX 4070 Ti.
The last row is the one that decides your purchase, and it gets its own section.
The 16GB Cliff
On the RX 9070 XT, sixteen gigabytes is not a soft constraint. It is a wall, and hitting it is dramatic.
A dense 27B model at Q4 produces a file around 17GB. That is one gigabyte over what the card holds. The moment part of the model lives in system RAM instead of VRAM, measured throughput fell to 6.3 tok/s, roughly 14.6 times slower than gpt-oss:20b on the same card.
Not 20% slower. Not half speed. A fifteenth.
The practical lane for an RX 9070 XT is therefore 9B to 20B models at Q4 to Q6. Inside that lane it is genuinely fast. One step outside it and the card becomes unusable for interactive work.
This is identical in shape to the constraint on NVIDIA’s 16GB cards, covered in our RTX 5070 Ti review. Capacity is binary and it comes first; speed only matters inside whatever ceiling you bought.
If you need 27B and above, the options are a 24GB card such as the RX 7900 XTX, an RTX 5090 at 32GB, or a high-capacity unified memory machine.
Vulkan vs ROCm: The Counterintuitive Result
Published March 2026 benchmarking compared two backends on identical conditions: same model, same prompt, same max tokens, temperature zero, fixed seed, ten measured runs with two warmup runs discarded.
llama-server on Vulkan came out 29% faster than vLLM on ROCm.
The explanation is not that Vulkan is superior. It is that vLLM currently performs FP32 dequantization on every operation on RDNA 4, while llama-server uses Vulkan compute shaders properly. The gap is a missing optimisation, not an architectural truth, and it closes when RDNA 4 FP8 kernel support reaches vLLM mainline. Community work on that is already underway.
Until then the practical guidance is blunt: use llama-server with Vulkan for RX 9070 XT inference, even though ROCm is the officially supported path. Our LLM inference servers compared guide covers the wider backend landscape, and this is a good example of why testing beats assuming.
One further wrinkle worth knowing: a July 2026 llama.cpp issue documents a 3 to 8% mixture-of-experts prompt-processing regression on 64-CU RDNA 4 cards following a June submission-heuristic change. Vulkan prefill on gpt-oss-20b Q4_K_M was measured at roughly 3,450 to 3,650 tokens per second despite it.
Where AMD Still Loses
The honest RX 9070 XT accounting, because ROCm working is not the same as ROCm winning.
Roughly 10 to 15% lower throughput than CUDA equivalents at comparable memory bandwidth, attributed to less mature driver optimisation. Not catastrophic, but real.
Fine-tuning remains an NVIDIA job. Published guidance is consistent on this: for training and fine-tuning workloads, stick with CUDA. LoRA and QLoRA tooling assumes NVIDIA, and the community troubleshooting you will need is overwhelmingly CUDA-shaped. Our fine-tuning vs RAG guide covers whether you need to fine-tune at all before this becomes a constraint.
Framework edge cases persist. A vLLM context-length issue on RDNA 4 is one documented example. You will occasionally patch something a CUDA user would not.
New models arrive on CUDA first. Being second in line is the structural cost of the smaller ecosystem, and no amount of driver maturity fixes it.
The summary from one detailed comparison is fair: if you are already on AMD, ROCm is finally a viable path. If you are building from scratch and want maximum flexibility, NVIDIA remains the safe bet.
RX 9070 XT vs RTX 5070 Ti
The direct comparison this site can now make, since both cards hold 16GB.
| RX 9070 XT | RTX 5070 Ti | |
|---|---|---|
| VRAM | 16GB GDDR6 | 16GB GDDR7 |
| Bandwidth | 645 GB/s | Higher (256-bit GDDR7) |
| Price | ~$550 to $599 | Higher, region-dependent |
| Software stack | ROCm 7.2 / Vulkan | CUDA |
| Fine-tuning | Poor | Good |
| Model ceiling | ~20B at Q4 | ~20B at Q4 |
| Board power | 220W | Higher |
They hit the same wall at the same place. Neither runs a dense 27B. What differs is price, software maturity and whether you ever intend to train anything.
The AMD advantage is more VRAM per dollar in the $500 to $700 bracket, plus lower power draw. The NVIDIA advantage is that everything works the first time.
What About Image and Video Generation?
Local AI is not only language models, and diffusion work has a different shape on this card.
Diffusion models are more compute-bound than LLM inference, which shifts the balance. SDXL and comparable image models fit comfortably in 16GB and run acceptably. Video generation is the harder case, because it pushes memory pressure far higher and is where the 16GB ceiling reappears.
The software picture improved here too. ComfyUI shipped official ROCm support on Windows during 2026, which removes the Linux-only constraint that pushed many AMD owners away from diffusion work entirely. Our ComfyUI 2026 guide covers that change alongside the environment isolation features that make experimenting with custom nodes less risky.
The same caveat applies as everywhere else on AMD: it works, and a node that assumes CUDA will still occasionally stop you. Budget some troubleshooting time rather than expecting parity.
Power and Build Considerations
At 220W the RX 9070 XT is undemanding compared with the flagship tier.
A quality 650W to 750W supply handles it alongside a typical CPU without drama, and the card fits most mid-tower cases without the clearance problems that triple-slot flagship designs create. That matters more than it sounds, because a GPU requiring a new power supply and case adds a few hundred dollars that never appears in the comparison.
For anyone considering two cards later, the modest power and thermal profile makes that considerably less painful than pairing flagships, though the software complications in our dual-GPU build guide apply regardless of vendor, and multi-GPU support on ROCm is another area where the ecosystem lags CUDA.
Who Should Buy What
| If you are | Start with | Why |
|---|---|---|
| Running 9B to 20B models, price-sensitive | RX 9070 XT | Best VRAM per dollar in the mid-range |
| On Linux and comfortable troubleshooting | RX 9070 XT | ROCm 7.2 removed the old workarounds |
| Fine-tuning or training anything | RTX 5070 Ti or above | CUDA tooling has no AMD equivalent |
| Needing 27B and above | 24GB or 32GB card | 16GB spills and throughput collapses |
| Wanting it to just work | NVIDIA | Fewer edge cases, faster new-model support |
| Already running AMD | RX 9070 XT | The 2026 software situation genuinely changed |
| Running the largest models cheaply | Framework Desktop | Capacity over bandwidth, same vendor |
Frequently Asked Questions About the RX 9070 XT
Does the RX 9070 XT work with Ollama?
Yes, natively, since ROCm 7.2 in March 2026. Older Ollama builds shipping ROCm 6.1 required manual library workarounds; those are no longer necessary.
What is the largest model it runs well?
Around 20B at Q4, with gpt-oss:20b measured at roughly 92 tok/s. A dense 27B at Q4 is about 17GB and does not fit, dropping throughput to roughly 6.3 tok/s once it spills to system RAM.
Should I use ROCm or Vulkan?
Vulkan, currently. Published benchmarking found llama-server on Vulkan 29% faster than vLLM on ROCm, because vLLM does FP32 dequantization on RDNA 4. Expect this to reverse when FP8 kernel support lands upstream.
Can I fine-tune on an RX 9070 XT?
Technically yes, practically no. Published guidance recommends NVIDIA for fine-tuning, and the LoRA and QLoRA ecosystem assumes CUDA. Use this card for inference.
Does ROCm work on Windows now?
Yes. ROCm 7.2 ships one installer covering Windows and Linux with official RDNA 4 support, ending the WSL-only era for these cards.
How does it compare to an RX 7900 XTX?
The older 7900 XTX has 24GB against 16GB, which clears the 27B wall the 9070 XT cannot. It runs Llama 3.1 8B at roughly 96 tok/s, described as about 75% of an RTX 4090. If capacity matters more than architecture, the older card may suit you better.
Setting It Up Properly
A short RX 9070 XT setup path that avoids the common mistakes.
Install ROCm 7.2 or later. Earlier versions do not officially support RDNA 4 and you will end up hand-copying libraries. One installer now covers both operating systems.
Verify the GPU is seen. rocm-smi --showproductname should list the card before you try anything else.
Build llama.cpp for the right target. If compiling rather than using a prebuilt binary, gfx1201 is the RX 9070 XT architecture. Getting this wrong produces a build that runs on CPU without obvious complaint.
Benchmark both backends on your model. Given the Vulkan result above, do not assume the official path is faster. Run the same prompt through both and measure.
Stay under 20B. The single most useful discipline with this card. Check a model’s quantised file size against 16GB before downloading it, not after.
Verdict
The RX 9070 XT is a genuinely good local inference card now, and that sentence would have been wrong six months ago. ROCm 7.2 in March 2026 brought official RDNA 4 support with one installer across Windows and Linux, ending years of workarounds. At roughly $550 with 16GB and 645 GB/s, it delivers around 92 tok/s on 20B-class models and more VRAM per dollar than NVIDIA offers in the same bracket.
The hard limit is 16GB, and it is unforgiving. A dense 27B model at Q4 overshoots by a single gigabyte and throughput collapses to a fifteenth of normal. There is no gentle degradation to plan around. Add roughly 10 to 15% lower throughput than CUDA at comparable bandwidth, a fine-tuning story that still says “use NVIDIA,” and the occasional framework patch, and the picture is a strong inference card with a narrow lane and a rougher road.
Concrete next step: before buying, list the three models you actually intend to run and check their quantised file sizes against 16GB with two gigabytes of headroom for context. If all three fit under about 14GB, this is the best value card in the mid-range. If any of them do not, no 16GB card solves your problem and you should be shopping at 24GB or above.
Sources and Further Reading
- LocalAIMaster RX 9070 XT guide — the cited Ollama benchmark figures and the 27B spill measurement
- ROCm local LLM setup reference — ROCm 7.2 release details, RDNA 4 support and installation across both platforms
- Vulkan vs ROCm benchmark on RDNA 4, the controlled comparison behind the 29% result
- FitMyLLM RX 9070 XT page, specifications, bandwidth and model fit data
- TechReviewer LLM suitability analysis, quantisation guidance and the ROCm maturity caveat
For related coverage on this site, see the RTX 5070 Ti review for the NVIDIA card at the same capacity, best GPUs for local AI for the full ranking, how much VRAM you need for the sizing arithmetic behind the 16GB cliff, and the Framework Desktop review for AMD’s capacity-first alternative.
