Disclosure: this guide is built from Google’s release materials, public leaderboard positions and third-party benchmark reporting rather than our own long-term testing on every size. Figures are attributed in the text, and memory estimates are rules of thumb, not measurements.
Gemma 4 came out on April 2, 2026, and seven months later it is still one of the most sensible open-weight families to build on. Four sizes, from a model small enough for a phone to a 31B dense model that ranked third on Arena’s open-model leaderboard, all under Apache 2.0.
The honest answer to which one to run: most people should start with the 26B mixture-of-experts model, which activates only about 4B parameters per token and runs fast on modest hardware. The 31B dense model is the one to choose when quality matters more than speed. The two small E-series models are for phones and edge devices, and they are very good at that narrow job.
Table of Contents
- What Gemma 4 Is
- Why the Gemma 4 Licence Change Mattered
- The Four Gemma 4 Sizes
- Gemma 4 Benchmarks
- Hardware: What Each Size Needs
- Gemma 4 vs Muse Glimmer, Qwen and the Rest
- Good Jobs for Gemma 4
- Where Gemma 4 Falls Short
- Getting Gemma 4 Running in an Afternoon
- Fine-Tuning Notes
- Who Should Run Which Size
- FAQ
- Verdict
- Sources and Further Reading

What Gemma 4 Is
Gemma is Google’s family of open-weight models, built from the same research as its closed Gemini models but released for anyone to download, run and fine-tune. Gemma 4 is the fourth generation. Its predecessors were popular: Google says past Gemma generations have been downloaded more than 400 million times, with over 100,000 community variants built on top of them.
The fourth generation brought three changes worth caring about. The licence switched to Apache 2.0. The range now includes a mixture-of-experts model alongside the usual dense ones. And the capability jump was large, especially in reasoning, where the 31B model went from barely functional on competition maths to near the top of the open field.
Context windows run up to 256,000 tokens on the larger models, which is plenty for most local work. The small models are designed to run on phones, a Raspberry Pi or an NVIDIA Jetson board.
Why the Gemma 4 Licence Change Mattered
Earlier Gemma releases shipped under a custom Google licence with usage restrictions. Lawyers at larger companies tended to read it and say no, or say yes with conditions that made the model awkward to deploy. That kept Gemma in the hobbyist and research lane even when its quality justified more.
Apache 2.0 removes that friction. It is a standard, well-understood permissive licence: commercial use, modification and redistribution are allowed, with no revenue thresholds and no acceptable-use policy bolted on. For a company deciding which open model to fine-tune and ship inside a product, that is often the deciding factor.
It also changed the competitive picture. When Meta released Muse Glimmer in August under Apache 2.0, coverage described it as a departure from Meta’s more restrictive past licences. Google had already made that move with Gemma 4 four months earlier. Meanwhile Meta’s larger Muse Spark Open uses a custom licence with attribution requirements, and Mistral’s Large 4 uses a modified MIT licence with a clause about model-as-a-service use. Licence terms are now a real point of differentiation, and Gemma’s are among the cleanest.
The Four Gemma 4 Sizes
| Model | Type | Built for |
|---|---|---|
| Gemma 4 E2B | Small, edge-optimised | Phones, Raspberry Pi, offline apps |
| Gemma 4 E4B | Small, edge-optimised | Higher-quality on-device work, Jetson |
| Gemma 4 26B | Mixture of experts, about 4B active | Fast local assistant on modest GPUs |
| Gemma 4 31B | Dense | Best quality, single-GPU workstation |
The E prefix stands for effective parameters, Google’s term for models designed so that the compute per token matches a much smaller model. In practice, E2B and E4B behave like phone-class models while drawing on more stored knowledge than their compute budget suggests.
The 26B mixture-of-experts model is the interesting middle option. It stores 26 billion parameters but uses only around four billion for each token, so it generates text at the speed of a small model while holding the knowledge of a larger one. The trade-off is memory: all 26 billion parameters still need to sit somewhere, even if most are idle on any given token.
The 31B dense model uses every parameter on every token. It is slower than the MoE but more consistent, and it is the size that earns the leaderboard headlines.
Gemma 4 Benchmarks
On Arena’s open-model leaderboard, the 31B dense model placed third and the 26B MoE sixth around launch. Arena rankings come from human preference votes, so they measure how much people like the answers rather than raw accuracy, but a top-three placing for a 31B model against much larger open rivals is a strong result.
The reasoning jump is the eye-catching number. Gemma 4 31B scores 89.2% on AIME 2026, the competition maths benchmark. Gemma 3 scored 20.8%. That is not an incremental improvement. It reflects a shift toward reasoning-style training that every lab adopted over the past year, and Google applied it well.
On MMLU Pro, a broad knowledge test, the 31B model scores 85.2%. That puts it in the same band as much larger open models from a year ago.
The usual caveats apply. These are Google’s reported figures, AIME is a narrow test, and benchmark scores at this level are partly a measure of how carefully a lab tuned for them. Treat them as evidence that Gemma 4 is competitive, not as proof it will beat a rival on your task.
Hardware: What Each Size Needs
A simple rule of thumb gets you most of the way. At 16-bit precision, a model needs roughly two bytes per parameter. At 8-bit, about one byte. At 4-bit, about half a byte. Then add headroom for the context cache and runtime overhead, which grows with the length of your prompts.
| Model | Weights at 4-bit (rough) | Realistic hardware |
|---|---|---|
| E2B | A few GB at most | Recent phone, Raspberry Pi 5 |
| E4B | A few GB | Laptop, Jetson, phone with headroom |
| 26B MoE | About 13 GB plus cache | 16 to 24 GB GPU or a Mac with 32 GB |
| 31B dense | About 16 GB plus cache | 24 GB GPU or a Mac with 32 GB or more |
These are estimates from parameter counts, not measurements, and exact figures depend on the quantisation method and runtime. The practical takeaway: a single 24 GB consumer GPU runs either of the larger Gemma 4 models at 4-bit with room for a reasonable context. If you push toward the 256K context limit, the cache will eat that headroom quickly.
The MoE model’s advantage shows up in speed rather than memory. Because only about four billion parameters are active per token, it generates much faster than the dense 31B on the same hardware. On a laptop where every token per second matters, that is the difference between usable and frustrating.
Gemma 4 vs Muse Glimmer, Qwen and the Rest
The local-model field got more crowded after April. The most direct competitor arrived on August 10, when Meta released Muse Glimmer, a 30B dense agentic model under Apache 2.0. DataCamp’s write-up benchmarked it against Qwen3.6-27B and Gemma4-31B, and Meta positioned it specifically for always-on local agents: tool calling, failure recovery and reading screenshots.
| Model | Licence | Size | Strength |
|---|---|---|---|
| Gemma 4 31B | Apache 2.0 | 31B dense | Reasoning, general quality |
| Gemma 4 26B | Apache 2.0 | 26B MoE, about 4B active | Speed on modest hardware |
| Muse Glimmer | Apache 2.0 | 30B dense, 131K context | Agentic tool use |
| Qwen3.6-27B | Open weights | 27B | Multilingual, coding |
My read: if you are building a local agent that calls tools all day, Muse Glimmer is designed for exactly that and deserves a test. If you want a general-purpose local assistant for writing, analysis and reasoning, the larger Gemma 4 models remain excellent and have a longer context ceiling. For phones and edge devices, the E-series has few real rivals from the big labs.
Good Jobs for Gemma 4
Private document work
Summarising contracts, analysing internal reports, drafting from confidential notes. Nothing leaves the machine. For regulated industries, that alone justifies a local model.
Offline and edge applications
Field apps, kiosks, devices with patchy connectivity. The E2B and E4B models are built for this, and running on a Raspberry Pi or Jetson opens up embedded uses that cloud models cannot serve.
Fine-tuning for a narrow domain
With Apache 2.0 and a strong base, Gemma 4 is a good starting point for a specialised model: legal clause classification, medical note structuring, support-ticket routing. The 100,000-plus community variants of earlier generations show how much of this already happens.
Cheap batch processing
If you already own a GPU, running thousands of classification or extraction jobs overnight costs only electricity. At volume, that can undercut even the cheapest API models.
Where Gemma 4 Falls Short
The open models are good, but the frontier is still closed. On hard coding and long agentic tasks, the best API models from Anthropic, OpenAI and Google remain clearly ahead of any 31B model. If your work is primarily autonomous coding, a local Gemma will frustrate you.
Context length is the second limit. 256,000 tokens is generous for local work but well short of the million-token windows on current API models, and filling that window locally needs a lot of memory.
The third is that the field moves fast. Gemma 4 is seven months old. Meta, Alibaba, DeepSeek and Mistral have all shipped open models since. It is still a strong choice, but it is no longer the obvious one, and Google has not yet announced a successor.
Getting Gemma 4 Running in an Afternoon
The quickest route for most people is a desktop runner that downloads quantised weights and handles the rest. Pick the 26B MoE at a 4-bit quantisation as your first download; it gives the best balance of speed and quality on typical hardware.
Start with a short context, around 8,000 tokens, and confirm everything works before raising it. Memory use climbs with context length, and running out of memory mid-generation is the most common first-day failure. Once stable, raise the context in steps until you find the ceiling your machine handles comfortably.
Then test on your own material. Paste in a document you know well and ask questions you already know the answers to. Small models can sound confident while being wrong, so verifying on familiar ground tells you how far to trust it on unfamiliar ground.
If you plan to serve the model to other applications, move from a desktop runner to an inference server that exposes an OpenAI-compatible API. That lets you point existing tools at your local model with a one-line configuration change, and swap in a newer model later the same way.
Keep notes on settings that work. Quantisation level, context length, temperature and the runtime version all affect results, and when something changes in an update you will want to know what the old configuration was.
Fine-Tuning Notes for a Narrow Domain
Fine-tuning is where a permissively licensed open model really pays off, and it is easier than it used to be. You do not need to retrain the whole model. Adapter methods such as LoRA train a small set of extra weights on top of a frozen base, which fits on a single consumer GPU for models in this size range and finishes in hours rather than days.
The hard part is not the training. It is the data. A few hundred carefully checked examples of exactly the task you care about beat tens of thousands of scraped ones. If you are teaching a model to classify support tickets, collect real tickets, label them consistently, and include the awkward edge cases your team argues about. Those are the examples that move accuracy.
Hold back a test set before you start, and never train on it. The most common fine-tuning failure is a model that looks brilliant on examples it has memorised and mediocre on anything new. A held-out set of a hundred real cases is the only honest measure of whether the tuning helped.
Start from the smaller model that might work, not the largest. A tuned small model is cheaper to run, faster to iterate on, and often matches a larger untuned one on a narrow task. If it falls short, move up a size with the same data and compare.
Finally, version everything: the base model, the dataset, the training settings and the resulting adapter. When a newer base model arrives, you will want to retrain on the same data and compare like for like, and that is only possible if you kept the recipe.
Who Should Run Which Size
| If you are | Start with | Why | Link |
|---|---|---|---|
| Building a phone or offline app | E2B or E4B | Designed for on-device use | Browse models |
| Running a fast local assistant | The 26B MoE | About 4B active parameters, quick on modest GPUs | Get weights |
| Wanting the best local quality | The 31B dense model | Ranked third among open models | Get weights |
| Building a local tool-using agent | Muse Glimmer | Tuned for agentic workflows | Read the specs |
Related on this site: for more context, see Muse Glimmer, Open-Weight Models in November 2026 and Qwen4-Max.
Frequently Asked Questions About Gemma 4
When was Gemma 4 released?
Google released Gemma 4 on April 2, 2026, in four sizes: E2B, E4B, a 26B mixture-of-experts model and a 31B dense model.
What licence does Gemma 4 use?
Apache 2.0, a standard permissive licence that allows commercial use, modification and redistribution. Earlier Gemma generations used a more restrictive custom Google licence.
Can I run Gemma 4 on a laptop?
Yes. The E-series models run on almost anything, and the 26B MoE and 31B dense models run at 4-bit on a machine with a 24 GB GPU or about 32 GB of unified memory.
How good is Gemma 4 31B?
It placed third on Arena’s open-model leaderboard and scores 89.2% on AIME 2026 and 85.2% on MMLU Pro, according to Google. It still trails the best closed API models on hard coding and agentic work.
What is the difference between the 26B and 31B models?
The 26B is a mixture of experts that uses about 4B parameters per token, so it is faster. The 31B is dense and uses all its parameters on every token, so it is slower but more consistent.
How long a context does it support?
Up to 256,000 tokens on the larger models. Long contexts use a lot of memory locally, so practical limits depend on your hardware.
Verdict
If you want a capable local model with a licence your legal team will sign off, start with Gemma 4. The 26B MoE is the best first download for most hardware, and the 31B dense model is worth the slower speed when quality matters.
The caveat is that seven months is a long time in this field. Newer open models from Meta, DeepSeek and Mistral beat it in specific areas, especially agentic tool use, and the best closed models remain well ahead on hard coding. Benchmark figures here are Google’s own.
Next step: download the 26B MoE at 4-bit, run it against ten documents you know well, and only then decide whether you need the 31B or a specialised agent model like Muse Glimmer.
Sources and Further Reading
- DataCamp: Muse Glimmer, for the head-to-head context against Gemma4-31B.
- Hugging Face blog, for how open models are distributed and run locally.
- VKTR: Muse Glimmer licensing, for the wider shift to Apache 2.0.
- The New Stack: Muse Spark Open, for the contrasting custom licence.
- AI TLDR: Mistral Large 4, for another licence model in the open field.
On this site: compare options in our best local LLMs guide, serve models with LLM inference servers compared, install tools easily with Pinokio, and ground answers with retrieval-augmented generation.

