What Is the Best Local AI Model? 13 Models Across 1,080 Real-World Tests

Acuántico Power’s own benchmark · August 2026
1,080 local executions · 20 configurations · 54 tests · Ryzen 7 5800X · RTX 2080 SUPER · LM Studio

I wanted to determine which was the best local AI model to run inside an agent on my own PC. Answering that question required much more than comparing specifications or repeating other people’s rankings.

You encounter names such as Qwen3.6-35B-A3B-Q4_K_M or Qwen3-4B-Thinking-Distilled-Q4_K_M-GGUF and are expected to know which one to download. Is 35B better than 4B? Is Q4 bad? What does K_M mean? Why can a model with 27 billion parameters occupy just over 4 GiB? And why can the model that reasons best perform worse as an assistant?

These questions arose while I was testing models for Agente Jiménez, my local AI agent laboratory. The experiment eventually grew to include 13 models, 20 configurations, 54 tests per configuration, and 1,080 local executions.

The local tests were performed by Adolfo-Assistant inside Agente Jiménez, using the model loaded in LM Studio in each case. I then added another 54 executions by Codex + OpenAI GPT-5.6 Sol, with high reasoning, as an external reference for what a complete frontier agent can achieve. In total, this study contains 1,134 responses.

Results at a glance

Swipe horizontally to view the full table →

Test environment: Ryzen 7 5800X, 32 GiB RAM, GeForce RTX 2080 SUPER, a 32,768-token context, and GPU offload set to the maximum in LM Studio. These are results for the specific variants tested in August 2026, not performance promises for every computer.

The ranking measures which configuration performed best as an agent within this architecture. It does not attempt to decide which model is the most intelligent in the abstract.

Asking which local model is best without saying what you want it for is almost an unanswerable question.

This article is not meant to hand you a champion so that you can close the tab and download the first result. Its purpose is to help you understand what you are comparing, see where each model excels and fails, and choose for yourself.

The short answer: there is no single winner

The best model for writing does not have to be the best for programming. The fastest one may invent facts. The model that produces the most elaborate answers may take too long for everyday conversation. An excellent model can also fail inside an agent because of its chat template, tool format, or multiturn behavior.

The audit of the stored responses places Gemma 4 26B-A4B at the top among local models: 94.4/100 with reasoning enabled, although it recorded one technical failure, and 92.5/100 with reasoning disabled and no failures. Qwen3.5 4B reached 90.7/100 with reasoning enabled. Qwen3.6 35B-A3B scored 89.6/100 with ON and 89.5/100 with OFF, but suffered three failures in both modes. Bonsai 27B scored 84.9/100 with ON and 80.9/100 with OFF, also with three failures per mode.

That is not a contradiction. Size buys potential capacity; it does not guarantee that this capacity will fit your hardware or the way the application uses the model.

The first mistake: choosing by parameter count

In greatly simplified terms, parameters are values learned by the model during training. A 4B model has around 4 billion parameters and a 27B model around 27 billion. That figure helps estimate capacity and requirements, but it is not an intelligence score.

Dense and MoE: when 35B does not mean what it seems to mean either

In a dense model, approximately the entire network participates in generating every token. A Mixture of Experts, or MoE, contains groups of parameters—the experts—and a router activates only some of them to process each token.

In Qwen3.6 35B-A3B, 35B indicates the total parameters, while A3B means that approximately 3B active parameters participate per token. This reduces computation, but the system must still store the model. The tested GGUF occupies 20.55 GiB. Gemma 4 26B-A4B, with around 4B active parameters, occupies 14.56 GiB.

When reading the name of an MoE model, separate two questions: how many parameters must I store? and how many participate in each token? They are different problems.

How to read a model name without getting lost

Imagine a fictional name: Family-4B-Instruct-Distilled-Q4_K_M.gguf. It contains several decisions that should not be confused with one another.

Swipe horizontally to view the full table →

Base, Instruct, Chat, Reasoning, Coder, and VLM

A base model learned to predict text, but not necessarily to behave as an assistant. Instruct indicates post-training to follow instructions. Chat points to conversation. Reasoning usually devotes more computation or generates intermediate steps. Coder specializes in code. A VLM adds vision and can process images.

These are clues, not certificates. A model called Reasoning can reason well and still leak its <think> tags, break the required format, or choose the wrong tool.

GGUF is not a quantization

GGUF is a file format. It can contain a model in F16, Q8, Q6, Q5, Q4, or much more aggressive quantizations. Think of GGUF as the container and Q4 as one possible way of representing the weights stored inside it.

What quantization means

The original weights often use considerable precision, such as FP16 or BF16. A rough calculation would be: 7 billion parameters × 2 bytes ≈ 14 GB, before adding other overhead. Quantization reduces memory and computation by representing those values with less precision and accepting some error.

Swipe horizontally to view the full table →

The llama.cpp K-quants use their own blocks and scales, and can mix tensor types. That is why Q4_K_M is a recipe, not a quality grade. The benchmark’s extreme case is Bonsai 27B: its Q1_0 variant occupies 4.41 GiB. The number 27B describes the original model; 4.41 GiB describes the specific representation that was executed.

Quantizing weights and quantizing the KV cache are not the same thing

During a conversation, the Transformer retains previously calculated keys and values to avoid repeating all the work. This temporary memory is the KV cache, and it grows with the context. Quantizing it reduces its footprint, but it does not compress what the model learned: it compresses part of what the model remembers during the current inference.

That is why a 3 GiB GGUF does not mean you need exactly 3 GiB of VRAM. You must also account for the KV cache, runtime buffers, context, the visual projector used by some VLMs, and other overhead.

Advertised context versus practical context

Several models advertise hundreds of thousands of tokens, but all local configurations in the test bench were loaded with 32,768 to keep the comparison consistent. LM Studio lets you choose context length and GPU offload; increasing context also increases memory use. A specification of 256K does not guarantee that you can use it comfortably or that the model will make good use of the information when it is full.

Why two variants from the same family can behave differently

Many local variants do not come directly from the creator of the model family. Someone fine-tuned, distilled, merged, or otherwise modified them afterward. That history matters as much as the parameter count.

A fine-tune specializes behavior or domain; LoRA and QLoRA make it possible to do so by training adapters with fewer resources; distillation transfers some of a teacher model’s behavior; and a merge combines related weights or adapters. You do not need to memorize the process to choose a model. You need to know that two downloads from a similar family can follow instructions, reason, and hallucinate very differently.

The model name is a clue. The model card is the evidence.

And “uncensored” does not mean “more intelligent”

The Qwen3.5 4B Uncensored HauhauCS Aggressive variant was the highest-scoring 4B with reasoning enabled: 90.7/100. It did not achieve this through superpowers. Uncensored usually indicates changes to alignment or refusal behavior; it may help some use cases and harm others. In this test bench, its honesty area scored 7.33/10, below its overall average.

Fewer refusals do not mean greater accuracy.

How the tests were conducted

The benchmark addresses one specific question: which variants perform best inside a local agent that must converse, reason, use tools, and avoid casually making up the world.

Swipe horizontally to view the full table →

The 54 tests cover format control, long-form responses, diagnosis, privacy, false premises, radio and SDR, programming, mathematics, multiturn memory, communication, translation, multiple languages, RAG, source evaluation, browsing, and agent behavior.

What the average score measures

Swipe horizontally to view the full table →

How each score was calculated

The scores were reconstructed directly from the persisted results. Each of the 54 tasks receives 0 to 10 points through explicit rules derived from its own criterion: correct mathematical values, required words and fields in output-control tests, data retrieved in RAG, concepts required in technical answers, and rejection of fictional entities in hallucination tests. The overall figure is the simple average of all 54 tasks multiplied by ten. Speed is reported separately and cannot conceal a worse answer or artificially improve the quality score.

• A convincing answer scores zero on a factual trap if it invents nonexistent specifications, authors, or features.
• A technical failure scores zero on that task, even if a partial answer useful for diagnosis was retained.
• ON, OFF, and native mode are retained as separate configurations; they are neither averaged together nor selectively chosen to hide one another.
• Scores by area remain available for interpreting strengths and weaknesses; the overall ranking does not replace that detail.

The prompts, responses, and JSON events are retained in run-20260823-095238. The reproducible audit also generates agent-score-audit.json, agent-score-audit.csv, and the per-test detail file agent-score-test-details.csv. The frontier reference completed the same 54 tasks and was judged with the same content rules, but is shown separately because it does not share hardware or an execution path with the local models.

Artifact limitation: the stored result for test 14.13 allows us to inspect the table, links, dates, and confidence levels that were delivered, but the run does not retain an independent trace of every browser action. The article therefore does not present that final response as proof that browsing occurred; it evaluates only what can be audited in the files.

This average represents the everyday task suite designed for Agente Jiménez, not universal intelligence. If you only write novels, program, or translate, you should consult the corresponding area and apply your own priorities. The average compares the complete test bench; it does not decide for you.

The 13 models head to head

This table shows every configuration. The score is the average of the 54 tasks, and the time is the actual recorded total. It is not a speed promise for another computer: it compares these executions on the same test bench.

Swipe horizontally to view the full table →

Model cards and repositories for the tested variants: Qwen3.5 Uncensored HauhauCS · Gemma 4 26B-A4B QAT Q4_0 · DeepSeek v4 Pro 4B MTP · Qwen3 VL 4B · Ministral 3 3B · Qwen3.8 4B Distilled · Gemma 4 E4B QAT · Tiny Aya Global · Nemotron 3 Nano 4B · Bonsai 27B Q1 · Qwen3.6 35B-A3B · Granite 4 H Tiny · Phi-4 Mini Reasoning.

Important: ok in the logs means that the execution finished and was saved; it does not mean that its content was correct. A model can complete all 54 calls and hallucinate in several answers.

The variants do not always share the same quantization either. This experiment does not isolate an architecture under academic conditions: it compares real packages that a person can load and use. The results therefore describe those specific variants inside Agente Jiménez and LM Studio, not some universal essence of each family.

Is it worth enabling Reasoning mode on a local model?

Seven models allowed reasoning to be toggled in LM Studio and completed all 54 tests in both modes. The impact on time was enormous and did not always produce better integration.

Swipe horizontally to view the full table →

Enabling reasoning is not a universal “make it smarter” switch. It added 11.9 points and eliminated three failures for Qwen3.5; it subtracted 0.2 points for Gemma E4B while multiplying time by 2.31; and it added only 0.1 points for Qwen3.6 while retaining all three failures. The choice must be made by model and task, not by label.

What each model taught us in real-world scenarios

To compare profiles without hiding them behind the average, this matrix takes the mode with the highest overall score for each model and shows particularly relevant measured areas. Zeros caused by failures or unmet requirements are retained.

Swipe horizontally to view the full table →

Gemma 4 26B-A4B: the highest local score, with an operational caveat

With reasoning enabled, it scored 94.4/100, the highest local score, but experienced one technical failure and needed 4,995.9 seconds. With reasoning disabled, it reached 92.5/100 with no failures in 1,715.2 seconds. The 1.9-point difference costs almost three times as much time; for this benchmark, OFF is the strongest practical option, while ON delivers the highest raw quality.

Qwen3.5 4B Uncensored: the best 4B depends entirely on the mode

With reasoning enabled, it scored 90.7/100, recorded no failures, and was the highest-scoring 4B. With reasoning disabled, it fell to 78.8/100 and recorded three failures, although it finished in only 299.1 seconds. This is the clearest example of why the model name alone is not enough: the two configurations differ by 11.9 points and a factor of 5.71 in time.

Qwen3.6 35B-A3B: almost the same score, much more time with ON

It scored 89.6/100 with ON and 89.5/100 with OFF. Both modes suffered three failures and scored zero in the context area because those tasks were not completed. ON took 6,687.4 seconds compared with 2,292.6 for OFF. In this integration, the additional reasoning neither fixed the operational problem nor justified the time cost.

Bonsai 27B Q1: substantial capacity compressed to the limit

Its 4.41 GiB file demonstrates how far Q1 can go. It scored 84.9/100 with ON and 80.9/100 with OFF; both modes recorded three failures and lost the context area. ON took 3,466.5 seconds. It is interesting for studying extreme compression, but these results do not place it among the fastest or most reliable options in the test bench.

Qwen3 VL 4B: the easiest balance to recommend

It occupies 3.10 GiB, completed the suite in 469.6 seconds, scored 87.5/100, and recorded no technical failures. Its context area reached 10/10 and research and RAG 9.04/10; honesty scored 3.33/10, so operational balance should not be confused with factual reliability. It also adds vision.

Gemma 4 E4B and Nemotron 3 Nano: caution before spectacle

Gemma 4 E4B scored 91.7/100 with OFF and 91.5/100 with ON, both without failures. OFF was 2.31 times faster and scored 10/10 for honesty; ON fell to 6.67/10 in that area. Nemotron rose from 78.5/100 with OFF to 86.3/100 with ON; its context area was 0 with OFF and 10 with ON. They remind us that the mode can change specific capabilities, not only speed.

Tiny Aya, Granite, and Ministral: speed can conceal a cost

Tiny Aya was the fastest at 166.8 seconds and scored 74.2/100. Granite took 200.5 seconds and reached 84.0/100; Ministral took 378.6 seconds and reached 86.4/100. All three finished without technical failures, but their honesty scores were 3.33/10, 1.33/10, and 5.33/10 respectively. For limited hardware, Granite offers much more quality here for only 33.7 additional seconds compared with Tiny Aya, although it requires strict factual verification.

They can be useful for classification, text transformation, or low-risk conversation. They require strict external controls for research or factual decisions.

“Thinking” models: reasoning is not enough if the process leaks

Qwen3 4B Thinking Distilled scored 85.1/100 and Phi-4 Mini Reasoning 83.6/100, both without technical failures. Their honesty scores were 5.33/10 and 0/10, and Phi-4 also scored 0 in output control. In a real product, reasoning should improve the final answer, not contaminate it or replace verification.

Gemma 4 26B-A4B and DeepSeek 4B: elaborate answers, different risks

DeepSeek 4B scored 88.9/100 with ON in 1,086.5 seconds and 86.6/100 with OFF in 380.6, with no failures in either mode. Reasoning added 2.3 points and multiplied time by 2.85. Gemma 26B-A4B also showed a large variation in honesty: 10/10 with ON and 5.33/10 with OFF. The higher average does not remove the need to inspect each area.

The frontier reference: Codex + GPT-5.6 Sol

The reference is not part of Agente Jiménez or LM Studio. The 54 tests were executed by Codex with OpenAI GPT-5.6 Sol, using high reasoning and one isolated session per test. It appears as openai_gpt-5.6-sol in the benchmark artifacts. It provides a practical ceiling for the comparison: what a frontier agent with mature orchestration and tools achieves compared with the laboratory’s local agents.

Swipe horizontally to view the full table →

The following chart compares the total time for every configuration. ON, OFF, and native modes are shown separately; the yellow bar identifies the external reference. A shorter bar means faster execution, not necessarily a higher-quality response.

Time is retained and compared because it is part of the real experience. It should not, however, be interpreted as tokens per second from the same engine: GPT-5.6 Sol uses an external route, and several tests include browsing and verification. Its value lies in showing the end-to-end time cost of a complete frontier-agent response.

The reference scored 94.9/100 and completed all 54 executions without technical errors. It followed the exact 50-word constraint and retained the multiturn scenarios. It was not perfect: the audit assigned it 8.06/10 in programming and 9.31/10 in translation. In the browsing test, the final response contains a table, links, dates, and confidence levels, but the run did not retain an independent trace that could certify every browser action.

The reference does not prove that the cloud “wins.” It shows which components a local agent must strengthen: grounding, tools, context, verification, and compatibility.

Which local AI model to choose for your intended use

Swipe horizontally to view the full table →

A five-minute test is worth more than somebody else’s table

1. Choose three tasks that you genuinely repeat.
2. Add a false premise to see whether the model dares to say “I don’t know.”
3. Include a strict format and a two- or three-turn conversation.
4. Measure memory, time to first token, and total duration.
5. Change only one variable at a time: model, quant, context, or reasoning mode.

Frequently asked questions about local AI models

What is the best local AI model?

There is no universal winner. In this test bench, Gemma 4 26B-A4B achieved the highest local average with ON (94.4/100, one failure); its OFF mode was the best local configuration without failures (92.5/100). Qwen3.5 was the best 4B with ON (90.7/100), while Qwen3 VL combined vision, 87.5/100, and a total time of 469.6 seconds.

Do more parameters mean a better model?

Not necessarily. Parameters provide potential capacity, but fine-tuning, quantization, the template, memory, speed, and compatibility with your tasks also matter.

What does Q4_K_M mean?

It is a quantization recipe from the K-quant family used to reduce the memory occupied by the weights. Q4 indicates roughly the four-bit range; K_M describes internal decisions concerning blocks and tensor types. It is not a quality score.

Is GGUF a quantization?

No. GGUF is the format or container. It can store weights in F16, Q8, Q6, Q5, Q4, or other quantizations.

Should Reasoning be enabled?

Only if it improves your specific tasks. In this laboratory, enabling it multiplied total time by between 2.31 and 5.71. It can help with diagnosis or planning and harm short formats, latency, and integration.

How much memory does a local model need?

More than the file size. You must add the weights, KV cache, buffers, context, and, for visual models, the projector. The reliable way to find out is to load the variant with your actual context and observe RAM and VRAM use.

What you should really take away from this comparison

A small model can be fast, private, and surprisingly capable. It can also invent a receiver, a scientific paper, or a language with the same confidence it shows when producing a correct translation. A large model can reason better and still become a worse agent if it is slow, loses context, or does not fit the runtime’s template.

That is why Agente Jiménez separates the model, context, tools, skills, permissions, and verification. As I explain in Inside Local AI: What Really Lies Behind an Artificial Intelligence Agent, the model is only one component of the system.

The best local model is not the one that wins the most tables. It is the smallest, fastest, and most reliable model that solves your tasks well within your hardware and architecture.

The next time you see a string of acronyms beside a download button, you will no longer be looking at noise. You will see total and active parameters, specialization, fine-tuning method, quantization, format, and requirements. And you will be able to decide which trade-offs you accept before handing your time—and your data—to the model.