In this article
- How Much Is Actually Being Spent on AI
- Are More Companies Using Open Weight Models?
- Why Open Models Win on Price
- How Close Open Weights Get to Frontier Models
- What Self-Hosting Open Models Really Costs
- Harnesses Matter More Than Most Model Debates
- What We Would Do First, and What We Would Skip
- Conclusion
- FAQ
- Supporting Links
- About Baytech
- About the Author
Depending on which survey you read, open-weight models handle either 19 percent or 34 percent of enterprise AI tokens. Both figures are published, and both can be right.
Menlo Ventures puts open-source models at 19 percent of token usage, up from 11 percent, while their share of spending fell from 6 percent to 4 percent. An Enterprise Technology Research survey puts the open-weight token share at 34 percent. The gap comes from who was asked, what was counted, and whether you measure tokens or dollars. Sorting that out is the first step toward a sound decision on which models to use.

How Much Is Actually Being Spent on AI
Three big numbers circulate, and they measure different things. Mixing them gives you a headline, not an answer.
The Stanford AI Index, released on April 13, 2026, counts corporate AI investment using data from the analytics firm Quid. That means private equity rounds, mergers and acquisitions, and minority stakes in privately held AI companies raising above $1.5 million. By that count the total reached $581 billion in 2025, more than double the $253 billion of 2024. These are historical actuals, not IT budget forecasts.
Gartner counts something else. In a September 2026 release, Gartner projected that worldwide AI spending will total $2.67 trillion in 2026, up 49.5 percent from about $1.79 trillion in 2025. That is a forecast, and it was revised up from Gartner's January 2026 projection of $2.52 trillion. Gartner says the upward revision came mainly from hyperscalers and technology service providers buying AI-optimized servers, semiconductors, networking equipment and cloud capacity. IDC uses a narrower scope and forecasts AI infrastructure spending alone at $487 billion in 2026.
Most of the Money Goes to Hardware, Not Models
Gartner's breakdown shows where the dollars land. Infrastructure is about 56 percent of the 2026 total. Spending on generative AI models themselves is about 1 percent.
| Segment | 2025 | 2026 | 2027 forecast |
|---|---|---|---|
| AI infrastructure | $981.9B | $1,484.4B | $1,977.7B |
| AI services | $434.0B | $576.5B | $745.7B |
| AI software | $288.2B | $461.6B | $656.4B |
| Generative AI models | $13.0B | $28.3B | $51.6B |
Source for the table is Gartner's September 2026 forecast, rounded to the nearest $0.1 billion.
Model spending is up 117 percent year over year, and it is still only about 1 percent of total AI spending. Gartner analyst John-David Lovelock said in September 2026 that the buildout of AI data center capacity is the largest infrastructure project humanity has ever undertaken.
For software, Gartner says vendors are embedding agentic AI in their existing products to defend against cross-functional agents. It also says generative AI entered what it calls the "Trough of Disillusionment" in 2026, so enterprises are leaning on simpler AI features from incumbent software providers rather than starting moonshot projects.
What Companies Spend at the Application Layer
Menlo Ventures published its enterprise report on November 3, 2026. It says enterprise generative AI deployment spending reached $177 billion by September 2026, a 5.4x jump over the $33 billion spent in 2025. Organizations spent on average 4.7 times their 2025 levels, against budgets that had planned for 1.9 times, which works out to roughly 2.5 times what they budgeted.

The money is mostly new. Menlo says 55 percent comes from net-new budgets and only 22 percent was redirected from existing software budgets.
The return side is much quieter. McKinsey surveyed 1,719 people from May 4 to June 8, 2026. Nearly nine in ten report regular AI use in at least one function, and 44 percent say AI is scaling across their enterprise, up from 38 percent in 2025. Yet only 37 percent attribute any EBIT impact to it, the same share as in McKinsey's 2025 survey, and only about 6 percent qualify as high performers under McKinsey's criteria. MIT NANDA research published in 2025 states that 95 percent of generative AI pilots fail to reach measurable profit and loss impact. Gartner predicts that 40 percent of agentic AI projects will be canceled by 2027 due to governance failures.
Our read is that companies are spending fast and measuring slowly. That makes unit cost a practical lever. If you cannot yet prove what an AI workflow earns, you can at least control what it costs, and that is where open-weight models come in.
Are More Companies Using Open Weight Models?
Yes in the Menlo Ventures and ETR token data, and the Ramp data that seems to disagree counts paying businesses, which is a different measure. The honest answer is that usage is rising while dollars stay with the frontier labs.
| Source | What it measures | What it says about open models |
|---|---|---|
| Menlo Ventures, Nov 3, 2026 | Enterprise deployment dollars and tokens | 19% of tokens (up from 11%), but 4% of spend (down from 6%) |
| ETR survey, Sept 2026 | 200 respondents, token usage and production status | 34% of tokens (up from 23% a year earlier), 42% running in production |
| Ramp AI Index, Sept 2026 | US corporate card and bill-pay data | Only 6.4% of AI-spending businesses pay for open-source models |
The Frontier Labs Are Still Taking Most of the Dollars
Menlo reports that Anthropic, OpenAI and Google together command 69 percent of enterprise deployment dollars, up from 50 percent in 2025. Inside the enterprise LLM API market, Anthropic grew from 40 to 55 percent of spend, OpenAI from 27 to 31 percent, and Google fell from 21 to 9 percent. Anthropic and OpenAI together hold 86 percent.
Frontier models also grew their token share from 18 percent to 42 percent between Q3 2025 and Q3 2026, which Menlo says outpaced open-source growth. So in Menlo's data both camps gained token share, and the frontier labs gained more.
The Survey That Points the Other Way
The ETR survey of 200 respondents published in September 2026 paints a different picture. It found 42 percent of respondents running open-weight models in production in September 2026, up from 31 percent in July 2026, with another 43 percent running pilots. Respondents expected the open-weight token share to reach 41 percent within twelve months. Cost savings was the main reason, cited by 69 percent of production users.
Two hundred respondents is a small sample, and the production figure moved 11 points in two months, so we would treat it as direction, not a precise measurement. Still, the direction agrees with Menlo on token share going up.
Cheaper Closed Models Muddy the Picture
Ramp's data adds a twist. Its card data shows frontier models falling from a 53 percent peak share of tokens in August 2026 to 45 percent. But that shift did not go mainly to open models, since only 6.4 percent of AI-spending businesses in the dataset pay for open-source models. Part of the explanation may be that closed models also got cheaper, which the next section covers.
Put together, the three sources do not contradict each other as much as they first appear to. Open-weight tokens are growing. They are also so cheap that a rising token share still produces a falling share of the money. Anyone who quotes only one of these numbers is giving you half the story.
Why Open Models Win on Price
The price gap is large and it is the main reason companies look at open models. Menlo says open-source token prices fell 3.3x over the past year to an average of $0.13 per million tokens. The blended price of closed-model tokens fell 24 percent to $0.70 per million.
List prices vary widely across the market. Here is a sample, per million tokens, taken from several pricing roundups.
| Model | Input | Output | Notes |
|---|---|---|---|
| GPT-5 Preview (OpenAI) | $50.00 | $200.00 | Highest-tier proprietary |
| GPT-4.1 (OpenAI) | $2.00 | $8.00 | Default production model |
| GPT-5.6 Luna (OpenAI) | $0.20 | $1.20 | Price floor for mainstream proprietary APIs as of Aug 20, 2026 |
| Llama 4 Maverick (Meta) | $0.17 | $0.60 to $0.97 | Managed platforms; price varies by host, and the $0.17 input price is from Deepinfra, the best provider listed |
| DeepSeek V4 Flash | $0.14 | $2.19 | Open weight |
Judged on a blended 3:1 input-to-output basis, Llama 4 Maverick comes out roughly 12.4x cheaper per token than the GPT-5 API. Note that OpenAI's own cheapest tier sits near the open-weight price range, so "open versus closed" is not always "cheap versus expensive."
What a Real Workload Costs
One pricing site ran a simulation on processing 10,000 documents. This is a simulated cost, not a measured one. On OpenAI's o3 reasoning model it came to $2,550. On Claude Sonnet 4.6, assuming an 80 percent prompt cache hit rate, it came to $201. On the open-weight DeepSeek Coder V2 it came to $22.40.
The jump from $2,550 to $22.40 deserves a caution. These models are not equal in ability, and the simulation does not say they would produce equally good results on your documents. It shows what the price tiers do to a high-volume job, and the answer is a lot.
Named Companies Moving Work to Open Models
Futuriom's August 2026 analysis of over 200 enterprise case studies points to organizations prioritizing the customization of open models with proprietary internal data over generic off-the-shelf commercial solutions. Several named cases show what that looks like.

- Bridgewater AIA Labs. In June 2026 the firm reported adapting the open-weight Qwen3-235B model for six financial document tasks. Using an expert-verification training process, the adapted model reached 84.7 percent average accuracy against 78.2 percent for the highest-performing frontier model tested. The firm reported inference cost per task fell by a factor of 13.8.
- LinkedIn. The company deployed a smaller adapted open model for the repetitive matching tasks behind its Hiring Assistant feature. That model handles nearly 90 percent of the feature's total model calls.
- Tinder. A Financial Times report on September 27, 2026 quoted Tinder CTO Vinay Kuruvila saying the company began routing queries from non-technical staff to open-weight models to control API costs. Those costs had grown from an annualized rate of $1 million in January 2026 to $10 million by July 2026.
- PNC Financial Services, CH Robinson and Siemens. According to the Financial Times, all three have talked about using open-weight models in recent weeks, sending routine, high-volume queries to models that are good enough for the job.
The pattern in these cases is routing. Routine, high-volume work goes to a cheaper open model, and the frontier API is kept for hard reasoning. None of these companies is described as dropping frontier models entirely.
How Close Open Weights Get to Frontier Models
Close on many tasks, behind on the hardest ones, and near the top on a few agent benchmarks. Which of those you see depends on the test. A separate look at Claude Opus 5.5 and agent swarms shows how orchestration changes those results in production.

A Mozilla State of Open Source AI report says open-model capability doubles every 3.9 months, compared with 5.5 months for closed models. It says open models score about 3 points lower than closed models on major benchmarks at roughly 60 percent of the price. That is Mozilla's own trend summary, so read it as a trend claim and not a guarantee for your workload.
The Standard Head-to-Head
The comparison the sources lean on most is Meta's open-weight Llama 4 Maverick (released April 2025) against OpenAI's proprietary GPT-5 (released August 2025). Both are 2025 releases, so newer open models are not covered here. On the shared benchmarks, GPT-5 beats Llama 4 Maverick across five core metrics: GPQA, MATH, MMLU, MMMU and MMMU-Pro.
| Benchmark | GPT-5 | Llama 4 Maverick |
|---|---|---|
| MMMU (multimodal) | 84.2% | 73.4% |
| HumanEval (coding) | Not given | 86.4% |
A gap of a few points on MMLU, a general-knowledge test, is described as meaningful in academic research but rarely decisive for customer-facing applications that handle standard knowledge tasks. Llama 4 Maverick's 86.4 percent on HumanEval makes it competitive for standard coding.
The hardest tests show a wider gap. On SWE-bench, which simulates resolving real GitHub issues across large codebases, the proprietary GPT-5.3-Codex leads with 74.9 percent, which keeps the frontier advantage for complex software engineering agents.
GPQA Diamond, a doctoral-level science test, shows how results depend on which models are compared. In one independent evaluation Llama 4 Maverick scored 69.8 percent, ahead of the 53.6 percent the source gives for GPT-4o. The same source puts GPT-5 at 88.4 percent and GPT-5.2 at 93.2 percent against Maverick's 69.8 percent, which it takes to mean proprietary frontier models still hold a commanding lead in doctoral-level multi-step scientific reasoning. Because the answer changes so much with the comparison, we would not make a purchase decision from a single number.
Context windows go the other way. Llama 4 Maverick has a 1,000,000-token context window. Meta's Llama 4 Scout, a specialist model, goes to 10,000,000 tokens. GPT-5 is documented with a 400,000-token input limit.
Agent Benchmarks Tell a Different Story
Static tests like MMLU do not show whether a model can work inside business systems, so 2026 brought benchmarks built around agent workflows. The results are mixed, and that is useful information.
- Zapier's AutomationBench tests agents on end-to-end workflows using 47 real tools across six business functions. Gemini 4 Argon (Medium) leads Sales at 58.12 percent and Claude Sonnet 5.5 leads Finance at 50.00 percent.
- The Artificial Analysis and BenchLM version covers 600 public automation tasks. The closed MiniMax M3.1 Flash leads at 62.2 percent, followed by the open-weight DeepSeek V4.1 Flash at 60.5 percent, and the proprietary Gemini 4 Carbon at 53.9 percent.
- AutomationBench-AA scores the share of objectives an agent completes without breaking business guardrails, on a private held-out task set. xAI's Grok 4.5 leads at 0.514, followed by Meta's Muse Wonder Full at 0.454.
An open-weight model places second on the BenchLM board and none leads the others. Automation Anywhere's GBA-Bench, a proprietary suite it built, makes a point worth carrying forward. It measures "trajectory accuracy," meaning whether an agent followed a correct, auditable reasoning path instead of a "scrappy win," a correct outcome reached through fragile or inefficient reasoning that public benchmarks often miss.
Our conclusion from all this is that the gap is small for routine work, still real for the hardest reasoning and code tasks, and too inconsistent across agent tests to rank models from a leaderboard. You need to test on your own tasks.
What Self-Hosting Open Models Really Costs
Open weight does not mean free. The weights cost nothing, and the machines to run them can cost as much as a house.
Memory decides what you can run. As a rule of thumb, one trillion parameters at 4-bit precision needs about 500 GB for the weights alone, before the KV cache. Flagship open models now regularly exceed a trillion parameters.
| Tier | Example models | Memory needed | Hardware cost |
|---|---|---|---|
| Single GPU | gpt-oss-20b (21B parameters, 3.6B active) | 16 to 24 GB | High-end workstation or laptop |
| Single H100 | gpt-oss-120b, Nemotron 3 Super (quantized) | 80 GB | $15,000 to $25,000 per GPU |
| 8x H100 | DeepSeek-V4.1-Flash (552B parameters) | 640 GB; 276 GB at INT4 | No figure given |
| 16x H100 | Kimi K2.6 (1T), DeepSeek V4 Pro (1.6T total, 49B active) | 1,280 GB; V4 Pro needs 800 GB at INT4 | $500,000 to $800,000 |
| 24x H100 | Kimi K3 (2.8T parameters) | 1,920 GB; 1,400 GB INT4 working set | $750,000 to $1.2 million |
The single H100 tier is the common enterprise starting point. A standard 70B model at full 16-bit precision needs 140 GB of VRAM, which already means more than one GPU. Onyx's guide, which also sells an enterprise application layer, lays out the hardware tiers and their costs in detail.
When Buying Hardware Pays Off
Activepieces' self-hosting guide puts the break-even for a standard H100 hardware configuration against SaaS alternatives at 19,070 hours of operation, just over two years of continuous use. That assumes the cluster stays busy. If your volume is spiky or small, the clock runs much longer.
That number is measured against SaaS alternatives. Managed hosting of open models, such as Llama 4 Maverick through a host like Deepinfra at the price above, is a different comparison and needs no capital outlay.
Software and Licensing
For high-throughput deployments of large open models, Onyx recommends splitting the stack. vLLM and SGLang serve as inference runtimes that maximize token throughput (64.1 tokens per second on DeepSeek-V4-Pro in its example), while Onyx Enterprise supplies the application layer with governed interfaces, role-based access and connectors to internal data. On consumer hardware, Ollama is the easiest setup and LM Studio offers a GUI.
Licenses matter more than people expect. As of October 29, 2026, the term "open weight" covers a fragmented set of licenses. OpenAI's gpt-oss uses Apache 2.0 plus OpenAI's usage policy, and Google's Gemma 4 uses the permissive Apache 2.0 license. Meta's Llama and Muse releases use custom community license agreements, and Mistral Large 4 is noted as having terms that can stall enterprise procurement reviews. Have legal read the license before engineers fall in love with a model.
Harnesses Matter More Than Most Model Debates
A harness is the software wrapped around a language model that makes it work as an agent. That covers prompt and context construction, long-term memory, tool integrations (web search, database queries, writing to a CRM), orchestration logic that decides when to call the model again, and output guardrails.

Florian Buetow, an AI engineer at Xebia, said on the Beyond Coding Podcast in September 2026:
"the harness matters more than the model... the harness is what makes or breaks your results, even with the same frontier model underneath"
The numbers behind that claim come from coding-agent tests. On Terminal-Bench 2.0 in May 2026, the Letta Code harness scored 59.1 percent using Claude Opus 4.5, while Anthropic's own Claude Code harness scored 41.6 percent with the same model, a 17.5-point difference. Winder.ai's harness comparison also reports that LangChain raised its coding agent's success rate from 52.8 to 66.5 percent on Terminal Bench by changing only the harness, and that the legal AI firm Harvey doubled the accuracy of its agents through harness improvements alone.
Which Harnesses Open Weight Models Use
Winder.ai and Heeya do not pair each open-weight model with one harness. The harnesses below are model-agnostic frameworks that work with open or closed models, and they are the ones those two sources describe for open-weight deployments.
- OpenHands. An MIT-licensed, general-purpose open-source harness managed by All-Hands-AI. You self-host agents inside a Docker sandbox that protects the host filesystem, and it talks to any model through the Agent Client Protocol. Its main drawback is that the sandbox does not protect you from token billing overruns if an agent gets stuck in an ambiguous infinite loop.
- OpenCode. A TypeScript harness built to get open-weight models running code on hardware you own. It is described as highly optimized for models such as Qwen 3.8 on automated back-office tasks.
- Pydantic AI Harness. An open-source framework that composes typed Python capabilities inside a Pydantic AI agent. It works as a library, not a standalone application.
- LangGraph and LlamaIndex Workflows. The leading open-source frameworks for custom, multi-node agentic retrieval-augmented generation pipelines, with explicit message-passing interfaces.
The OpenHands billing warning is easy to skip past. If you move work to self-hosted open models, a runaway loop burns your own GPU time. If you call a paid API, it burns money directly. Either way, put a step cap and a token budget in the harness before the first pilot.
Evaluation Harnesses Are a Separate Thing
Two other tools carry the word "harness" and do a different job. EleutherAI's LM Evaluation Harness is the de facto standard for academic benchmarking and powers the Hugging Face Open LLM Leaderboard. It is an open-source Python framework with over 300 built-in tasks, including MMLU, GSM8K and HellaSwag, and it locks down prompt format, scoring and few-shot examples so models compare fairly. Stanford CRFM's HELM looks at safety, fairness, bias, toxicity and calibration using scenario-based benchmarks.
Those tools measure a model under controlled conditions. The Letta result above shows that the same model can swing by 17.5 points depending on the production harness around it. Our view is that a leaderboard score for an open model tells you little about how your agent will perform.
Agentic RAG Patterns
Much enterprise harness work is retrieval. Standard retrieval-augmented generation pulls documents by similarity search and pastes them into a prompt. Agentic RAG uses agents that plan retrieval, check data quality and run across several systems. One implementation guide says 85 percent of GenAI pilots still fail to scale past proof of concept, and that standard RAG can hand a model the "most similar" text, not the text that is right for a multi-step decision, producing fluent answers that lack the surrounding business logic.
The same guide says five harness patterns cover 95 percent of use cases. Each costs latency, which is worth planning for.
| Pattern | Good for | Added latency or cost |
|---|---|---|
| Router agent | Sending a query to the right knowledge domain, such as HR or legal | 200 to 400 ms |
| ReAct | Troubleshooting and research | 1 to 3 seconds per reasoning loop |
| Plan-and-Execute | Predictable, structured data aggregation | 2 to 5 seconds |
| Self-RAG | High-stakes answers that need evidence checks | 30 to 60 percent more cost and latency per call |
| Multi-agent retrieval | Cross-department reasoning across separate knowledge silos | 5 to 15 seconds |
What We Would Do First, and What We Would Skip
The evidence leaves a fairly clear order of operations. Here is ours.
Start With a Test Set From Your Own Work
Before comparing any models, collect 50 to 200 real questions paired with expected answers and the exact source documents they should map to. Everything else depends on this. Public benchmarks shift with the models compared. On GPQA Diamond, Llama 4 Maverick's 69.8 percent beat GPT-4o's 53.6 percent but trailed GPT-5 at 88.4 percent, so public numbers will not settle your decision.
Also check whether the rules governing the workflow are written down. If policies exist only as tribal knowledge, the workflow is not ready for agentic automation. Clean the data too. Remove superseded documents, because a model cannot know a 2023 policy was replaced if both versions sit in the vector store. Dynatrace research says data quality issues affect 99 percent of AI and ML projects.
Try Open Models on Managed Hosting Before Buying Hardware
Run your test set against an open-weight model on a managed platform first. Llama 4 Maverick is available through managed hosts such as Deepinfra at $0.17 per million input tokens. That tells you whether quality is acceptable for a given task without a $500,000 to $800,000 outlay for a 16x H100 cluster.
We would hold off on buying GPUs until you have steady, high token volume. The 19,070-hour break-even is just over two years of continuous use. Only a workload that stays busy that long makes the hardware pay.
Route by Task Difficulty
Send routine, high-volume queries to an open model and keep a frontier API for complex reasoning, which is the pattern in the Tinder, LinkedIn and Bridgewater cases. SWE-bench and doctoral-level science reasoning are where the frontier lead is clearest. Classification, matching, extraction and standard document tasks are where open models have the best case on price.
Put Engineering Effort Into the Harness
The Letta and Claude Code gap is larger than most model-to-model gaps on the standard benchmarks above. Choose a model-agnostic harness so you can swap models as prices and quality change. Add token caps, because OpenHands' sandbox will not stop a billing overrun. Apply access controls at the retrieval stage so the model never sees data the user is not allowed to see.
If you are weighing buying versus building, MIT NANDA research from 2025 says purchased AI solutions succeed roughly 67 percent of the time, against 22 percent for internal builds. That argues for building only the layers that are specific to your business.
What We Would Not Bother With
- Chasing small benchmark gains. Gaps of a few points on MMLU rarely decide customer-facing outcomes.
- Deciding from one usage statistic. Menlo, ETR and Ramp disagree because they measure different things.
- Treating "open weight" as one license. Check Apache 2.0 and custom community terms individually.
Conclusion
Open-weight models are gaining token share and costing far less per token, but the frontier labs still collect most of the enterprise dollars. In Menlo's data they hold 69 percent of deployment spend. The two facts fit together because open tokens are priced so low.
The capability gap is narrow on routine work and wide on the hardest code and science reasoning. For most companies, the first real move is a test set built from their own tasks, a trial of an open model on managed hosting, and a harness they can reuse with any model. Hardware purchases come last.
FAQ
Are more companies using open weight models now?
Token share is rising in both datasets that measure it. Menlo Ventures puts open-source models at 19 percent of enterprise token usage, up from 11 percent. An ETR survey of 200 respondents reports 34 percent, up from 23 percent a year earlier, with 42 percent of respondents running open-weight models in production in September 2026. Dollar share is a different story. Menlo has open-source at 4 percent of spend, down from 6 percent, and Ramp's card data shows only 6.4 percent of AI-spending businesses pay for open-source models.
How much is being spent on AI?
It depends on the yardstick. Gartner projects $2.67 trillion of worldwide AI spending in 2026, up 49.5 percent from about $1.79 trillion in 2025, with roughly 56 percent going to infrastructure. The Stanford AI Index counts $581 billion of corporate AI investment in 2025, which is investment in AI companies and not IT spending. Menlo Ventures reports $177 billion of enterprise generative AI deployment spending by September 2026.
How close are open weight models to frontier models?
A Mozilla report says open models score about 3 points lower than closed models on major benchmarks at roughly 60 percent of the price. In inference.net's Llama 4 Maverick comparison, GPT-4o leads by 5 points on MMLU and GPT-5 leads by more than 10 points on MMMU. The frontier lead is clearest on SWE-bench, where GPT-5.3-Codex scores 74.9 percent. On the BenchLM automation board, the open-weight DeepSeek V4.1 Flash scores 60.5 percent, just behind the closed MiniMax M3.1 Flash at 62.2 percent.
Do I need to self-host to use open weight models?
No. Llama 4 Maverick is sold through managed hosts. Deepinfra, the best provider listed on llm-stats.com, charges $0.17 per million input tokens. Self-hosting a frontier mixture-of-experts model over 750B parameters needs a 16x H100 cluster costing $500,000 to $800,000. Activepieces' guide puts break-even against SaaS alternatives at 19,070 hours, just over two years of continuous use.
Which harnesses do open weight models use?
Winder.ai and Heeya describe model-agnostic harnesses, not a fixed pairing. OpenHands is MIT-licensed, runs agents in a Docker sandbox and connects to any model through the Agent Client Protocol. OpenCode is a TypeScript harness built to run open-weight models such as Qwen 3.8 on your own hardware. LangGraph and LlamaIndex Workflows are the leading frameworks for custom agentic RAG pipelines. For measuring raw model quality, EleutherAI's LM Evaluation Harness is the standard.
Does the harness really matter more than the model?
The best evidence is from coding agents. On Terminal-Bench 2.0 in May 2026, the same Claude Opus 4.5 model scored 59.1 percent in the Letta Code harness and 41.6 percent in Anthropic's Claude Code harness. LangChain also moved from 52.8 to 66.5 percent on Terminal Bench by changing only its harness. These are results on one kind of task, so test with your own workload.
What should I check before picking an open weight model?
Check the license first. Apache 2.0 covers gpt-oss (plus OpenAI's usage policy) and Gemma 4, and Llama and Muse use custom community licenses. Mistral Large 4 is noted as having terms that can stall enterprise procurement reviews. After that, run your own 50 to 200 question test set before committing.
Supporting Links
- Menlo Ventures 2026 state of enterprise AI
- Gartner forecast of worldwide AI spending for 2026
- US companies turn to open models as AI bills climb
- The enterprise case for open models
- BenchLM AutomationBench results
- Best self-hosted LLMs in 2026 and their hardware tiers
- AI agent harness comparison
About Baytech
At Baytech Consulting, we specialize in guiding businesses through this process, helping you build scalable, efficient, and high-performing software that evolves with your needs. Our MVP first approach helps our clients minimize upfront costs and maximize ROI. Ready to take the next step in your software development journey? Contact us today to learn how we can help you achieve your goals with a phased development approach.
About the Author
Bryan Reynolds is an accomplished technology executive with more than 25 years of experience leading innovation in the software industry. As the CEO and founder of Baytech Consulting, he has built a reputation for delivering custom software solutions that help businesses streamline operations, enhance customer experiences, and drive growth.
Bryan’s expertise spans custom software development, cloud infrastructure, artificial intelligence, and strategic business consulting, making him a trusted advisor and thought leader across a wide range of industries.




