70-80% Accuracy Isn't Good Enough for Enterprise AI

Quick Answer 

• Leading LLMs achieve only 70-80% text-to-SQL accuracy on controlled benchmarks, with accuracy degrading further on complex multi-table enterprise queries.

• Strategy Software's benchmark on a real 28-table insurance schema found 88.2% overall accuracy with direct text-to-SQL, but 0% accuracy on complex multi-table queries — while Strategy Mosaic achieved 100% on both.

• The fix is architectural: grounding natural language queries in a semantic layer encodes correct join paths and business logic once, so SQL is generated from verified contracts rather than inferred from raw schema definitions on every query.

Enterprise data teams have an AI accuracy problem when querying complex enterprise data, and Forrester Research has put a number on it. A June 2026 Forrester report, Make Data AI Ready Via Semantic Layer Platforms, on semantic layer platforms states that benchmarks such as Spider and PICARD show 70-80% execution accuracy under controlled conditions, with accuracy degrading sharply on the multi-table queries that enterprise analytics requires. We ran a parallel benchmark on a real-world insurance schema.

Overall accuracy with direct text-to-SQL: 88.2%. On complex multi-table queries: 0%. Strategy Mosaic scored 100% on both. This post examines what that gap means, why it's structural rather than a prompting problem, and how grounding queries in a semantic layer closes it.

The 70-80% accuracy figure comes from Spider and PICARD, the standard benchmarks for measuring text-to-SQL quality. They test how accurately LLMs convert natural language into executable SQL against documented schemas under controlled conditions. Real enterprise schemas are neither controlled nor simple, and accuracy falls faster than the benchmark suggests.

Multi-fact tables, bridge tables with authoritative join paths, and business logic embedded nowhere in the schema itself create conditions where LLMs don't just underperform; they generate results that look valid and are materially wrong.

We tested insurance claims analytics questions against a 28-table real-world data model, comparing direct PostgreSQL text-to-SQL against the same questions routed through Strategy Mosaic.

The three text-to-SQL failures were structurally predictable:

Neither failure triggered a SQL error. Both results would pass review by an analyst without prior knowledge of the correct answer. The LLM selected the wrong join path because the correct path isn't derivable from column names. It lives in the business logic, which the raw schema doesn't expose. Without a semantic layer to enforce that relationship, the model had to guess, and it guessed wrong without triggering a single SQL error.

Forrester's June 2026 report prescribes a specific architectural fix. Figure 2 is titled "Use A Semantic Layer For Natural Language To SQL Translation To Achieve 100% Accuracy." The recommendation: map each question to governed metrics, dimensions, and policy-aware views, then generate SQL from those contracts.

That's an architectural requirement, not a prompting technique. A semantic layer encodes that reasoning once. Every subsequent query, regardless of user or model, follows the validated path.

The insurance benchmark illustrates why this matters. The fan-out error that inflated Policy 336's coverage limit by 8-10x happens because a 5-year policy generates 5 rows in the coverage detail table. When the LLM joins that table to claims through a shared key without semantic isolation, each claim row matches all 5 coverage year rows and SUM() inflates every dollar by a factor of 5. The query shape looks correct. The number is not.

Strategy Mosaic prevents this by computing coverage and claim metrics independently via canonical paths, then combining pre-aggregated results at the policy level. The multiplicative error cannot form because the semantic model doesn't allow that join to run unconstrained.

How the two approaches compare on the queries where accuracy diverges:

Accuracy isn't the only benefit. Because Strategy Mosaic exposes a semantic API rather than the raw schema, token consumption drops 37% per query. For a 10-person team running 200 queries per day against a 200-table schema, raw text-to-SQL generates over $15,000 per month in schema context overhead before a single query executes against the data.

Most teams have spent the last two years trying to prompt their way out of the accuracy problem: narrowing schema scope, adding guardrails, curating context windows. The problem with that approach is that it treats the symptom. As long as the LLM generates the SQL, answers will be inconsistent across models, teams, and use cases. The same question can produce a different query on every run.

Strategy Mosaic's architecture separates the language problem from the query problem. The LLM handles natural language. Mosaic handles the query. SQL generation happens in Mosaic's deterministic query engine, not the LLM. Every answer derives from pre-defined business logic, not what the model happens to infer from table and column names.

Three protections in the semantic model prevent the failure modes the benchmark uncovered:

Fan-out protection. Fact tables are isolated at the semantic model level. Coverage and claim metrics compute via independent canonical paths. Results combine only as pre-aggregated values at the correct grain. The multiplicative error that inflated the PostgreSQL result by 8-10x cannot form.

Canonical bridge routing. The semantic model encodes one authoritative join path for each entity relationship. Every query, regardless of user or LLM, follows the same validated route. The plausible-but-incorrect shortcut that produced cross-policy misattribution in direct SQL isn't available.

Compressed semantic context. Instead of the full raw schema (4,428 tokens for a 28-table model), Mosaic returns business-friendly definitions and validated relationships (2,073 tokens). The SQL generated from this context is simpler, more precise, and 37% cheaper per query.

When Forrester's June 2026 report covered the Headless BI platforms, it highlighted a critical structural shift: the decoupling of business logic from the traditional visualization layer. But for the modern enterprise, "headless" cannot just mean serving decoupled metrics back to human dashboards.

A true universal semantic layer must operate completely standalone, sitting above any individual BI tool, LLM, or cloud warehouse. That architectural independence is what transforms Strategy Mosaic from a standard data modeling utility into a true control plane for enterprise AI analytics.

The 70–80% benchmark range shows the baseline AI accuracy challenge. Our direct text-to-SQL benchmark shows the deeper enterprise risk: 88.2% overall accuracy still masked 0% accuracy on complex multi-table queries. It's structural. LLMs generate SQL by reasoning about schemas they can't fully understand, and on complex queries, that reasoning produces confident, wrong answers.

Removing the LLM from SQL generation is the fix. Ground natural language questions in governed business context, generate SQL from verified contracts, and the accuracy problem resolves at the architecture level rather than the prompt level.

Our benchmark found 100% accuracy when queries ran through Strategy Mosaic on a real insurance schema. Enterprise data teams now have both third-party validation and first-party evidence pointing to the same conclusion: 88.2% accuracy is not good enough, and the path to 100% runs through the semantic layer.

Information in Forrester publications is based on Forrester’s efforts to compile and analyze the best resources reasonably available to Forrester at any given time. Opinions reflect judgment at the time and are subject to change.  This report is part of a broader collection of Forrester resources, including interactive models, frameworks, tools, data, and access to analyst guidance.