The proof

Independently verified: 8× cheaper, zero errors, on real enterprise data.

Not our benchmark. BoonAI ran OpenSymbolic against their own production system, on their own document corpus, scored by an independent judge.

01 · The setup

What was tested.

BoonAI evaluated OpenSymbolic against their production multi-turn RAG system on 104 real enterprise queries over their proprietary construction-document corpus: a mix of factual, multi-hop, document-comparison, and vision questions. Answers were scored by an LLM judge; failures scored 0.1 and were never excluded. Same task, same data, head to head.

02 · Headline results

Same or better, at a fraction of the cost.

Cost / query

$0.61→$0.08

8× cheaper

Errors

13.5%→0%

zero failed queries

Tokens (median)

67,262→9,450

86% fewer

Latency (median)

46.5s→29.7s

36% faster

Accuracy (mean)

0.777→0.813

+4.6%

Same or better answers, a fraction of the cost, because the baseline’s 14 errors (silent context overflows) score zero, and OpenSymbolic had none.

03 · Per-category breakdown

Where the gap is widest.

Category	n	Baseline	OpenSymbolic	Tokens saved	Latency
Simple factual	55	0.924	0.898	89% fewer	9% faster
Multi-hop	25	0.584	0.744 (+0.160)	80% fewer	45% faster
Doc comparison	15	0.560	0.655 (+0.095)	92% fewer	47% faster
Vision selection	9	0.783	0.753*	93% fewer	38% faster

OpenSymbolic wins biggest on multi-hop (+0.160) and doc comparison (+0.095), exactly the hard, high-value queries where the baseline’s context overflows caused silent failures.

On simple factual it’s a hair lower (−0.026) at 89% fewer tokens, a trade most teams take gladly.

* The vision dip was an S3 permissions issue during the eval, not a framework limitation.

Head-to-head: 27 OpenSymbolic wins, 29 baseline, 48 ties, and OpenSymbolic dominates the multi-hop queries that matter most.

04 · Why it’s cheaper

The introspection boundary.

The planner sees summaries and highlights, never raw page content. Context grows ~0.5–2K tokens per step instead of the 5–30K a raw-data loop re-sends every iteration. 64% of queries resolve in a single iteration, with zero errors.

How it works: the mechanism in full →

05 · Public benchmarks

And it’s not just one dataset.

MULTIHOP-RAG

82.9%+22.9 pts

Over the next-best approach across 2,556 multi-document queries. 99.6% goal completion.

TRAVELPLANNER · ICML 2024

97.9%4–13× cheaper

Pass rate at 4–13× lower cost than a ReAct loop. 7 of 11 models hit 100%.

FOLIO · FIRST-ORDER LOGIC

89.2%96% human bar

Near the human bar, backed by a theorem prover rather than a guess.

Want these numbers on your data?

We’ll prove it on one of your workflows in two weeks, measured head-to-head against what you run today.

Book a pilot