Artificial Intelligence1 min read

Genie One Accuracy: Why Curated Agents Need Governed Context

By · Published by Everything Blog

In short

In an evaluation of 125 insurance questions, combining curated agents with a governed layer in Genie One lifted overall response accuracy to between 93 and 95 percent. The study found that raw data tables produced correct answers just 64 percent of the time because departments often calculate identical metrics using different operational baselines. A modeled governance framework built on Unity Catalog Metric Views successfully resolved these conflicting definitions while boosting accuracy on a retail test case from 54 percent to 82 percent.

Key points

  • Curated departmental agents excel at document retrieval but require a governed layer to r…: Curated departmental agents excel at document retrieval but require a governed layer to resolve divergent metric definitions.
  • The Genie Ontology merges automatically learned insights from existing dashboards and que…: The Genie Ontology merges automatically learned insights from existing dashboards and queries with authoritative, modeled business rules.
  • Centralized governance guarantees baseline operational accuracy because Genie One does no…: Centralized governance guarantees baseline operational accuracy because Genie One does not automatically inherit an individual agent's specific instructions without direct routing.

Sources checked by our analysis pipeline: 1

What 125 insurance questions taught us about routing, business definitions, and documents

by Richa Sethi, Patrick Chan, Marcela Granados and Amit Kumar Jha

Ask Finance and Underwriting whether the loss ratio improved from 2024 to 2025, and you will get two contradictory answers that are both correct. In a fictional commercial auto insurer we built for this test, Finance reported an 8.2 percentage-point deterioration, and Underwriting reported a 1.5 percentage-point improvement. Neither was wrong. Finance tracks the calendar year net of reinsurance, so an adverse reserve development on an older claim shows up in today's numbers. Underwriting looks strictly at the policy year, gross of reinsurance and excluding catastrophes, because that's the baseline used to set rates. Reserving, meanwhile, relies on a third metric entirely: the accident-year ultimate loss ratio.

The SQL can be perfectly correct and still produce an answer that doesn't make sense for the person asking. The challenge is knowing which business definition to apply when the same term has several valid definitions, while keeping shared facts, such as total policies written, consistent. This is the problem…

Artificial Intelligence