The benchmark tested systems against hundreds of data tables, dashboards, notebooks, and calculation rules modeled after a fictional UK energy supplier named Calderwick Energy. Genie achieved a 91% pass rate on business questions, substantially higher than the 45% and 58% marks posted by competing frontier agents. Genie Ontology drove this success by automatically extracting institutional knowledge and reporting exceptions to streamline context discovery. Consequently, the system completed tasks faster while requiring an average of only 431,829 tokens per run.
Key points
Genie Ontology acts as an enterprise context layer that automatically organizes instituti…: Genie Ontology acts as an enterprise context layer that automatically organizes institutional knowledge across diverse assets.
Standard evaluations strictly test pipeline code, but this benchmark required extracting…: Standard evaluations strictly test pipeline code, but this benchmark required extracting context from notebooks, dashboards, and calculation rules.
Establishing a shared understanding of business definitions allows agents to pinpoint rel…: Establishing a shared understanding of business definitions allows agents to pinpoint relevant data efficiently and reduce computational costs.
Sources checked by our analysis pipeline: 1
An enterprise benchmark developed with Turing compares Genie using Genie Ontology with two leading frontier agents.
by Jeff Fletcher and Rishabh Singh
We developed a new enterprise data-agent benchmark in collaboration with Turing to compare Genie with two major frontier agents and their respective harnesses on business questions in a live Databricks workspace. The evaluation measures answer accuracy, token usage and time to complete the task. Genie had a 91% pass rate, compared with 45% and 58% for the other frontier agents, while using fewer than half the tokens.
Unlike previous data agent benchmarks that typically only include tables, this benchmark also includes notebooks, dashboards, saved SQL queries, documents, and other analytical assets that capture enterprise knowledge, business definitions, and calculation rules. These assets provide key enterprise context that table schemas alone cannot establish. Genie Ontology is the enterprise context layer that gives every AI agent a shared understanding of the business. It automatically extracts and organizes this knowledge, surfacing relevant context for Genie to use when answering questions.
Our previous evaluation of Genie with ontology used a proprietary dataset.…
The cross-border startup is targeting the growing enterprise ecosystem by designing specialized analytics infrastructure tailored to autonomous digital software. This substantial launch capital allows the team to scale development across its dual headquarters in Israel and California. The move signals heightened investor appetite for back-end data architectures built expressly to serve next-generation artificial intelligence workflows.
Healthcare technology developers are increasingly pushing to ingest extensive patient records and administrative files to power broader clinical recommendations. This rapid transition from narrow, single-purpose functions to wide-ranging medical decision support challenges traditional clinical perspectives on data limitation. The shift marks an aggressive effort by digital health platforms to expand their operational footprints across patient care.
To accelerate therapeutic development, Anthropic has forged partnerships with pharmaceutical companies and biotechs like Bristol Myers Squibb and BigHat Biosciences. The initiatives leverage the company's Claude AI models to engineer treatments for rare diseases, food allergies, and various forms of cancer. While laboratory tests show promise in optimizing drug molecules, the technologies remain strictly in preclinical development phases.