Artificial Intelligence1 min read

How ArisGlobal scaled generative AI inference on Amazon Bedrock

By · Published by Everything Blog

In short

ArisGlobal uses Anthropic's Claude on Amazon Bedrock to extract around 130 fields from complex medical reports. As tenant-specific rules expanded the shared prompt prefix from 72,000 to 180,000 tokens over six months, per-case inference costs rose by more than 200%. To resolve this issue, the company shifted from caching source documents to caching the shared prompt prefix. This technical adjustment lowered effective input costs, allowing NavaX to onboard additional enterprise clients without proportional cost increases.

Key points

  • Expanding regulatory rules and client onboarding caused per-case inference costs to escal…: Expanding regulatory rules and client onboarding caused per-case inference costs to escalate by over 200%.
  • Caching shared prompt prefixes instead of source documents mitigated the financial impact…: Caching shared prompt prefixes instead of source documents mitigated the financial impact of expanding tenant instructions.
  • Optimizing prompt caching enabled ArisGlobal to scale its multi-tenant platform without s…: Optimizing prompt caching enabled ArisGlobal to scale its multi-tenant platform without sustaining proportional cost spikes.

Sources checked by our analysis pipeline: 1

Artificial Intelligence

How ArisGlobal scaled generative AI inference on Amazon Bedrock

ArisGlobal, a provider of life sciences technology spanning pharmacovigilance, regulatory, medical affairs, clinical, and quality functions, developed NavaX, an AI engine purpose-built for life sciences on Amazon Bedrock, to automate adverse-event case processing. ArisGlobal chose Amazon Bedrock for managed access to foundation models, EU geographic cross-Region inference, and managed prompt caching. As enterprise customers added distinct compliance requirements, the shared prompt prefix grew by about 150% over 6 months, while per-case inference costs increased by more than 200%.

ArisGlobal first cached each source document because it was the largest part of a request. That left the growing shared prompt prefix outside the cache. By reversing the order and caching the shared prefix instead, ArisGlobal reduced effective input cost and supported additional tenants without proportional cost growth.

Who is ArisGlobal?

ArisGlobal is the creator of LifeSphere, a unified SaaS platform for pharmacovigilance, regulatory affairs, quality, and medical information. LifeSphere serves pharmaceutical companies globally, including multiple Top 20 pharma organizations by revenue (source: ArisGlobal press release). The entire system runs on…

Artificial Intelligence