Technology1 min read

Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot

By · Published by Everything Blog

In short

AssemblyAI deployed a hybrid Amazon Web Services architecture to manage large-scale speech-to-text inference without sacrificing reliability. The design uses Amazon Elastic Container Service capacity providers to distribute workloads across g4dn, g5, and g6 GPU instance families. To safeguard customer service level agreements against compute reclamation, an automated retry mechanism re-queues interrupted tasks within a two-minute warning window. Deployments across regions like US West and Europe further satisfy strict data residency mandates for healthcare and finance clients.

Key points

  • Amazon EC2 Spot Instances absorb compute demand spikes while Amazon EC2 On-Demand instanc…: Amazon EC2 Spot Instances absorb compute demand spikes while Amazon EC2 On-Demand instances maintain a dependable operational baseline.
  • An automated AWS Lambda and Amazon EventBridge workflow safely re-queues in-flight infere…: An automated AWS Lambda and Amazon EventBridge workflow safely re-queues in-flight inference tasks during two-minute Spot reclamation notices.
  • Amazon Simple Queue Service and Amazon CloudWatch work together to coordinate task proces…: Amazon Simple Queue Service and Amazon CloudWatch work together to coordinate task processing and monitor ongoing fleet health.

Sources checked by our analysis pipeline: 1

Artificial Intelligence

Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot

Speech-to-text inference is computationally intensive. Each audio file triggers a pipeline of deep learning models (acoustic models, language models, and post-processing stages) running on NVIDIA GPUs to meet latency requirements. At scale, this means thousands of GPU instances running continuously.

AssemblyAI needed a way to run GPU-intensive inference workloads cost-effectively at scale, without sacrificing the availability and latency guarantees their customers depend on. Running entirely on On-Demand Instances would be cost-prohibitive. Spot Instances offer significant savings, while Amazon Elastic Compute Cloud (Amazon EC2) can reclaim them with a 2-minute warning when capacity is needed elsewhere. AssemblyAI needed an architecture that absorbs Spot interruptions gracefully while maintaining service level agreement (SLA) commitments. Additionally, customers in regulated industries, including healthcare, finance, and legal, must keep audio data within specific geographic boundaries. They implement strict data residency configurations in European AWS Regions.

Available on AWS Marketplace, AssemblyAI provides an API for transcription, speaker detection, sentiment analysis, chapter detection, and personally identifiable information (PII) redaction. Their models transcribe and analyze speech for…

Technology

More in Technology