Artificial Intelligence1 min read

How Standard Bank Group automates AI risk assessment for generative AI

By · Published by Everything Blog

In short

The automated framework operates completely within the bank's AWS account and relies on a four-step pipeline to evaluate generative tools. An automated agent on AWS Fargate initiates the process by scanning repositories and documentation to address approximately 40 risk questions. An LLM-as-a-judge model hosted on Amazon Bedrock then evaluates the evidence against internal bank controls. Standardized evidence reports and dashboards are subsequently delivered to the risk committee, which retains final approval authority.

Key points

  • Final approval authority remains firmly with the bank's risk committee rather than being…: Final approval authority remains firmly with the bank's risk committee rather than being delegated to automated tools.
  • Operating the entire pipeline within the bank's cloud environment guarantees internal sec…: Operating the entire pipeline within the bank's cloud environment guarantees internal security and compliance controls are preserved.
  • Adopting an LLM-as-a-judge approach allows the institution to evaluate project findings a…: Adopting an LLM-as-a-judge approach allows the institution to evaluate project findings against stringent internal controls quickly.

Sources checked by our analysis pipeline: 1

Artificial Intelligence

How Standard Bank Group automates AI risk assessment for generative AI

This post is co-written with Tala Qraitem from AWS and Jörg Fischer from Standard Bank Group

Having an AI policy is one thing. Proving, one use case at a time, that AI systems follow it is another. Every new generative AI application adds one more system to prove. Teams can now build one in days, but each still needs a risk committee’s approval before it reaches customers or employees. When that approval depends on reviewers reading documents and testing the system by hand, governance becomes the slowest step in the pipeline.

Standard Bank Group chose to automate that proof. Working with AWS Professional Services, the bank deployed an automated assessment that scores each generative AI use case against international AI standards and its own policies. The assessment produces reports and a dashboard that show what is already in place, what is working well, and what still needs mitigation. This post explains how the assessment runs on AWS and what Standard Bank Group learned from its first runs.

Artificial Intelligence