Artificial Intelligence1 min read

CERT-In seeks checks on access of AI agents

By · Via Times of India · Published by Everything Blog

In short

India's cybersecurity agency has directed auditors to evaluate whether automated systems can perform unauthorized actions or leak sensitive business data. Under the newly released guidelines, AI deployments must incorporate strict boundaries that stop tools from inheriting permissions across different users or systems. Additionally, emergency interruption mechanisms and detailed activity logs must be maintained for any high-risk use cases. Auditors are specifically tasked with ensuring that human oversight mechanisms cannot be circumvented during consequential operations.

Key points

  • Auditors must actively verify that safeguards against data leaks and prompt injection att…: Auditors must actively verify that safeguards against data leaks and prompt injection attacks remain functional.
  • Deployments require mandatory human approval gates that cannot be bypassed during consequ…: Deployments require mandatory human approval gates that cannot be bypassed during consequential operations.
  • Authorized human reviewers must retain the explicit capability to reject, correct, or ove…: Authorized human reviewers must retain the explicit capability to reject, correct, or override AI outputs.

Sources checked by our analysis pipeline: 1

NEW DELHI: India's cybersecurity agency has called for testing whether AI agents can leak confidential data or take unauthorised actions, putting permissions and human oversight at the centre of its audit guidelines.

CERT-In's AI Security Auditing Guidelines, dated Nov 4, specify limits on agents' access to business systems, records of their actions and emergency interruption mechanisms for high-risk deployments. Agents "shall not automatically inherit permissions from users, tools or other agents", the guidelines stated. The safeguards address prompt injection, where malicious instructions hidden in documents or other inputs attempt to hijack an AI system's behaviour.

"Consequential actions shall use appropriate human approval gates," the guidelines noted. Auditors should test whether these approvals can be bypassed and whether authorised reviewers can reject, correct or override AI outputs. The framework also calls for testing safeguards against data leaks.

Artificial Intelligence