OpenGuard

OpenGuard is a policy framework that enforces customizable rules to ensure safe interactions with Large Language Models (LLMs). With AgentGuard, OpenGuard provides policy-based monitoring and control of AI agents, enabling compliant behavior and transparent decision-making throughout execution.

In addition, OpenGuard features Hallucination Detection, a capability designed to identify responses whose content deviates from verifiable facts or established ground truth.

The Hallucination Checker combines Beam Search Sampling (BSS) with semantic consistency analysis to systematically detect factual inconsistencies. BSS generates multiple response candidates that capture the model's confidence distribution across different plausible answers. These responses are then grouped based on semantic similarity and analyzed using Natural Language Inference (NLI) to assess entailment and contradiction relationships.

To quantify potential hallucinations, OpenGuard employs a scoring mechanism that combines token probabilities with semantic similarity metrics, providing a more accurate assessment of factual consistency. When Beam Search Sampling produces only a single response, OpenGuard applies a Chain-of-Verification (CoVe) mechanism to enhance self-consistency checks and further improve the reliability of the results.

By combining policy enforcement, agent monitoring, and advanced hallucination detection, OpenGuard provides a structured and trustworthy approach to improving the reliability of LLM-generated content. This makes it a valuable tool for the responsible, secure, and compliant deployment of AI applications and AI agents.