An increasing number of organizations are rolling out artificial intelligence or large-language model (AI/LLM) chatbot experiences across their attack surface.

Such rapid adoption comes with rapid risk. These include vulnerabilities defined in the OWASP AI/LLM Top 10 such as:

Prompt Injection

Insecure Output Handling

Training Data Poisoning

Beyond traditional cybersecurity vulnerabilities, chatbots can also deliver undesirable results. Think about:

Reputational risk

Discrimination and bias

Employee trust and confidence

Synack’s pentesting has evolved to test deployed LLMs, using the skills of the Synack Red Team (SRT). The SRT is a community of over 1,500 global, vetted researchers with a diversity of expertise.

When you test with Synack, you receive findings in real-time through our Penetration Testing as a Service (PTaaS) platform, analytics and reporting capabilities and diverse perspectives from an elite researcher team.

Testing for AI/LLM Cybersecurity Risks and Vulnerabilities

Synack’s AI/LLM Pentesting Methodology

The Open Web Application Security Project (OWASP) compiled 10 common and critical vulnerabilities that span potential abuses of an LLM.

Synack tests eight of the OWASP LLM Top 10, described below:

Prompt Injection: Prompt Injection describes a scenario where a particular input to the LLM produces an undesirable output. This can range from inappropriate responses from a chatbot to sensitive data exposure from a search bot.

Insecure Output Handling: If an LLM’s output interacts with a plugin susceptible to common vulnerabilities like cross-site scripting or remote code execution, the LLM may be leveraged by an attacker as a tool to exploit the flaw.

Training Data Poisoning: If an LLM learns from user feedback and input, an attacker may purposefully poison the model by providing false or harmful input.

Supply Chain: An implementation of an LLM may involve calls to libraries or services that are vulnerable, for example, an outdated Python library.

Sensitive Information Disclosure: LLMs may leak sensitive information in a response or mistreat sensitive information that is inputted into the model.

Insecure Plugin Design: LLM plugins are called by models during interaction. If an attacker knows of a vulnerable plugin being called, they may craft specific input to exploit known vulnerabilities in that plugin.

Excessive Agency: An LLM has unnecessary permissions in an environment. For example, an LLM may need to read documents but may erroneously have write/edit permissions to the same documents.

Model Theft: An individual model may be trained on proprietary information, making the model itself unique IP. A copy of the model should not exist, however, attackers may be able to abuse the model in such a way that they are able to make a functional copy.

Results for testing of each of these vulnerabilities will be delivered in real-time through the Synack Platform and can be easily exported for sharing with relevant stakeholders:

Synack AI/LLM Content and Bias Audit

Synack’s AI Content and Bias Assessment goes beyond cybersecurity vulnerabilities to assess generative AI applications for content violations and evidence of bias that can affect these domains.

Content auditing checks for:

Content accuracy, i.e. can the AI be made to state falsehoods

Content violations, i.e. suggestions of violence or overly sexual content

Relevance, i.e. off-topic information or commentary in unrelated domains

Privacy/secrecy violations, i.e. disclosing classified information

Bias is checked across a spectrum of categories such as:

Gender/Sex

Politics

Race

Age

Religion

Methodology:

The SRT has performed millions of hours of cybersecurity testing and delivered 70K+ exploitable vulnerabilities throughout their tenure. When you initiate an AI/LLM content and bias audit, they probe the target to see if the AI/LLM exhibits bias or gives concerning responses.

These results will be made available in the Synack Platform in real-time, just like vulnerabilities.