...
Back to Articles
Blog

AI SaaS Security Checklist: What to Ask Before You Trust an LLM Vendor

AI SaaS Security Checklist: What to Ask Before You Trust an LLM Vendor

Choosing an AI or LLM service is not just a software procurement decision. You may be sending the vendor customer records, source code, internal documents, support tickets, business plans, or other information that your company would never put into an ordinary public tool.

That is why a normal SaaS security review is not enough. You still need the basics such as access control, encryption, secure development, logging, and incident response. But an AI service adds another layer of questions: Which model receives your data? Is your data used for training? Can the AI retrieve private documents? What tools can an AI agent call? Can one customer access another customer’s data? And what happens when the vendor changes its model or subprocessor?

This AI SaaS security checklist is designed to help security, procurement, privacy, and engineering teams ask those questions before approving an AI or LLM-based service.

What makes an AI SaaS security review different?

Traditional SaaS security reviews usually focus on the application and the company operating it. AI systems add more moving parts. Your request may pass through an application, an orchestration layer, a retrieval system, a vector database, a foundation model, and one or more third-party providers.

If the product also has agent capabilities, the AI may be able to read files, call APIs, send messages, update records, or perform other actions. The security question is no longer only “Can someone break into the application?” It is also “What can the AI access and what can it do if something goes wrong?”

1. Start with the data

Before looking at models or certifications, find out exactly what happens to your data. A vendor should be able to explain the data flow in plain language.

  • What customer data does the service collect?
  • Is data stored, and for how long?
  • Are prompts, responses, uploaded files, embeddings, and logs retained?
  • Is customer data used to train or fine-tune models?
  • If training is disabled, is that restriction documented in the contract or data processing terms?
  • Which third-party model providers or subprocessors receive customer data?
  • Where is the data processed and stored?
  • Can customers choose a data residency region where required?
  • What happens to backups when data is deleted?
  • Can the customer obtain confirmation that data has been deleted?

One common mistake is to treat RAG as an alternative to model training. They are different things. A service can use retrieval-augmented generation and still have separate policies for training, logging, retention, and human review. If a vendor uses RAG, ask how documents and embeddings are stored and how access controls are enforced before information is added to the model context.

2. Check tenant isolation

AI SaaS products often serve many customers from the same platform. Your data therefore needs to remain isolated from other tenants at every stage.

  • How are customer accounts and data logically separated?
  • Are prompts and conversation histories isolated between tenants?
  • Are uploaded files and vector embeddings isolated?
  • Are authorization checks applied before retrieving documents for an AI response?
  • Has the vendor tested for cross-tenant access?
  • Can one customer’s content influence another customer’s retrieval results?

This matters especially for RAG systems. If an authorization check happens after retrieval instead of before it, the application may already have exposed information to the model that the user should never have been allowed to retrieve.

3. Understand the model and supply chain

You should know more than the name of the AI product. Ask what actually sits behind it.

  • Which foundation or language models are used?
  • Does the vendor operate its own model or rely on another model provider?
  • Can the vendor change the underlying model without notifying customers?
  • Are prompts or customer data sent to another AI provider?
  • Are models fine-tuned using customer data?
  • How are model updates tested before they reach production?
  • What happens if a model provider has a security incident?
  • Can the customer restrict or approve certain subprocessors?

An AI service can have a much longer supply chain than it appears to have from its product page. Your data may move through several providers, so the vendor’s subprocessor list and data flow diagram are worth reviewing.

4. Test for prompt injection and AI-specific attacks

Prompt injection happens when an attacker places instructions into input that the AI should treat as data, but the model interprets them as instructions. This can become more serious when the application connects the model to private data or external tools.

For example, imagine an AI assistant that reads support tickets and can create refunds. An attacker could place instructions inside a ticket designed to influence the assistant. If the application gives the model too much authority, the problem is no longer just an incorrect answer. The AI may take an unwanted action.

  • Does the vendor test direct prompt injection?
  • Does it test indirect prompt injection through documents, web pages, emails, or other retrieved content?
  • Are model outputs treated as untrusted data?
  • Are sensitive actions validated outside the model?
  • Are tool permissions enforced by the application rather than by the model’s instructions?
  • Does the vendor regularly red-team the AI application?

Do not accept “we have guardrails” as the whole answer. NIST’s current AI security work notes that AI security requires ongoing testing and hardening rather than assuming a fixed set of controls will remain effective forever. NIST’s 2026 research on AI security also highlights the need for continuous red teaming, updates, and operational resilience.

If you want to understand prompt injection in more detail, see our guide on mitigating prompt injection attacks.

5. If it is an AI agent, review what it can do

This is one of the most important checks for modern AI applications. A chatbot that only produces text has a different risk profile from an agent that can take actions.

  • Which tools can the agent access?
  • Can it read email, files, databases, tickets, or source code?
  • Can it send messages or make changes to external systems?
  • Which actions require human approval?
  • Are permissions limited to the minimum needed for each task?
  • Can a prompt or retrieved document cause the agent to call a privileged tool?
  • Are tool calls logged with enough detail to reconstruct what happened?
  • Can access be revoked quickly if the agent or an account is compromised?

A useful rule is simple: the model should not be the security boundary. If an AI says it is allowed to delete a database, the application must still enforce whether the user and the agent actually have permission to perform that action.

For a simple explanation of agents and agentic systems, see AI Agent vs Agentic AI.

6. Review identity and access controls

  • Does the service support single sign-on where appropriate?
  • Is MFA available and enforced for administrators?
  • Does the platform support granular roles and permissions?
  • Can access to sensitive AI features be restricted by role?
  • Are API keys scoped to the minimum permissions required?
  • Can API keys be rotated and revoked?
  • Are privileged actions logged and reviewed?
  • Are service accounts protected separately from normal user accounts?

For APIs, ask how authentication and authorization work rather than simply checking whether the vendor supports OAuth or JWT. A secure protocol does not make an API secure if permissions are poorly designed.

7. Look at the normal application security controls too

AI security does not replace ordinary application security. The AI feature still runs on software, infrastructure, databases, APIs, identity systems, and cloud services.

  • Is secure software development part of the vendor’s development process?
  • Are dependencies and containers scanned for known vulnerabilities?
  • Are SAST, DAST, SCA, infrastructure security checks, and manual security reviews used where appropriate?
  • Are APIs tested for authorization flaws such as broken object-level authorization?
  • Are secrets kept out of source code and protected in a secrets-management system?
  • Are production systems separated from development and testing environments?
  • Are backups protected and regularly tested?

Do not ask whether SAST or DAST “tests the AI model.” These tools are mainly used to find weaknesses in the surrounding software. AI-specific testing should be assessed separately.

8. Check logging, monitoring, and privacy

AI logs can be useful during an investigation, but they can also become another place where sensitive information is stored.

  • What AI inputs and outputs are logged?
  • Are prompts, responses, uploaded files, or retrieved documents stored in logs?
  • Who can access those logs?
  • How long are they retained?
  • Can sensitive information be redacted before logging?
  • Are administrative and tool actions separately auditable?
  • Can the vendor detect unusual access, token use, data retrieval, or tool activity?

Ask the vendor to explain its retention policy for both product data and security logs. “We encrypt everything” does not answer the more important question of who can access the data and how long it remains available.

9. Ask how incidents are handled

  • Does the vendor have an incident response plan covering AI-specific incidents?
  • How will customers be notified if customer data is exposed?
  • What happens if the underlying model provider is compromised?
  • How quickly can compromised API keys, sessions, or integrations be revoked?
  • Can the AI feature be disabled while an incident is investigated?
  • How does the vendor handle a newly discovered prompt injection or model security issue?
  • Are incident notification times defined in the contract?

Do not stop at “we have an incident response policy.” For a high-risk deployment, ask what the notification commitment is and whether the vendor can explain how an AI-specific incident would be contained.

10. Review the right security and AI standards

Certifications and frameworks can provide useful evidence, but they should not be treated as proof that an AI product is automatically secure.

  • SOC 2 Type II: Ask for the current report and check its scope, covered services, exceptions, and reporting period. SOC 2 is an attestation report, not an ISO-style certification.
  • ISO/IEC 27001: Useful for assessing the vendor’s information security management system. Check whether the AI service you are buying is included in the certification scope.
  • ISO/IEC 42001: The international standard for an AI management system. It focuses on how an organization manages AI-related risks and responsibilities across the AI lifecycle. ISO explains ISO/IEC 42001 here.
  • NIST AI RMF: A voluntary framework for managing AI risk. NIST also provides a Generative AI Profile for risks specific to generative AI. See the NIST AI RMF.
  • OWASP AISVS: A security verification standard with testable requirements for AI-enabled systems. The current stable release is AISVS 1.0. See OWASP AISVS.

The important point is scope. A vendor may have ISO 27001 or SOC 2 and still have an AI feature that deserves additional technical review. Ask what was actually assessed.

11. Do not forget privacy and regulatory requirements

Compliance depends on what the AI system does, what data it processes, where the organization operates, and which people or customers are affected. A vendor saying “we are GDPR compliant” is not enough to determine your own obligations.

For organizations in India, the Digital Personal Data Protection Rules, 2025 are an important part of the privacy review when personal data falls within the applicable framework. Review the roles of the organization and vendor, the data being processed, retention, transfers, security measures, and contractual responsibilities.

Organizations operating in or serving the European market may also need to consider the EU AI Act. The Act entered its main application phase on 2 August 2026, with different obligations applying on different dates. The European Commission currently lists 2 December 2027 for certain high-risk AI rules and 2 August 2028 for high-risk AI embedded in regulated products. Check the European Commission’s current AI Act timeline before relying on a date in a contract or compliance plan.

12. Ask for evidence, not just yes or no answers

This is where many vendor assessments become too easy. A questionnaire can say “yes” to almost everything. The useful question is what supports that answer.

  • “We do not train on customer data.” Ask for the contractual language and product documentation that supports it.
  • “We are SOC 2 compliant.” Ask for the current report and review its scope.
  • “We test for prompt injection.” Ask what was tested, how often, and whether indirect prompt injection and tool use were included.
  • “Customer data is isolated.” Ask how tenant isolation is implemented and tested.
  • “We encrypt customer data.” Ask what is encrypted, where keys are managed, and who can access them.
  • “We use secure AI models.” Ask which providers are involved and how model changes are managed.

You do not need every vendor to hand over confidential security details. The goal is to collect enough evidence to understand the control, its scope, and whether it applies to the AI service you are actually buying.

A simple AI SaaS security checklist

  • Know what data enters the AI service.
  • Confirm whether prompts, files, responses, or logs are retained.
  • Confirm whether customer data is used for training or fine-tuning.
  • Identify every important model provider and subprocessor.
  • Check tenant isolation for prompts, files, conversations, and embeddings.
  • Test or obtain evidence of prompt injection and indirect prompt injection defenses.
  • Review every tool an AI agent can access.
  • Make sure high-impact actions have strong authorization and, where appropriate, human approval.
  • Review normal application security controls as well as AI-specific testing.
  • Check logging, retention, monitoring, and access to AI data.
  • Review incident notification commitments.
  • Check the scope of SOC 2, ISO 27001, or other assurance reports.
  • Consider ISO/IEC 42001, NIST AI RMF, OWASP AISVS, and applicable privacy or AI regulations.
  • Ask for evidence instead of accepting security claims at face value.

Final thoughts

A secure AI SaaS product is not defined by one certification, one model, or one prompt filter. The real picture comes from how the vendor handles your data, separates customers, controls access, manages models and subprocessors, limits AI actions, tests its systems, and responds when something goes wrong.

If you are reviewing an AI vendor for a real production deployment, start with the data flow and the permissions. Those two areas often tell you more about the practical risk than a long list of security badges on a vendor’s website.

Subscribe
Notify of
guest

0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x
Seraphinite AcceleratorOptimized by Seraphinite Accelerator
Turns on site high speed to be attractive for people and search engines.