AI Security Rules
Deterministic Pattern Library

Code Example:

// Detected Pattern
const systemPrompt = `You are a helpful assistant. User context: ${userInput}`;
await openai.chat.completions.create({
  messages: [{ role: 'system', content: systemPrompt }]
});
// CRITICAL: User input in system prompt

Remediation:

Never interpolate user input into system prompts. Use separate user message roles. Implement input validation and content filtering before AI processing.

Code Example:

// Detected Pattern
const action = await model.generateAction(userRequest);
await executePrivilegedAction(action); // No validation!
// CRITICAL: AI output directly executes

Remediation:

Implement human-in-the-loop for privileged actions. Add output validation before execution. Use allowlists for permitted actions.

Code Example:

// Detected Pattern
const toolCall = response.tool_calls[0];
await tools[toolCall.name](toolCall.arguments);
// CRITICAL: No argument validation

Remediation:

Validate all tool arguments against strict schemas (Zod, JSON Schema). Implement argument sanitization. Use typed function signatures.

Code Example:

// Detected Pattern
const docs = await vectorStore.similaritySearch(query);
const context = docs.map(d => d.content).join('\n');
// HIGH: No boundary markers on retrieved content

Remediation:

Add clear boundary markers around retrieved content. Implement content filtering on RAG results. Use separate context windows for trusted vs untrusted content.

Code Example:

// Detected Pattern
export async function POST(req: Request) {
  const { prompt } = await req.json();
  return await callAI(prompt);
  // HIGH: No auth check before AI call
}

Remediation:

Add authentication middleware to all AI endpoints. Verify session/token before processing. Implement rate limiting per user.

Code Example:

// Detected Pattern
const data = await db.query(aiGeneratedQuery);
// HIGH: No tenant filter on AI query

Remediation:

Always include tenant ID in data queries. Use row-level security. Validate AI-generated queries include proper scoping.

Code Example:

// Detected Pattern
while (!task.complete) {
  await agent.executeStep();
  // HIGH: No max iterations or timeout
}

Remediation:

Implement max iteration limits. Add timeout controls. Require human approval for long-running agents. Log all agent actions.

Code Example:

// Detected Pattern
const prompt = `Connect to DB with password: ${dbPassword}`;
console.log('Processing:', prompt);
// HIGH: Secret in prompt and logs

Remediation:

Never include secrets in AI prompts. Implement secret detection in logging. Use secret managers with runtime injection.

Code Example:

// Detected Pattern
const url = await model.generateUrl(userRequest);
const response = await fetch(url);
// HIGH: AI-controlled URL fetch

Remediation:

Implement URL allowlists. Validate AI-generated URLs against permitted domains. Block internal network access from AI-triggered requests.

Code Example:

// Detected Pattern
await openai.chat.completions.create({
  model: 'gpt-4',
  temperature: 0.9, // HIGH temperature
  messages: securityDecisionPrompt
});

Remediation:

Use temperature=0 for security-critical AI operations. Set fixed seeds for reproducibility. Document AI configuration for audit.

Code Example:

// Detected Pattern
async function aiAdminAction(userId, action) {
  await executeAdminAction(action);
  // HIGH: No permission check
}

Remediation:

Implement RBAC checks before AI operations. Verify user permissions match required action level. Audit all privileged AI actions.

Code Example:

// Detected Pattern
const html = await model.generateHTML(userRequest);
element.innerHTML = html;
// HIGH: AI output to innerHTML

Remediation:

Validate and sanitize all AI output before use. Use schema validation for structured output. Implement content security policies.

Code Example:

// Detected Pattern
export async function POST(req: Request) {
  // No rate limit check
  return await expensiveAICall(req);
}

Remediation:

Implement rate limiting per user/IP. Add request quotas. Monitor for abuse patterns. Implement circuit breakers.

Code Example:

// Detected Pattern
const messages = [
  { role: 'system', content: systemPrompt },
  { role: 'user', content: userInput } // Unbounded!
];

Remediation:

Validate input length before AI context. Implement token counting. Truncate user input to safe limits. Reserve space for system prompts.

Code Example:

// Detected Pattern
const model = req.query.model || 'gpt-4';
await openai.chat.completions.create({ model });
// LOW: User controls model selection

Remediation:

Use fixed model selection or strict allowlist. Do not expose model choice to users. Document model selection rationale.