Quick Win
Complete this audit in 2 hours and have a full inventory of AI tools by end of day.
Table of Contents
Why You Need an AI Tool Audit
You can't manage what you can't see. The average company has 15-30 AI tools in use, but most are unknown to leadership. This "Shadow AI" represents tools used without approval—and it's a compliance nightmare.
Real Example: Carol's Discovery
Carol discovered her team was using 23 different AI tools. She only knew about 3. One of them (ChatGPT) was being used to debug code containing patient data. That's a HIPAA violation worth $50,000.
Shadow AI Statistics
Why This Matters
- •Legal and compliance will ask for this list
- •Fines for non-compliance: $50,000+ per violation
- •Data breaches can cost millions in damages
- •Your reputation and customer trust are at stake
The 3 Types of AI Tools You'll Find
Type 1: Code Assistants
HIGH RISKDeveloper tools that suggest code completions. High risk if handling PHI/PII because code often contains sensitive data in comments, variable names, and test data.
Examples:
- • GitHub Copilot
- • Amazon CodeWhisperer
- • Tabnine
- • Codeium
Type 2: General AI Chatbots
CRITICAL RISKGeneral-purpose AI assistants. Critical risk if used for work because most don't offer BAAs, data may be used for training, and there are no audit logs.
Examples:
- • ChatGPT
- • Claude
- • Gemini (Google Bard)
- • Perplexity
Type 3: Specialized AI Tools
MEDIUM-HIGH RISKPurpose-built AI tools for specific tasks. Risk depends on data being processed and whether a BAA is available.
Examples:
- • Grammarly (writing)
- • Notion AI (documentation)
- • Jasper (marketing)
- • Otter.ai (transcription)
| Tool Type | Example | PHI Risk | BAA Available? | Action |
|---|---|---|---|---|
| Code Assistant | GitHub Copilot | HIGH | Get BAA | |
| General Chatbot | ChatGPT | CRITICAL | Block or upgrade to Enterprise | |
| Specialized | Grammarly | MEDIUM | Get BAA |
The 5-Step Audit Process
Follow these five steps to discover every AI tool in your organization. Total time: 2 hours.
Survey Your Team
Time: 30 minutes
Send an anonymous survey to all employees. Make it anonymous—people won't admit to using unapproved tools if they think they'll get in trouble.
Survey Questions:
- 1. Are you using any AI tools for work? (Yes/No)
- 2. Which AI tools do you use? (List all)
- 3. How often do you use them? (Daily/Weekly/Monthly)
- 4. What do you use them for? (Code/Writing/Research/Other)
- 5. Do you input company data? (Yes/No/Not sure)
Email Template:
Subject: Quick Survey: AI Tools (2 minutes, anonymous)
Hi team,
I'm working on ensuring we're compliant with AI regulations.
Could you take 2 minutes to fill out this anonymous survey about AI tools you use for work?
[Survey Link]
No judgment - just trying to get visibility so we can make sure we're set up properly.
Thanks!
Check Browser Extensions
Time: 15 minutes
Ask IT to run a browser extension audit across the company.
Common AI Extensions:
- • Grammarly
- • ChatGPT for Chrome
- • Notion AI
- • Jasper
- • Copy.ai
- • Wordtune
How to Check (Non-Technical):
- 1. Ask IT: "Can you pull a list of all browser extensions installed across the company?"
- 2. Filter for AI-related keywords: "AI", "GPT", "assistant", "copilot", "writer"
- 3. Export to spreadsheet
Review Expense Reports
Time: 20 minutes
Check company credit cards and expense reports for AI subscriptions.
Common Charges to Look For:
- • OpenAI ($20/month = ChatGPT Plus)
- • GitHub Copilot ($10-19/month)
- • Jasper ($49-125/month)
- • Grammarly ($12-15/month)
How to Check:
- 1. Export last 6 months of expenses
- 2. Filter for "AI", "GPT", "OpenAI", "Anthropic", "Copilot"
- 3. Add to inventory
Check API Usage
Time: 30 minutes
Review developer API keys and integrations in your codebase.
Common AI APIs:
- • OpenAI (GPT-4, GPT-3.5)
- • Anthropic (Claude)
- • Cohere
- • Hugging Face
- • Google PaLM
How to Check (Non-Technical):
Ask your tech lead:
"Can you check if we're using any AI APIs in our code? Specifically OpenAI, Anthropic, Cohere, or Hugging Face?"
Interview Team Leads
Time: 45 minutes
Conduct 15-minute 1-on-1s with each team lead. They know what their team is doing.
Questions to Ask:
- 1. "What AI tools is your team using?"
- 2. "Are there any tools you've heard about but aren't sure if they're approved?"
- 3. "Has anyone asked about using ChatGPT or similar tools?"
- 4. "Do you know if anyone is using personal AI accounts for work?"
- 5. "What would make your team more productive with AI?"
How to Discover "Shadow AI"
What is Shadow AI?
Definition: AI tools used without IT/leadership approval.
Why It Happens:
- • Developers want to be productive
- • Approval process is too slow
- • They don't realize it's a compliance issue
- • Free tools are easy to start using
5 Places Shadow AI Hides
1. Personal Accounts
Developer uses personal ChatGPT account for work. No company visibility, no BAA protection.
2. Free Tiers
GitHub Copilot Individual (free), ChatGPT Free, Claude Free—all without company oversight.
3. Browser Extensions
Installed without IT approval, auto-updates, access to all browsing data.
4. Mobile Apps
ChatGPT app on personal phone used for work emails and code reviews. No MDM control.
5. Shared Accounts
Team shares one ChatGPT Plus account. No audit trail, can't track who used what.
How to Find Shadow AI
Network Traffic Analysis:
Ask IT to check for traffic to: openai.com, anthropic.com, chat.openai.com, claude.ai, bard.google.com
Code Repository Scan:
Search GitHub for: `import openai`, `anthropic`, `@anthropic-ai/sdk`, API keys starting with `sk-`
Exit Interviews:
Ask departing employees: "What tools did you use that weren't officially approved?" (They'll be honest since they're leaving)
Creating Your AI Tool Inventory
Now that you've discovered all the tools, it's time to organize them into a comprehensive inventory.
The Spreadsheet Template
Use this structure to track every AI tool in your organization:
| Tool Name | Category | Users | Cost/Month | BAA Status | Risk Level | Action |
|---|---|---|---|---|---|---|
| GitHub Copilot | Code | 15 | $285 | Low | None | |
| ChatGPT | General | 50 | Free | Critical | Block or upgrade | |
| Grammarly | Writing | 30 | $450 | Medium | Get BAA |
How to Fill It Out:
- 1. List every tool you found (Steps 1-5)
- 2. Categorize each tool (Code/General/Specialized)
- 3. Count users (from survey + expense reports)
- 4. Calculate monthly cost
- 5. Check if BAA is available
- 6. Assign risk level (use matrix from earlier)
- 7. Determine action needed
Risk Assessment Framework
Use this framework to score the risk level of each AI tool.
3 Risk Factors
1. Data Sensitivity
- • Does it process PHI? → HIGH RISK
- • Does it process PII? → MEDIUM RISK
- • No sensitive data? → LOW RISK
2. BAA Availability
- • BAA available and signed? → LOW RISK
- • BAA available but not signed? → MEDIUM RISK
- • No BAA available? → HIGH RISK
3. User Count
- • 1-10 users → LOW RISK
- • 11-50 users → MEDIUM RISK
- • 51+ users → HIGH RISK
Action Based on Risk
CRITICAL
Immediate action (block or get BAA within 7 days)
HIGH
Urgent (resolve within 30 days)
MEDIUM
Important (resolve within 90 days)
LOW
Monitor (annual review)
What to Do With Your Findings
Organize your findings into three buckets and take action.
Bucket 1: Approved & Compliant
Examples: GitHub Copilot Business (with BAA), Grammarly Business (with BAA)
Action: None. Document and monitor.
Bucket 2: Approved But Non-Compliant
Examples: GitHub Copilot Individual (no BAA), ChatGPT Plus (no BAA for Plus tier)
Action: Upgrade to compliant tier or get BAA.
Bucket 3: Unapproved & Non-Compliant
Examples: Personal ChatGPT accounts, Free AI tools, Shadow AI
Action: Block immediately or create approval process.
The 30-Day Action Plan
Week 1: Communicate
- • Email team with findings
- • Explain why compliance matters
- • No blame, just education
Week 2: Upgrade
- • Upgrade tools to compliant tiers
- • Get BAAs signed
- • Set up company accounts
Week 3: Block
- • Block unapproved tools at network level
- • Remove browser extensions
- • Disable personal accounts
Week 4: Monitor
- • Set up ongoing monitoring
- • Monthly usage reports
- • Quarterly audits
Email Template to Team
Subject: AI Tool Audit Results - Action Required
Hi team,
I completed an audit of AI tools we're using. Here's what I found:
✅ APPROVED (keep using):
- GitHub Copilot Business
- Grammarly Business
⚠️ NEEDS UPGRADE (action required by [date]):
- GitHub Copilot Individual → Upgrade to Business
- ChatGPT Plus → Upgrade to Enterprise (or stop using)
🚨 NOT APPROVED (stop using immediately):
- Personal ChatGPT accounts for work
- Claude Free
- Any AI tool not on approved list
Why this matters:
- HIPAA compliance (we handle patient data)
- Potential fines: $50,000+ per violation
- Protecting our company and customers
New policy: [link]
Approved tools list: [link]
Questions? Reply to this email.
Thanks for your cooperation!
Conclusion
You've now completed your AI tool audit. You have a complete inventory, know which tools are compliant, and have a 30-day action plan to fix any issues.
Next Steps
- 1. Download the audit template
- 2. Complete the 5-step audit
- 3. Create your inventory spreadsheet
- 4. Assess risk for each tool
- 5. Execute 30-day action plan
- 6. Set up ongoing monitoring
Remember:
- • This is an ongoing process (not one-time)
- • Quarterly audits recommended
- • New AI tools launch every week
- • Stay vigilant
Download: AI Tool Audit Template
Get our free Google Sheets template with pre-built formulas, auto-risk scoring, and action plan generator.
Related Articles
You're Not Alone: Why Every PM Over 50 is Struggling with AI Compliance
73% of project managers don't understand AI compliance. Learn why this is hard and how to get started.
GitHub Copilot and HIPAA: What Every Healthcare PM Needs to Know
Complete guide to GitHub Copilot HIPAA compliance in plain English. Includes BAA checklist.
Found this helpful? Share it!
Help other project managers discover this audit guide.
AI Program Management Framework (CSM6)
A structured approach to AI governance. Free interactive checklist, templates, and step-by-step guide for project managers.