Published by Roast & Rise
Shadow AI: Tools, Detection, Policy, and Prevention
A practical playbook for finding unsanctioned AI tools, choosing detection signals, setting policy, and fixing confirmed exposure.
Find shadow AI with disclosure, identity, browser, network, endpoint, DLP, and procurement signals. Set policy, review findings, and remediate risky AI use.
Course thesis
Shadow AI is a visibility problem before it becomes a control problem. Organisations need an inventory of tools and workflows, data boundaries people can follow, layered signals that show where use is happening, and a calm review process that distinguishes legitimate experiments from risky exposure.
What you leave with
By the end, you'll have a current AI tool inventory, a layered detection plan, a one-page shadow AI policy, and a remediation workflow for confirmed findings.
For
Operations, IT, security, privacy, and AI leads who need to discover unsanctioned AI use without guessing or turning monitoring into covert surveillance.
Workflow
Inventory AI tools and workflows, assess data exposure, publish a usable policy, pilot layered detection, and close findings through a documented remediation loop.
Change
Move from occasional surveys and blanket bans to an owned discovery, policy, monitoring, and remediation loop with transparent handling rules.
What you can do
Use these as checks while you move through the plan.
Build a Shadow AI Inventory that connects tools to workflows, accounts, data, integrations, owners, and decisions.
Assess tool-workflow pairs using company data classifications and provider controls.
Write and test a shadow AI policy with usable examples, exceptions, monitoring notice, and ownership.
Compare disclosure, identity, SaaS, OAuth, browser, endpoint, network, DLP, and procurement signals.
Pilot one authorised detection layer and review its gaps, handling obligations, and false positives.
Validate, contain, decide, communicate, retest, and close a confirmed finding.
Chapters
01
Identify Shadow AI Tools and Workflows
Build a usable shadow AI inventory that connects each tool to a workflow, account, data type, owner, and approval decision.
Shadow AI is an AI service, feature, integration, or model used for company work without the visibility or approval your organisation requires. It can be a chatbot, browser extension, meeting bot, code assistant, API, SaaS MCP server, or an AI feature inside an otherwise approved product.
Record the tool, workflow, account, device, data involved, connected systems, owner, and approval state. A list of product names cannot show whether a harmless experiment and a customer-data workflow carry the same risk.
NIST's Generative AI Profile recommends documenting and regularly monitoring risks from third-party generative AI resources. Use that principle to make the inventory recurring, owned, and tied to decisions.
Start with disclosed use, identity records, and systems you are authorised to review. State what will be collected, who can see it, how long it is retained, and how findings will be handled. A survey can explain why people use a tool. It cannot prove that the inventory is complete.
Worked example
A complete inventory row names one real tool and workflow, the account used, the data category, connected systems, approval state, evidence source, owner, and next review date. Use your own environment; do not fill the inventory with invented findings.
Quality checklist
Scope, purpose, access, retention, and review process were stated before collection.
Each row connects a tool to a real workflow and data category.
The evidence source is recorded for every finding.
Unknown items have an owner and review date.
Shared summaries avoid unnecessary personal detail.
Common mistakes
Recording product names without the workflow or data involved.
Calling every unreviewed tool prohibited before checking context.
Promising anonymity that a small team or identifiable workflow cannot provide.
Collecting prompt content before defining necessity, access, retention, and notice.
Publishing raw responses instead of an aggregated inventory.
Checkpoint
Can you show who uses each discovered AI tool, for which workflow, with which data, and what decision comes next?
Exercise
Build the First Shadow AI Inventory
Steps
- Choose one team or workflow with regular AI use.
- Ask people which AI services, built-in features, browser extensions, APIs, and meeting tools they use for that work. Explain how the answers will be reviewed and avoid promising anonymity where the team is too small to support it.
- Add identity, SaaS, browser, endpoint, network, procurement, or expense records that you are authorised to inspect. Do not collect prompt content by default.
- Create one row per tool and workflow. Record the account type, device, data category, connected systems, owner, approval state, and source of the finding.
- Mark each row as sanctioned, under review, restricted, or unknown. Unknown is a review state, not a verdict.
- Share an aggregated summary with the team and name the next decision for each under-review item.
Output to complete
Complete a Shadow AI Inventory that links tools to workflows, accounts, data, integrations, owners, and approval state.
Copyable template
| Tool or AI feature | Workflow | Account and device | Data category | Connected systems | Approval state | Evidence source | Owner | Next review |
|---|---|---|---|---|---|---|---|---|
| Sanctioned / Under review / Restricted / Unknown |
Use this at work tomorrow
Inventory the AI tools used in one workflow and assign a decision owner to every unknown item.
Core idea
A tool name is not an inventory. Add the workflow, account, data, owner, approval state, and evidence source.
02
Assess Data Exposure and Risk
Assess each tool and workflow against the data involved, provider controls, connected systems, and unresolved questions before deciding what is allowed.
Risk belongs to the combination of tool, workflow, account, data, and integration. The same chatbot can be low risk for rewriting public copy and unacceptable for processing identifiable customer or employee information through an unreviewed account.
Classify the data first. Then check the provider and configuration: purpose, retention, model-training use, access controls, admin logs, region, subprocessors, deletion, incident support, and connected-system permissions. Record unknowns instead of assuming the safest or worst answer.
The ICO's AI guidance and audit resources treat data protection as a lifecycle responsibility, including purpose, roles, security, transparency, and data minimisation.
The NCSC's AI and cyber security guidance advises leaders to understand critical assets, AI data and software supply chains, accountability, and incident response. Use those questions to review each tool-workflow pair.
Worked example
Illustrative decision record: customer-support drafting; Restricted data; approved enterprise account required; provider training disabled; admin logging enabled; no personal account use; data owner and security owner review unresolved retention and deletion terms before approval.
Quality checklist
No live sensitive data was copied into the assessment.
The existing company classification was used or a provisional scheme was clearly labelled.
Provider settings, account type, and connected systems were checked.
Unknowns have owners and review dates.
The decision names conditions and evidence.
Common mistakes
Treating every AI tool as equally risky regardless of workflow and data.
Copying live sensitive data into the assessment document.
Reviewing the product name but not the account, configuration, integrations, or contract.
Turning an unknown into an approval or prohibition without an owner and review date.
Checkpoint
Can the data owner and security or privacy owner explain why each reviewed tool-workflow pair is allowed, conditional, paused, or prohibited?
Exercise
Assess Three Tool-Workflow Pairs
Steps
- Select three real rows from the Shadow AI Inventory.
- Describe the data category without copying personal, confidential, or regulated content into the exercise. Use redacted samples only when necessary.
- Record the business purpose, account type, provider retention and training settings, admin controls, connected systems, region, and deletion path.
- Apply your organisation's existing data classification. If none exists, start with Public, Internal, Confidential, and Restricted and ask the privacy or security owner to confirm the mapping.
- Decide: allow, allow with conditions, pause for review, or prohibit. Write the reason and unresolved questions.
- Ask the data owner and security or privacy owner to review any Restricted or unclear case.
Output to complete
Complete a Tool-Workflow Risk Record for three to five inventory rows using data descriptions or redacted samples, never copied live sensitive data.
Copyable template
| Tool and workflow | Business purpose | Data class | Account and controls | Retention and training | Connected systems | Decision | Unknowns | Owners | Review date |
|---|---|---|---|---|---|---|---|---|---|
| Public / Internal / Confidential / Restricted | Allow / Conditional / Review / Prohibit |
Use this at work tomorrow
Assess one tool-workflow pair using data descriptions, provider controls, integrations, and a written decision.
Core idea
Assess the tool, workflow, account, data, and integrations together. Product names alone do not determine risk.
03
Write a Shadow AI Policy People Can Use
Write a short shadow AI policy that names approved tools, account rules, data boundaries, exceptions, monitoring, ownership, and the route for questions or incidents.
A shadow AI policy should answer the decision in front of the employee: which tools and accounts may I use, for which work, with which data, and what do I do when the case is unclear?
Keep the quick rule short, then link to the tool register, data classification, exception process, and incident route. Name the policy owner, review date, sanctioned tools, restricted data, required account settings, connected-system rules, monitoring notice, and route for questions.
Policy is one control. Identity, access, configuration, DLP, logging, procurement, and review must enforce the boundaries that matter. A blanket ban without a usable approved path usually leaves the original work need unresolved.
Monitoring can involve employee and personal data. Explain what is collected and why, restrict access and retention, and involve the relevant privacy, legal, security, and worker-representation owners for your jurisdiction. The ICO's transparency guidance is a useful starting point.
Worked example
Illustrative policy excerpt: use company-managed accounts for sanctioned AI tools; do not enter Restricted data unless the exact workflow and provider configuration are approved; request a review before connecting an AI app to company systems; report accidental exposure through the existing security route; ask the named AI policy owner when the rule is unclear.
Quality checklist
The quick rule fits on one page and links to the detailed controls.
Sanctioned tools, accounts, data classes, and connection rules are specific.
Approved, prohibited, and conditional examples come from real work.
Monitoring, access, retention, exceptions, incidents, ownership, and review date are named.
Test readers make consistent decisions without extra explanation.
Common mistakes
Publishing a banned-tool list without an approved alternative for the work.
Naming data rules but not account, integration, or plugin rules.
Using vague labels such as sensitive without linking to the company classification.
Hiding monitoring practices or leaving access and retention undefined.
Leaving exceptions, incidents, and questions without a named owner.
Checkpoint
Can three people make the same decision on an approved, prohibited, and conditional AI use case using only the quick rule?
Exercise
Draft and Test the Shadow AI Quick Rule
Steps
- Fill the template using real sanctioned tools, account requirements, data classes, workflows, owners, and contact routes.
- Add two approved examples, two prohibited examples, and one conditional example from normal work. Use descriptions, not copied sensitive data.
- State which monitoring signals are used, their purpose, who can access findings, and where the full notice lives.
- Give the draft to three people who do the work. Ask each person to decide what they would do in the conditional example without extra explanation.
- Rewrite every point that produces different answers.
- Publish the quick rule beside the approved-tool list and record the owner and review date.
Output to complete
Draft and test a one-page Shadow AI Quick Rule linked to the tool register, data classification, exception process, and incident route.
Copyable template
Shadow AI Quick Rule:
Purpose and scope: [Who and which work this covers]
Sanctioned tools and required accounts: [Link to live register]
Data boundaries: [Link to company classification and name prohibited classes]
Approved, prohibited, and conditional examples: [Use real workflow descriptions]
Connections and plugins: [When approval is required]
Monitoring notice: [Signals, purpose, access, retention, full notice]
Exceptions and questions: [Owner and route]
Accidental exposure or incident: [Existing reporting route]
Policy owner and next review date: [Name and date]
Use this at work tomorrow
Write the quick rule for one workflow, test it with three people, and fix the first point they interpret differently.
Core idea
A usable shadow AI policy names the tool, account, workflow, data boundary, exception route, monitoring notice, and owner.
04
Choose Shadow AI Detection Tools
Compare shadow AI detection methods, understand what each signal finds and misses, then pilot a transparent review loop with explicit privacy and retention boundaries.
Shadow AI detection tools observe different signals. No single signal gives a complete or automatically correct answer. Build coverage in layers and write down what each layer misses.
- Disclosure and interviews: reveal purpose, workflow, and workarounds; miss unreported use.
- Identity, SSO, SaaS, and OAuth inventory: reveal company-account access and connected apps; miss personal accounts and unmanaged devices.
- Browser and endpoint inventory: reveal installed extensions, desktop apps, and local assistants on managed devices; miss activity outside managed endpoints.
- DNS, proxy, CASB, or SSE signals: reveal connections to known AI services; may not show the data or purpose and can miss off-network traffic.
- DLP and sensitivity controls: reveal or stop supported transfers of classified data; depend on accurate classification, coverage, configuration, and licensing.
- Procurement and expense records: reveal purchased services; miss free tools and personal payments.
Application discovery and sensitive-data detection are separate questions. Microsoft's current deployment model illustrates a staged sequence: discover AI apps, block selected unsanctioned apps, prevent sensitive data from reaching sanctioned apps, then govern interactions. Treat it as one platform example, not a universal stack.
Choose the smallest useful combination for your environment. Define purpose, lawful basis where relevant, notice, access, retention, false-positive review, escalation, and deletion before collection. A detection is a lead for review, not proof of misuse.
Worked example
Illustrative pilot: an authorised SaaS inventory finds an unreviewed AI meeting assistant connected through a company account. The reviewer records the OAuth scopes, owner, meeting type, data categories, retention terms, and business need before deciding whether to approve, restrict, or remove it. The discovery event alone does not determine the outcome.
Quality checklist
The detection question and scope are written before collection.
Purpose, notice, access, retention, deletion, and escalation are defined.
The matrix states what the signal finds and misses.
Every finding records source, confidence, context, owner, and review status.
False positives and coverage gaps are reviewed before expansion.
Common mistakes
Buying a shadow AI detection tool before defining the question and response process.
Treating application discovery as proof that sensitive data was shared.
Inspecting prompt content by default when metadata would answer the scoped question.
Ignoring personal accounts, unmanaged devices, APIs, plugins, and built-in AI features.
Expanding a pilot without reviewing false positives, access, retention, and employee notice.
Checkpoint
Can you explain what the pilot signal detects, what it misses, what data it collects, who reviews it, and how a finding becomes a decision?
Exercise
Pilot One Shadow AI Detection Layer
Steps
- Choose one scoped question, such as which generative AI web apps are reached from managed devices or which AI browser extensions are installed.
- Compare the available signal layers. Record what each can detect, what it misses, data collected, required access, retention, cost, and owner.
- Select one signal that is already available and authorised. Do not add prompt or payload inspection unless its necessity, legal basis, notice, access, and retention have been reviewed.
- Notify the affected team in plain language before the pilot.
- Run the check and log each finding with the source, confidence, context requested, decision owner, and review status.
- Measure false positives and gaps. Do not treat a domain hit, extension, or OAuth grant as proof of data exposure.
- Decide whether to keep, change, or stop the signal before expanding scope.
Output to complete
Create a Shadow AI Detection Coverage Matrix and complete one authorised pilot review with a finding log.
Copyable template
| Signal layer | Question answered | Finds | Misses | Data collected | Access | Retention | Cost or licence | Owner | Pilot decision |
|---|---|---|---|---|---|---|---|---|---|
| Disclosure / Identity-SaaS-OAuth / Browser-endpoint / Network-CASB-SSE / DLP / Procurement |
Finding log: Date | Tool or service | Signal source | Confidence | Context requested | Data category | Review status | Decision owner | Next action | Due date
Use this at work tomorrow
Choose one authorised signal, document its gaps and handling rules, and run a scoped pilot review.
Core idea
Shadow AI detection needs layers. Every signal finds something, misses something, and creates its own handling obligations.
05
Monitor, Review, and Remediate
Turn shadow AI monitoring into a review and remediation loop that validates findings, contains exposure, makes a decision, updates controls, and retests the result.
Shadow AI monitoring is the recurring review of new tools, changed permissions, new integrations, policy exceptions, repeat findings, and unresolved inventory rows. Set a cadence that matches the risk and pace of change instead of collecting data continuously by default.
Shadow AI remediation begins with validating the finding. Confirm the tool, account, workflow, data, integration, and user context. Then classify the exposure, contain it where necessary, preserve the minimum evidence required, and assign a decision owner.
Use a consistent response sequence: validate; assess data and access; contain; approve, restrict, or remove the tool; revoke or rotate access where needed; notify the right owners; update the register and policy; communicate the decision; retest the control; close with evidence.
Shadow AI prevention improves when the review explains why the tool appeared. If the sanctioned path is missing, slow, or unclear, blocking one tool can move the same workflow elsewhere. Track time to review, false positives, repeat findings, open exceptions, and overdue remediation. Do not claim risk reduction from discovery counts alone.
Worked example
Illustrative remediation record: an unreviewed meeting assistant is confirmed through an OAuth grant. The owner pauses the integration, checks meeting categories and retention terms, removes access to Restricted meetings, completes the provider review, updates the sanctioned-tool register, communicates the permitted use, and retests the grant before closure.
Quality checklist
The finding was validated before a decision was made.
Data, access, and connected-system exposure were classified.
Containment and incident escalation matched the confirmed risk.
The decision, owner, reason, and due date are recorded.
The register, policy, approved path, training, or detection control was updated.
Retest and closure evidence are attached.
Common mistakes
Treating an automated signal as a confirmed incident.
Blocking the tool without addressing the workflow need that caused its use.
Closing the ticket before access, policy, communication, and retest are complete.
Counting discoveries as risk reduction without tracking decisions or repeat findings.
Keeping exceptions open without owners, conditions, or review dates.
Checkpoint
Can you show one finding that moved from signal to validated decision, control update, retest, and documented closure?
Exercise
Close One Shadow AI Finding
Steps
- Select one real open finding from the inventory or detection pilot.
- Validate the signal with the tool owner or affected team. Record what is confirmed and what remains unknown.
- Classify the data and connected-system exposure. Use the existing incident process immediately if the finding meets its threshold.
- Contain proportionately. This can include pausing an integration, removing an extension, restricting a workflow, revoking a token, or moving work to a sanctioned account.
- Decide whether to approve, conditionally approve, restrict, or remove the tool. Record the owner and reason.
- Update the tool register, quick rule, approved path, training, or detection control that allowed the gap.
- Communicate the decision to the affected people. Retest the control and attach closure evidence.
- Set the next review date and record whether the same need could reappear through another tool.
Output to complete
Complete a Shadow AI Finding and Remediation Log, including validation, containment, decision, communication, control update, retest, and closure evidence.
Copyable template
Finding ID and date:
Tool, account, workflow, and integration:
Signal source and confidence:
Validated context and remaining unknowns:
Data and access classification:
Containment taken:
Decision: Approve / Conditional / Restrict / Remove:
Access or credential changes:
Owners notified:
Register, policy, approved path, or training update:
Retest and closure evidence:
Decision owner, due date, and next review:
Use this at work tomorrow
Take one open finding through validation, decision, control update, retest, and documented closure.
Core idea
Detection finds a lead. Remediation validates context, contains exposure, makes a decision, updates the system, and retests it.
30-day path
Week 1: Build the Shadow AI Inventory for one team or workflow and assign decisions to unknown items.
Week 2: Assess the highest-risk tool-workflow pairs and publish the Shadow AI Quick Rule.
Week 3: Compare available signal layers, complete monitoring review, and run one authorised detection pilot.
Week 4: Take a confirmed finding through remediation, update the controls, and retest.
Day 30: Review coverage gaps, false positives, open exceptions, repeat findings, and overdue remediation.
Success signals
Discovered tools linked to workflows, accounts, data classes, owners, and approval states.
Under-review and unknown items have decision owners and dates.
Policy understanding is tested with approved, prohibited, and conditional examples.
At least one detection signal is piloted with coverage gaps and false positives recorded.
Findings track time to review, decision, remediation, retest, and closure.
Repeat findings and overdue exceptions are visible in the review cadence.
Reflection prompts
Which AI-enabled workflows are absent from the current inventory?
Which detection question can existing signals answer with the least additional collection?
Which policy rule is hardest to apply during real work?
Which findings repeat because the approved path is missing or slow?
What evidence shows a finding was closed instead of merely discovered?
Manager checklist
Name owners for the inventory, policy, detection pilot, and finding review.
Confirm privacy, legal, security, and worker-representation review where monitoring requires it.
Keep the approved-tool path usable for the work people need to complete.
Review gaps and false positives before expanding monitoring scope.
Require retest and closure evidence for remediation.
In this library
Related RisePlans
AI-Ready Data Foundation Sprint
Map the data one AI workflow uses, test its fitness and access, fix the blocking gaps, and assign ownership for ongoing review.
AI Sprawl Cleanup Sprint
Bring order to fragmented AI use. Map every tool, prompt, and cleanup hour. Diagnose waste and workslop. Set the rules and rhythms that make team experiments compound instead of costing trust, time, and money.
Company Knowledge System
Turn scattered documents, decisions, examples, and team know-how into a maintained company memory people can trust and use in daily work.
Want this shaped around your company?
Risey can research your company foundation first, then build a version of this path around your real workflows, customers, and culture.
Start with your company