You don’t need an enterprise governance framework. You need six policies, a few checkpoints, and someone who owns the answer when something goes wrong. Here it is, briefly.
What you don’t need
Most AI governance content on the internet is written for the Fortune 500. NIST frameworks — the AI Risk Management Framework, released January 2023, with its four functions of Govern, Map, Measure, and Manage. ISO 42001, the international standard for an AI management system. AI impact assessments. Model risk management. Algorithmic accountability boards.
If you’re running a $20M service business, you don’t need most of that. You need a practical, defensible posture that protects the business without creating bureaucracy that nobody follows.
This is the version that fits a mid-market service business. It’s shorter than the enterprise version, but it’s not weaker — it’s just appropriately scoped.
The six policies you actually need
1. Data handling
What it covers: What types of company and customer data can be put into which AI tools.
Practical version: Public information can go in any tool. Internal-only data goes only in enterprise instances of approved tools (Claude for Work, ChatGPT Enterprise, Microsoft Copilot in your tenant). Customer-confidential data goes only in tools with a signed BAA or DPA. Regulated data (PII, PHI, financial) requires a documented review before any AI exposure.
Length: One page. Specific examples beat abstract categories.
2. Approved tools list
What it covers: Which AI tools are sanctioned for company use.
Practical version: A short list of approved tools (usually 3-5 for a mid-market business), with the tier of data each is approved for. Unapproved tools require IT review before use. Personal AI accounts on personal devices are out of scope, but employees should not use them for company work.
Length: Half a page. Update quarterly.
3. Human-in-the-loop checkpoints
What it covers: Which AI outputs require human review before being acted on.
Practical version: Always required for outbound customer communication, financial decisions over a set threshold, anything that creates a legal commitment, anything that goes to a regulator, and anything that affects employment. Not required for internal drafts, internal research, summarization of internal documents, or productivity tasks where the human is the final user.
Length: One page with the list of “always required” categories.
4. Disclosure and transparency
What it covers: When you tell customers, employees, or partners that AI was involved.
Practical version: Customer-facing AI interactions (chatbots, AI-drafted responses sent without significant human edit) are disclosed. AI used internally to support a human’s work doesn’t need to be disclosed. Customers asking directly always get a truthful answer. Marketing claims about AI in your delivery should be specific and defensible — not “AI-powered” hand-waving.
Length: Half a page.
5. Vendor and tool evaluation
What it covers: How AI vendors and tools get approved before deployment.
Practical version: Any new AI tool that handles internal or confidential data goes through a short review: who’s the vendor, where’s the data hosted, what’s the data retention policy, what’s the security posture, is there a SOC 2 or equivalent. A simple scorecard. Sign-off from operations plus IT before deployment.
Length: A one-page checklist.
6. Incident response
What it covers: What happens when AI gets something wrong in production.
Practical version: Define what an incident is (any AI output that caused customer harm, financial loss, or reputational risk). Define who’s notified within what timeframe. Define how the incident is documented. Define what triggers a temporary shutdown of the workflow versus a fix-in-place. Define the post-incident review process.
Length: One page. Mostly process, not content.
The roles you need
Three roles. Often part-time. Often combined.
AI Owner (usually the COO or VP Operations): Owns the policies, the approved tools list, and the incident response process. The person who can be paged at 9 PM when something goes wrong.
AI Reviewer (usually IT director plus legal counsel on retainer): Reviews new tools, new vendors, and new use cases that touch sensitive data. Signs off before deployment.
AI Auditor (usually the CFO, internal audit if you have one, or an external auditor on an annual basis): Samples AI workflows for compliance with policies. Doesn’t need to be deep — a quarterly spot-check.
For most mid-market businesses, this is 5-15% of one person’s time across all three roles combined. Not a full-time job.
The checkpoints worth keeping
A few specific checkpoints worth installing, regardless of policy maturity.
Before any AI workflow goes live:
- Has a human reviewed at least 20 sample outputs?
- Is the error mode documented (what does it look like when this fails)?
- Is there a kill switch (a way to disable the workflow in under an hour)?
- Is there a monitoring dashboard showing key quality metrics?
Once a quarter:
- Sample 5-10% of outputs from each higher-risk workflow; review for drift
- Review the incident log; identify patterns
- Re-confirm vendor security postures
- Update the approved tools list
Once a year:
- Full policy review and refresh
- Outside audit of the highest-risk workflows
- Tabletop exercise on an AI-related incident (1-2 hours, useful)
The risks that actually bite mid-market companies
The enterprise frameworks worry about systemic AI risk, algorithmic bias at scale, and macro labor displacement. Worthy concerns, but not your day-to-day.
The risks that actually bite mid-market businesses:
- Customer-facing AI says something wrong or insensitive — visible, embarrassing, harms trust.
- AI processes customer data through a tool that shouldn’t have seen it — a privacy or compliance issue, sometimes regulatory.
- AI-generated content gets sent to a customer with an error nobody caught — minor but cumulative damage to brand.
- An AI workflow drifts and produces bad outputs for weeks before anyone notices — operational impact that’s hard to fully measure.
- An AI vendor changes its terms or pricing, or sunsets a capability you depend on — operational fragility.
- An employee inputs confidential data into a personal AI account — data leakage you may never detect.
The six policies and three roles above cover all six of these risks at a level appropriate for a mid-market business. You don’t need more than this.
Where to start if you have nothing today
If you’re starting from zero governance, do three things this quarter:
- Write the six policies. Use this article as the starting point. One operations leader can draft them in a week. Get sign-off from the owner and legal counsel.
- Name the three roles. Often the same one or two people, with explicit responsibilities documented.
- Install the pre-launch checkpoints on every existing AI workflow. This will surface workflows that should never have gone live without them.
That’s enough governance to operate seriously. Revisit annually as you scale.
If you’d like a 30-minute call to pressure-test your current posture, that’s a useful conversation we have often.