Every vendor in your stack has launched AI features, or is about to. High Alpha’s 2025 SaaS Benchmarks Report puts a number on it: 92% of SaaS companies have either shipped an AI feature or have one on the roadmap, with roughly a third (36%) treating AI as core to the product rather than a bolt-on. Some of them have launched AI features for features you didn’t know they had. Your upcoming renewals will include AI tier pricing, AI add-on pitches, and conversations about AI roadmaps designed to justify price increases.
The question you need to answer before any of those conversations: which AI features in your current stack are actually changing how your team works, and which are marketing fixtures that nobody uses?
This is the AI tools audit — and it requires a different evaluation lens than the standard SaaS usage review. Usage data alone doesn’t tell you whether an AI feature is delivering value; it tells you whether employees are clicking on it. Those are different things.
Why Standard Usage Metrics Fail for AI Features
In a traditional SaaS audit, usage is a reasonable proxy for value. If a tool is used daily by most of its licensed users, it’s probably delivering something. If utilization is low, you have a conversation.
AI features break this model in two directions.
Some AI features show high usage but low value — they’re easy to invoke, produce plausible-looking output, and get used as a shortcut even when the output requires significant correction before it’s useful. Employees use them because they’re there, not because they’re faster or better than the alternative. The usage data looks healthy. The actual productivity impact is negligible or negative (if you count the time spent reviewing and fixing AI-generated work).
Other AI features show low measured usage but high value — they run automatically in the background, handle an exception queue, or produce outputs that flow into other tools without a user actively invoking them. A CRM that automatically enriches contact records with company data doesn’t require a user to click anything. Usage logs show zero activity. The value is real.
An AI tools audit needs to measure behavioral change and workflow impact, not just click counts.
The Evaluation Framework
A practical AI feature evaluation has four questions:
1. What did this feature replace or eliminate?
The strongest AI features have a clear answer: “We used to spend X hours per week on Y task. Now we don’t.” This could be manual data entry, report generation, support ticket triage, meeting note-taking, or contract review. If you can’t identify what the feature eliminated or substantially reduced, it’s probably additive (a nice-to-have generating AI output alongside existing work) rather than substitutive (actually removing a task from your team’s plate).
Ask the feature’s primary users directly: what would you do if this feature disappeared tomorrow? If the answer is “go back to what we did before” — and that alternative is not significantly worse — the feature is a convenience, not a productivity change. If the answer is “I don’t know, I’d have to figure that out,” the feature is embedded in the workflow in a meaningful way.
2. Is the output quality high enough to use without review?
This is the most common failure mode in AI feature theater. The feature produces outputs — draft emails, meeting summaries, suggested responses, analytical narratives — that require nontrivial human review before they’re usable. The employee still does the intellectual work of evaluating, correcting, and finalizing. The AI feature added a step (generating the draft) that didn’t necessarily save time net of review.
Track rejection rate: what percentage of AI-generated outputs get used as-is versus substantially rewritten versus ignored entirely? If your team is rewriting more than half of what the AI produces in a category, the feature is creating work, not removing it. If outputs are used as-is most of the time, the feature is genuinely substituting.
3. Is there a usage cost you’re not tracking?
AI features increasingly come with consumption pricing layered on top of seat costs. The feature may look valuable in isolation — it does something useful — but the marginal cost per use makes the economics unattractive compared to the manual alternative.
Map the cost per AI interaction for any consumption-priced feature. If your CRM charges $0.05 per AI-enriched record and you’re enriching 50,000 records a month, that’s $2,500/month in a feature that might be replacing 20 hours of contractor research work. That’s a real value question — and many operators are not running this math because the charges appear as usage fees that roll into a larger invoice.
4. What’s the switching cost if this feature disappears?
Vendors use AI features as retention tools. If your team’s workflows have been built around an AI feature — prompt templates, integration pipelines, downstream automations — those workflows become switching costs. This is not necessarily a problem (embedded features often represent real value), but it’s worth making conscious.
Before committing to an AI-tier upgrade that significantly embeds vendor AI in your operations, assess exit cost: if this vendor raises prices 40% in 18 months, what does it cost to move to an alternative? Operationally, not just commercially.
Applying the Framework by Category
CRM AI features (lead scoring, email generation, deal risk prediction): Focus on behavioral change at the rep level. Are reps spending less time on manual data entry? Are follow-ups happening faster? Deal risk prediction is often a feature with high visual appeal and low actual influence on decision-making — track whether the prediction scores actually change rep behavior or just generate anxiety.
Project management AI features (status reports, blocker detection, workload recommendations): These tend to be additive rather than substitutive at SMB scale. Ask whether the AI outputs are producing decisions that wouldn’t otherwise happen, or producing formatted summaries of information managers already had.
Operations and finance AI features (invoice processing, expense categorization, variance analysis): This is where AI features tend to deliver genuine substitutive value because the baseline tasks are high-volume, low-variability, and rule-based. An AI that categorizes 90% of expense line items correctly with the remainder flagged for review is actually faster than manual processing. Evaluate based on exception rate and processing accuracy, not just whether the feature exists. This is also the territory where purpose-built agentic AI workflows tend to clear the value bar for SMBs — worth checking whether a narrower, purpose-built agent handles the exception-routing case better than the AI tier bundled into your existing platform.
Communication and productivity AI features (meeting summaries, email drafting, document summarization): Highly variable. Some teams find these transformative; others find the output requires so much editing that it’s faster to write from scratch. Gather self-reported data from actual users and weight individual results by the type of work being AI-assisted.
What to Do with the Results
After running the framework for each AI feature in your stack, you’ll have a clearer picture: features that are genuinely changing how work gets done, features that are being used without clear value, and features nobody is using.
For genuine-value features: model the cost against the value and make a deliberate decision about the pricing tier. Some features justify a consumption-priced upgrade; others deliver enough value at a free tier that upgrading doesn’t pencil.
For theater features: turn them off or don’t renew them. Vendor AI add-ons are often the first thing to cut when rationalizing SaaS spend, but only if you’ve confirmed they’re actually theater rather than quietly embedded in workflows you haven’t mapped.
For unused features: treat as a training opportunity. Some AI features are genuinely useful but unadopted because nobody showed the team how to use them effectively. A 90-minute training session sometimes rescues a feature that would otherwise appear on the cancellation list.
The vendors in your stack are moving toward AI pricing regardless of whether their AI features are ready for the pricing tier they’re selling. Your leverage in that conversation depends entirely on knowing which features your team actually needs — not which ones were demonstrated in the sales call.
Where the Audit Connects to Bigger Stack Decisions
The AI tools audit is a lens on top of a broader stack-rationalization process, not a replacement for it. If you haven’t run a full SaaS stack audit in the last year, do that first — it surfaces the redundant and unused tools that make the AI-feature question moot, because you’re cutting the whole platform, not just its AI tier. The AI audit matters most for tools that survive that first pass and are coming up for renewal with an AI-tier upsell attached.
The cost side of this audit is inseparable from how consumption-based AI pricing actually works — a feature can pass the value test in this framework and still be a bad deal once the per-use cost is modeled against realistic team-wide adoption rather than pilot-level usage. And if your organization has meaningful shadow IT exposure, individual teams may already be paying for AI add-ons that never went through a formal evaluation at all — the audit only works if it covers tools bought outside the main procurement channel too, not just the ones IT signed off on.
Finally, treat “should we fire this tool” and “should we pay for its AI tier” as related but separate questions. A platform can warrant replacement on core functionality grounds even if its AI features test out well — don’t let a genuinely good AI feature talk you out of a platform decision that was already overdue.
Frequently Asked Questions
How is an AI feature audit different from a regular software usage audit?
A regular usage audit treats clicks and logins as a proxy for value. AI features break that proxy in both directions — some get used constantly and add no value, others run invisibly in the background and add real value with zero visible usage. The AI audit measures workflow and behavioral change instead of raw usage counts.
What’s the fastest way to tell if an AI feature is “theater”?
Ask the feature’s regular users one question: what would you do if it disappeared tomorrow? If the honest answer is “go back to what we did before, and it wasn’t much worse,” the feature is a convenience, not a productivity gain. If the answer is “I’d have to figure that out,” it’s genuinely embedded.
Should we track how often employees reject or rewrite AI-generated output?
Yes — it’s one of the clearest value signals available. If your team is substantially rewriting more than half of what an AI feature produces in a given category, it’s adding a review step rather than removing work. If outputs are used as-is most of the time, the feature is functioning as a real substitute for manual work.
Which categories of AI features tend to deliver the most reliable value at SMB scale?
Operations and finance features — invoice processing, expense categorization, variance flagging — tend to score best, because the underlying tasks are high-volume, rule-based, and easy to measure by exception rate. Communication and productivity features, like meeting summaries and email drafting, are the most variable and depend heavily on how the specific team uses them.
What should we do with an AI feature that tests as valuable but is switching to consumption pricing?
Model the per-use cost against the staff time or contractor cost it’s actually replacing, using realistic team-wide volume rather than pilot volume. If the economics hold at full adoption, keep it and negotiate spend caps. If they only work at pilot-level usage, treat the upgrade as a red flag, not a foregone renewal.
