Login counts measure licenses, not maturity
Picture a CIO walking into a board meeting with a Copilot usage dashboard: 400 seats activated, 78% logged in this month. The board asks the obvious question anyway: "Are we actually more capable with AI, or did we just buy licenses?" The dashboard has no answer, and neither does a login-based AI maturity assessment template.
The common move is to build your first AI maturity assessment template straight from that admin console: logins, seats, license utilization. It's fast, it's free, and nobody has to design a survey. But a login only proves someone opened the tool. It says nothing about whether they can configure an agent, whether governance exists, or whether anyone shares what works.
Score capability, not access. An AI maturity assessment template that scores more than tool logins and licenses needs separate pillars for skills, governance, culture and strategy, not one number pulled from a usage report. Microsoft's own agentic AI adoption maturity model, documented on Microsoft Learn as of May 2026, is built this way: five maturity levels across five capability pillars covering strategy, process, governance, architecture and organizational readiness.
- AI strategy and experience
- Business strategy
- AI governance and security
- Technology and data
- Organization and culture
One of those five pillars is organizational culture, skills and enablement, sitting alongside governance and architecture rather than folded into a usage count.
The stakes of getting this right are real: a 2020 study by MIT Sloan Management Review and BCG, cited in MITRE's AI Maturity Model guide, found that despite rising investment, only 10% of organizations were achieving significant financial benefit from AI. Counting seats wouldn't have caught that gap either.
Sources: Introduction to the Agentic AI adoption maturity model | Microsoft Learn
What you need before you start scoring
Before you open a spreadsheet, gather what the template will actually run on. Perception alone won't fill it in correctly; some pillars need records, not opinions.
You need four things in hand: admin-level access to usage and access logs, named owners for each pillar who can speak to evidence, a short list of stakeholders to interview about skills and culture, and a decision on how many pillars you'll track.
MITRE's AI Maturity Model and Assessment Tool guide, published in November 2023, organizes maturity across six pillars: ethical, equitable and responsible use; strategy and resources; organization; technology enablers; data; and performance and application. You don't need six, but it shows how far a pillar set can stretch once skills and usage aren't the only categories.
Build your AI maturity assessment tool in seven steps
With the prerequisites in hand, work through the template in order. Whether you build this in a spreadsheet, a one-page AI maturity assessment PDF for the board, or a dedicated AI maturity assessment tool, the seven steps below stay the same.
Steps two and three are where most templates go soft, so borrow the specificity of Microsoft's governance pillar as a model for writing descriptors. At its lowest level, agents run without dedicated monitoring because no formal operational support exists for them.
A universal anti-pattern it flags at any level is that security posture isn't continuously validated. Another is that controls stay guidance-only instead of enforceable, so compliance depends on individual behavior rather than a verifiable system. That's the level of specificity a good descriptor needs: a condition you can check, not a feeling you can rate.
For step four, Microsoft's own how-to guidance for the model instructs teams to score each pillar on its own scale and explicitly avoid collapsing results into one overall number. We think a blended, single-digit maturity score is close to useless for planning: it averages a strong usage pillar against a weak governance pillar and tells leadership nothing about where to spend the next budget.
Step five is visualization: the guidance recommends plotting current maturity levels as a radar chart, so stakeholders see the shape of the gap, not just an average.
Step seven is cadence, covered in its own section below, but watch for one thing between cycles: Microsoft's model names "champion dependency" as a Level 200 anti-pattern, where adoption rests on a few motivated individuals without systematic enablement, and stalls the moment those people move on.
Sources: Agentic AI maturity model - Organization and culture | Microsoft Learn, Agentic AI maturity model - AI governance and security | Microsoft Learn, How to use the agentic AI adoption maturity model | Microsoft Learn
What a finished template looks like
When the template is done, it reads like something a CFO would actually trust: a handful of pillars, each scored independently, each with a named owner, a next action, and a label that says whether the evidence came from a system of record or from someone's own account of things.
| Pillar | Level descriptor example | Required evidence | Data type |
|---|---|---|---|
| Usage | Agents run in production across three teams | Production agent inventory | System of record |
| Skills | Most staff can write a prompt but not configure an agent | Skills self-assessment survey | Self-reported |
| Governance | Controls are guidance-only, not enforceable | Policy audit against technical controls | System of record |
| Culture | Adoption, confidence and sentiment are measured and acted on | Sentiment survey plus usage correlation | Mixed |
That last column matters more than it looks. Microsoft's model describes its highest culture-pillar maturity level as one where adoption, confidence and sentiment are measured and acted on together, not where sentiment alone stands in for adoption. Keeping self-reported and system-of-record columns visibly separate is what keeps a finished template honest rather than optimistic.
Sources: Agentic AI maturity model - Organization and culture | Microsoft Learn
Where templates quietly go wrong
Even a well-built template breaks in predictable ways. Run this troubleshooting list before the results reach the board.
One pitfall worth naming on its own: a July 2026 newsletter post by an independent practitioner argues that a maturity level should be read from what agents and systems actually run in production, not from questionnaire answers alone, under the author's own proprietary scale.
We're skeptical of any maturity score built entirely from questionnaire answers, however carefully it's worded, because a self-rating can rise without a single new capability going live.
Sources: Agentic AI maturity model - Organization and culture | Microsoft Learn, AI Maturity Assessment: Turn Scores Into Decisions
Where a predefined survey fills the gaps a spreadsheet can't
System logs can fill in your usage pillar. They can't self-populate perceived skills, culture, barriers or training effectiveness; for those, you need people to answer directly.
This is the gap AI Adoption Score is built to close: it runs nine predefined survey types covering exactly those categories, so you're not writing survey questions from scratch to fill the self-reported half of your template.
Its results map every person, team and department onto one of five maturity levels across skills, usage, culture, impact, vision and governance, which lines up neatly with the pillar structure your template already uses.
Keep one thing clearly labeled when you plug survey results in: perceived productivity is a self-report, not a measured causal effect. A 2026 presentation at QCon AI, as reported on InfoQ, cited DORA's 2025 study finding that individual effectiveness with AI rose while software delivery throughput did not meaningfully change.
“increasing the number of tokens we spend or code that goes through the system might not necessarily translate into higher throughput”
— Lizzie Matusov, Co-founder and CEO, Quotient · Source: The Five Stages of AI Maturity in Engineering Organizations - Where and Why Teams Get Stuck - InfoQ
We'd pair the two rather than pick one: system logs for usage and governance, a predefined survey for skills, culture and barriers, both columns labeled for what they actually are.
Sources: The Five Stages of AI Maturity in Engineering Organizations - Where and Why Teams Get Stuck - InfoQ
Frequently asked questions
Who should own the AI maturity assessment template once it's built?
Each pillar should have its own named owner, not one person owning the whole template. A usage pillar might sit with IT operations, a culture pillar with L&D, and a governance pillar with security or compliance, so the owner closest to the evidence is accountable for the score.
How often should you reassess maturity once the template exists?
Revisit it on a fixed schedule rather than once and done, since usage and incidents accumulate over time. Microsoft's own how-to guidance for its maturity model recommends periodic reassessment without naming a specific interval, so pick a cadence, such as quarterly, that matches how fast your AI usage is changing.
Can I use this template across teams using different AI tools, not just Microsoft Copilot?
Yes, but adapt the pillar structure first. Microsoft's agentic AI adoption maturity model is built around its own Copilot, Copilot Studio and Foundry ecosystem, so its specific level descriptors may need rewriting for other platforms even though the five-pillar, evidence-based approach still applies.
Does a higher maturity score prove AI is actually improving productivity?
A maturity score shows capability and adoption across pillars such as usage, skills and governance, not a productivity number by itself. For impact claims, weigh the evidence behind each pillar: production usage logs and incident records carry more weight than a single survey item on perceived output.
Is a single overall AI maturity number ever useful?
It's useful for a one-line board update, but not for deciding where to invest next. Keep pillar-level scores as the working document and only summarize them into one number for an audience that won't act on the detail anyway.
Sources: How to use the agentic AI adoption maturity model | Microsoft Learn, Introduction to the Agentic AI adoption maturity model | Microsoft Learn
