Anonymity settings aren't enough for honest answers
Imagine your Chief AI Officer sends out an anonymous employee survey asking who's using AI tools without IT's sign-off. Three weeks later the results come back showing almost no shadow use, no skill gaps, and barely any worry about job security. Nobody on the leadership team quite believes it.
The common assumption is that flipping on an "anonymous mode" setting fixes this. That helps, but it only answers part of the question. Nielsen Norman Group's guidance distinguishes anonymity, where a response can never be traced to a name, from confidentiality, which only limits who can view the data.
We think a checkbox labeled anonymous promises more than it delivers. The real protection comes from how you word each question, where you place it in the survey, and how small a group you're willing to report results for, not from a single toggle.
Sources: Handling Sensitive Questions in Surveys and Screeners - NN/G
Three questions your staff won't answer straight
Three topics make this survey harder to word than a routine engagement check: unapproved AI tool use, honest skill and confidence gaps, and worry about job security. Each one invites a polished answer instead of a true one.
Qualtrics' own guidance on survey methodology, published in 2026, warns that on sensitive topics respondents tend to overreport behavior that looks good and underreport behavior that doesn't. Expect the same pull here: almost nobody wants to admit, in writing, that they've been pasting client data into an unapproved chatbot.
Pew Research Center's 2026 survey of adults across 37 countries found that in 34 of those countries, people lean toward expecting AI to mean fewer jobs overall rather than more. That's public sentiment, not your workforce, but it's the baseline anxiety your employees likely bring into the survey.
We're skeptical of leaning on national mood figures to predict how your own team feels about its own roles. A country-level forecast and a specific engineer's fear about her specific job are different questions entirely.
Sources: What is Response Bias and How Can You Avoid It? - Qualtrics, Globally, More People Expect AI to Cause Job Loss Than Growth | Pew Research Center
Suggested anonymous employee survey questions to adapt
The question set below gives you a plain-language starting point, not a validated instrument: nobody has tested this exact anonymous employee survey template against real disclosure rates. Adjust the wording, tasks and tools to match your own organization before sending anything.
Send this question set to every employee who has access to AI tools, not just the team you suspect of shadow use. Limiting it to one department makes answers easier to trace back to a person, which defeats the point of running it anonymously.
| Topic | Suggested question | Response options | Interpretation limit |
|---|---|---|---|
| Unapproved AI tool use | In the past month, how often have you used an AI tool that isn't officially approved for work tasks? | Never; 1-2 times; 3-9 times; 10 or more times; Prefer not to say | Shows reported frequency, not verified tool logs |
| Skill and confidence | How confident are you using AI to draft a first version of a document? | Not at all confident to Extremely confident, five-point scale | Self-rated on one specific task, not a tested skill level |
| Job-security worry | Many employees have questions about how AI will affect their role. How concerned are you that AI could reduce the need for your current job in the next two years? | Not at all concerned to Extremely concerned, plus Prefer not to say | Captures reported worry, not an actual risk assessment |
Notice the shadow-use question asks about frequency, not a flat yes or no. Nielsen Norman Group's guidance points to this pattern in youth risk-behavior surveys: framing a sensitive act as 'how often' rather than 'did you' tends to produce more honest reporting than a binary choice.
We'd rather run five well-placed questions than fifteen. Every extra sensitive item is one more reason for someone to close the tab, or to start answering carelessly just to get through it.
Sources: Handling Sensitive Questions in Surveys and Screeners - NN/G
Response scales and reporting rules that protect anonymity
Wording is half the job; scale design and reporting rules are the other half. Nielsen Norman Group's guidance notes that offering ranges instead of asking for an exact figure feels less exposing to someone answering a sensitive question.
Sequencing matters too. The same guidance warns that placing a sensitive question too early in a survey raises the chance someone abandons it before finishing, so save the shadow-use and job-worry items for after a few easier warm-up questions.
We'd also suppress any breakdown smaller than roughly ten respondents. A single anxious reply in a four-person team is too easy to trace back to one desk, whatever the tool's anonymity setting claims.
Sources: Handling Sensitive Questions in Surveys and Screeners - NN/G
What these answers can and can't prove
Every answer above is a self-report: what someone says about their own use, confidence or worry, not a verified log of tool activity or a measured effect on anyone's job. Microsoft's own 2026 analysis of its Work Trend Index data makes a related point: a link it found between workplace factors and reported AI impact is a statistical association, not a causal effect.
For context, Gallup's workforce studies found that roughly 19% of U.S. workers, as of the first quarter of 2026, said their job was somewhat or very likely to be eliminated by AI. Writing about that data, Gallup senior researcher Christos Makridis points to a tension at the center of adoption itself.
“The same tool that speeds up a task can also raise the question of whether the person doing the task is becoming redundant.”
— Christos Makridis, Content Senior Researcher, Gallup · Source: Using AI More Does Not Reassure Workers, Managers Do
McKinsey's 2026 survey of more than 10,000 senior executives across 15 countries found that 86% of leaders feel their organizations aren't well prepared to adopt AI in day-to-day operations. If leadership itself reports that much uncertainty, don't expect employee answers on skill confidence to come back any cleaner.
Sources: 2026 Work Trend Index report: Agents, human agency, and opportunity, https://www.mckinsey.com/~/media/mckinsey/business%20functions/people%20and%20organizational%20performance/our%20insights/the%20state%20of%20organizations/2026/the-state-of-organizations-2026.pdf
Ask once, then ask again with the same words
Run this anonymous employee survey once and you get a snapshot of today's worry and shadow use. Run it again next quarter or next half, with the same wording and the same reporting thresholds, and you get a trend you can actually act on.
AI Adoption Score's AI Adoption Barriers Survey is built around the same instinct: find the real reasons a team isn't using AI more, not just whether people clicked through a training module.
Word it carefully, report it carefully, and the answers will tell you where to spend your next training budget. Skip the care, and you'll just have a prettier version of the silence you started with.
Frequently asked questions
Who should see the raw, unaggregated survey responses?
As few people as possible, ideally only the survey administrator. Keep raw data separate from any dashboard managers see, and share results only once aggregated above your minimum reportable group size. This is editorial practice drawn from general sensitive-survey design, not a documented standard from the cited research.
How often should you repeat an anonymous employee survey on AI use?
Quarterly or twice a year works for most teams, but keep the exact wording frozen across cycles once you start tracking. If you must revise a question, run the old and new wording side by side for one cycle first, so a sharp swing in results reads as a real shift rather than an artifact of changed phrasing.
Should a manager see their own team's results separately?
Only if the team is large enough to protect individual answers, typically the same minimum group size you use for every other breakdown. Below that, roll the team into a larger unit, such as a department, before sharing anything back.
Does a longer survey produce more honest answers?
No, usually the opposite. A shorter, carefully sequenced set of questions holds attention and respect for the respondent's time better than a long form, and fewer sensitive items means fewer chances for someone to disengage partway through.
