What is AI fluency? How to test it when hiring

AI fluency definition and meaning

AI is the fastest, most confident junior colleague most people have ever had. AI fluency is how well you manage it: what you hand it, how you brief it, whether you notice when it is wrong, and what you let go out with your name on it.

Stanford HAI’s definition of AI fluency

Defines AI fluency as “the capacity to collaborate productively and responsibly with AI systems”. Read it on hai.stanford.edu

Anthropic’s AI Fluency Framework

By Rick Dakan and Joseph Feller with Anthropic. It names four competencies, the 4Ds: Delegation, Description, Discernment and Diligence. Read it on Anthropic Academy

What is AI literacy, and how is AI fluency different?

The two words are often used for the same thing. The difference matters when you test for it: one is about knowing, the other about doing.

	AI literacy	AI fluency
The question	Do they know what AI is and what can go wrong with it?	Can they get good, correct work done with AI?
What it covers	How AI tools work, what they are good and bad at, privacy and bias	Briefing the AI, checking what it gives back, fixing it, and handing over work that is right
How it is usually tested	Questions with a right answer	A task, done with an AI tool, judged on how they worked and what they handed in
Who needs it	Everyone who uses AI at work	Everyone whose job is done with AI, in any role

AI fluency skills: the six we measure

The same six skills in every role. Next to each, the closest of Anthropic’s 4Ds; that match is our reading, not theirs.

Skill	Strong looks like	Closest of the 4Ds
Understand Do they understand the task before handing it to AI?	Reads the material first. Breaks the problem down. Spots when the brief is wrong.	Delegation
Brief Do they give AI what it needs?	Gives context, limits, examples and what done looks like. Keeps the judgment calls.	Description
Check Do they catch it when AI is wrong?	Tests against the source. Catches the made-up fact and the wrong number.	Discernment
Correct Do they fix it when it isn’t working?	Says what is wrong and why. Changes approach early.	Discernment and Description
Sign off Is what they hand over ready to send?	Ready for the reader. Says what was checked and what was not. Keeps confidential data out.	Diligence
Reuse Do they make it repeatable?	Saved prompts, skills, workflows and automations. The next run is faster and the same.	No direct match

Not part of AI fluency, and never scored: how long or how many the prompts are, speed, which AI tool they chose, and polished writing or perfect English.

How we score each skill

How to test AI fluency when hiring

There are three common ways to run an AI fluency test. They measure different things.

A quiz about AI Multiple choice or short answers about how AI works and when to use it. ShowsWhat people know about AI. MissesWhether they can use it. Knowing that AI makes things up is not the same as catching it.
A simulation with a built-in AI A short task inside a test platform, with an AI chat built into it. ShowsHow they prompt and react, in a standard setting that is easy to compare. MissesHow they work in their own tool and setup, on work as long and messy as the job.
A real task in their own AI tool A task from the job, done in the tool they would use at work, with the whole conversation recorded. ShowsHow they understand, brief, check, fix and hand over, on work like the job. MissesIt takes longer: one to three hours of their time, and someone to read and score it.

An AI fluency assessment on real work, in four steps

1Pick a real task from the job.Something they would do in their first month, with the files that come with it. Not a puzzle.
2Build in one place where AI tends to go wrong.A number that has to match a source, a brief with a gap. That is where you see who checks.
3Let them use their own AI tool, and say AI is expected.Claude Code, Codex, Claude Cowork or ChatGPT. People who are unsure whether AI is allowed hide how they work.
4Score the conversation, not only the result.AI makes most finished work look good. The conversation shows who did the thinking and who checked.

How to run an AI-assisted interview

An AI fluency test result, for a marketer

In Lemonsignal, candidates do a real task in their own AI tool. This marketer got the brief: “We launch in Germany next month with €50k. Using our positioning, last year’s UK campaign results and three competitor sites, decide where the money goes, in five lines for the CMO, and write the launch email.” Lemonsignal read six skills from her conversation with the AI, and every level links to the turn it came from.

Sarah MillerGrowth marketing manager · Task: Plan a launch budget · ChatGPT · 3 h 20 min

Checked by Lemonsignal

78%Good match, one open question

Planned on what UK customers cost, not clicks.
Used no number she could not trace to a source.
Checked every claim about competitors before it went out.

Understand Strong Saw that clicks and paying customers point to different channels, and planned on customers. Turn 3
Brief Solid Gave it the UK numbers and the budget, and said to use only those. Turn 5
Check Strong Asked where every number came from before using it, and checked the AI’s claims against the three competitor sites. Turns 9, 21
Correct Partial Said “make it better” four times on the email before saying what was wrong. Turns 14 to 18
Sign off Strong Checked every claim in the email before signing off. Said the German costs are a guess to check after two weeks.
Reuse Solid Did France with the prompt she wrote for Germany.

Skills for the job

A budget tied to real numbersAbove the bar
Choosing the right channelsAt the bar
PositioningAt the bar

For a specific role: a full assessment

One real task for the role, the six skills, and two or three skills for the job, with a first read from Lemonsignal that your team checks.

Tasks by role

Questions about AI fluency

What is AI fluency?

Being able to get good, correct work done with AI: understanding the task, briefing the AI well, catching it when it is wrong, fixing it, and handing over work that is ready for its reader. Stanford HAI calls it the capacity to collaborate productively and responsibly with AI systems.

What is the difference between AI fluency and AI literacy?

AI literacy is knowing what AI is, what it is good at and what can go wrong. AI fluency is using it well on real work. Someone can know that AI makes things up and still miss it when it happens.

Is AI fluency the same as prompt engineering?

No. Writing a good brief for the AI is one part of it. Most of the rest happens around the prompt: understanding the task first, checking what comes back, and deciding what is ready to hand over.

How do you test AI fluency in an interview?

Give a real task from the job, let candidates use their own AI tool, and read their conversation with the AI. Then ask about one moment in it. See our guide to AI-assisted interviews for the steps.

Can you test AI fluency for non-technical roles?

Yes. A marketer can plan a launch budget with ChatGPT, an analyst can explain a revenue drop, anyone can summarise a long document. The six skills are the same in every role; the task changes.

Does AI score Lemonsignal’s AI fluency test?

Lemonsignal reads the whole conversation, marks the moments that matter and gives a first read. Every level links to the moment it came from. Your hiring team watches the session, checks the read, adds comments and decides.

What is AI fluency? And how to test it when hiring.