Skip to content
Get started
Use cases · Engineering

AI skills test for software engineers, on real engineering work.

A good engineer with AI still owns the work. They find the cause before they let the AI write code, and they prove the change works with a test.

Get started See what you get

Try it free on 3 candidates.

Who the software engineer AI skills test is for

  • Software engineers
  • Backend engineers
  • Full-stack engineers
  • Product engineers

People who build and look after the services behind a software product: APIs, payments, integrations. They build features, fix bugs in code they did not write, and ship changes someone else can review.

LevelJunior to staff

Hiring frontend, mobile or platform engineers?Bring your own task

An example AI skills task for software engineers

One of our engineering tasks. Your dashboard has the rest, and you can bring your own.

The brief

Fix a double charge

Customers are being charged twice. Find out why in the payment service and fix it, without breaking anything else.

Then fix the same problem in the refunds service.

Files
  • The payment service code
  • Logs from three double charges
  • The retry settings
Time
  • Deadline Fri 24 Oct
Work in
  • Claude Code
  • Codex
  • Where AI goes wrongEvery task has a few spots where AI typically gets it wrong. That’s how you see who checks the work.
  • The part that repeatsAfter the payment service, they fix the same bug in the refunds service. You see whether they reuse what they built the first time, or start again by hand.
  • TimeYou set it: a deadline, a time limit, or both. We recommend a deadline and no time limit, so they work the way they would in the job.
  • Where they workClaude Code or Codex, on their own computer or in the browser.
  • All tasks

The AI skills we look for in software engineers

Plus the skills for the job Read from the same task.
  • Finding the root cause
  • Proving a fix with a test
  • Code others can maintain

What you get for each engineering candidate

Dana OkaforBackend engineer · Task: Fix a double charge · Claude Code · 2 h 40 min
Checked by Lemonsignal
89%Strong match
  • Found the cause before letting the AI write a fix.
  • Tested the AI’s fix instead of trusting it.
  • Fixed the second service with what she built for the first.
  • Understand Strong Had it reproduce the double charge, and followed one payment through the code before any fix. Turn 2
  • Brief Solid Gave it the logs and the files that matter, and said what it must not change. Turn 4
  • Check Strong Ran the tests on the AI’s first fix and turned it down. Proved her own fix with a test. Turns 12, 19
  • Correct Strong Told it why the first fix was wrong and where to look instead. Turn 13
  • Sign off Solid A small change with its test, and a note on what she did not test.
  • Reuse Strong Saved the steps as a skill and used it to fix the refunds service.
Skills for the job
  • Finding the root causeAbove the bar
  • Proving a fix with a testAbove the bar
  • Code others can maintainAt the bar
How we score AI skills in engineering job ads, 2026

Questions about testing software engineers’ AI skills

Can engineers use Claude Code or Codex in their own setup?

Yes. They work on their own computer with their own settings, skills and saved prompts, or in our browser sandbox if they would rather not install anything.

Is this a coding test?

No. It is a real bug in a real codebase, and they are meant to use AI to fix it. We score how they work with the AI as well as the fix itself.

Which languages do you support?

The task we have today is a payment service. If you need another stack, bring your own task and we set it up with you.

30 Interview questions to see who’s good with AIStart with the engineering questions, with what a good answer sounds like.

Test how well software engineers use AI.

Get started

Try it free on 3 candidates.