Let the Robot Get It Wrong
A make-or-buy assignment designed to turn ChatGPT from a cheating threat into a critical-thinking workout.
Early signal Single study, n = 39 · 45 min prep · 2nd year+
About this evidence
- Evidence strength
- Early signal
- Source type
- Single study · n = 39
- Design
- Teaching framework + student survey
- What we read
- Full text
- Works for
- Tutorials · large classes in pairs
- Time to try
- 45 min prep
- Level
- 2nd year+
How many of us have spent the past two years playing whack-a-mole with AI-written assignments? Denise Wiseman and Deanna Foster suggest a different game. Instead of hiding the chatbot, give it the problem. Then give your students the job of checking its answer, like an auditor ticking and bashing a dodgy working paper.
What they did
Their framework, published in the Journal of Accounting Education, puts students "in the role of practicing cost accountants." In a classic make-or-buy decision scenario, students work with AI-generated solutions. They independently verify the calculations and assess the reasoning behind the recommendations.
Two techniques do the heavy lifting. The first is reperformance: redo the numbers yourself. The second is critical error review: work out where the AI's analysis went wrong, and why. The authors describe this "dual engagement" (AI-driven analysis alongside manual recalculation) as a way to build a fuller understanding of accounting principles and say it "cultivates students' ability to critically evaluate AI outputs."
What they found
The assignment was run in an undergraduate Cost Accounting course over two spring semesters (2024 and 2025) at a small private university in the north-eastern US. Thirty-nine students evaluated AI-generated analyses using an AI tool of their choice (the authors focus on ChatGPT). In the survey, students reported strong engagement and rated their learning highly: mean ratings ranged from 4.28 to 4.56 on a 5-point scale ("an engaging learning experience": 4.51).
The reality check: this is a small study (39 students at one small institution), and the evidence comes from student survey ratings, not a controlled comparison of exam marks. The authors say themselves that learning outcomes and transfer weren't measured objectively. Treat it as a well-designed recipe with happy early taste-testers rather than proof that it beats your current approach.
The clever part is that students have to explain why a number is right or wrong, which is the professional judgement we keep telling employers we're developing. The AI simply supplies a convincing first draft to argue with. It costs almost nothing to try, and it brings AI into your module without pretending it doesn't exist.
Try this in class next week
- Pick one decision topic you already teach (make-or-buy, special order, limiting factor) and ask a chatbot to solve it before class.
- Hunt for a juicy answer. Try a few prompts until you get a plausible-looking answer with a flaw, such as a sunk or unavoidable fixed cost treated as relevant. (Or use our printable one, below.)
- Reperform first. In pairs, students redo every calculation in Excel or on paper before they look at the AI's working.
- Red-pen review. Students annotate the AI answer: "calc error", "unsupported assumption", "missing qualitative factor".
- Sign-off memo. Finish with three lines: "Would I sign off on this recommendation? Why or why not?"
- Keep it formative. Grade the quality of the critique, not just whether students found the "right" number.
These suggestions are ours, not the authors'.
Printable: the flawed AI answer exercise + critique rubric →
After you try it, ask yourself…
- Which AI error did most pairs miss, and what does that tell you about where their understanding is thin?
- Did students argue about the qualitative factors, or only the numbers?
- Would you grade the critique next time, or keep it formative?
Source card
Wiseman, D., & Foster, D. (2025). Bridging AI with cost accounting education: A pedagogical framework using reperformance and critical error review. Journal of Accounting Education, 72, 100988. Read the paper (doi.org) · What we read: full text.
Get next week's ideas in your inbox.
Four evidence-based reads for accounting lecturers, every week. Confirm by email; unsubscribe any time. Prefer feeds? RSS.