When course surveys flag a problem but not why

A course evaluation reports that students found an audit analytics workshop confusing. The rating is clear, but the reason is not. Was the software unfamiliar? Were the instructions unclear? Did students lack the necessary spreadsheet skills? Or did the technology distract them from the audit objective?

An anonymous survey may identify dissatisfaction without giving a course leader enough information to respond. Free-text comments can help, but they may be sparse, contradictory or too general to guide a change.

A study in Perspectives on Medical Education examines whether student-led focus groups can add some of that missing context. The evidence comes from one US medical school, not accounting education, and the research explored stakeholder perceptions rather than improvements in courses or student learning. Its most useful contribution for accounting educators is therefore a design possibility: retain the survey, then use a small facilitated discussion when a result needs explanation.

Discussion can reveal what a rating conceals

The medical school introduced eight one-hour focus groups across three preclerkship courses, taught before the clinical phase of medical training. Ten randomly selected students were invited to each group, and trained students facilitated the discussions without teaching staff present. Sessions took place every other week. Facilitators later prepared reports of two to four pages for course directors and the programme evaluation committee.

In total, 42 of the 80 invitations resulted in attendance. Individual groups contained between two and seven students. The researchers subsequently interviewed 17 participating students, five course directors and three administrators, then analysed the interviews for recurring themes.

Students commonly described the discussions as manageable and productive. Hearing peers agree or disagree sometimes prompted them to remember another experience or refine their own view. Some also contrasted this reflective discussion with the routine completion of numerous online evaluation forms.

Course directors and administrators said the reports frequently contained themes they recognised from surveys and existing student-representative channels. The reports could also add explanations and, in some cases, possible solutions. Administrators particularly valued concise narrative context when reviewing courses they did not know closely.

This perceived similarity was not tested through a systematic comparison of survey and focus-group content. Nor did the study establish that the groups generated more accurate or actionable feedback. What it does show is why participants found discussion useful: a rating can identify a problem, while conversation can explore how students experienced it and why their experiences differed.

For an accounting programme, that distinction could matter when a broad result hides several possible causes. Suppose students report that an intermediate accounting course moves too quickly. A targeted discussion might reveal that the issue is concentrated in particular topics, arises from assumed prior knowledge, or reflects the sequencing of worked examples and independent practice. Those explanations would imply different responses.

The focus group should not be asked to decide what the programme must do. Its role is to help course leaders understand the problem more clearly.

Depth comes at the cost of coverage

The same features that made the medical school discussions useful also created their central weakness. A small group allowed each participant time to speak, but it could not represent the full range of student experience. Almost half the invitations did not result in attendance, and some sessions had only two participants.

Faculty participants worried that an unusually vocal or dissatisfied group could exert disproportionate influence. Peer discussion can bring out disagreement, but it can also amplify a view or discourage someone from challenging an emerging consensus. The study did not measure whether either occurred.

The sessions were also not anonymous. Some students associated the absence of faculty with greater candour and described the environment as safe and nonjudgmental. However, participants remained visible to their peers and to the student facilitator. In a small accounting cohort, where students know one another and may work together repeatedly, that distinction could be especially important.

Anonymous surveys therefore retain functions that a focus group cannot reproduce. They can reach more students, generate comparable quantitative results, collect comments on individual teaching activities and provide a private route for sensitive concerns. Focus groups offer something different: interaction, explanation and the opportunity to explore possible responses.

The choice is not between breadth and depth. A programme can use each method for the purpose it serves best.

Use a focus group to investigate a defined question

The most promising use suggested by the study is selective rather than routine. Faculty participants saw particular value in using a group to investigate a known course problem, a proposed change or an issue raised through another feedback channel.

An accounting course leader might begin with a survey finding such as: “Students valued the audit simulation overall, but many reported that the final stage was difficult to follow.” A student facilitator could then explore several questions:

• Which part of the final stage created difficulty?
• Did students experience the problem in the same way?
• What knowledge or instructions did the task appear to assume?
• What helped some students make progress?
• Which change might address the difficulty without removing the intended challenge?

These questions seek examples and variation rather than a quick consensus. Asking “Would another demonstration solve the problem?” too early may direct the group towards the lecturer’s preferred answer. Asking first where and why students became stuck leaves room for other explanations.

Facilitation matters here. The article reports that facilitators were trained, but the main paper does not give enough detail to reproduce that training as an established model. A local pilot would therefore need its own careful preparation.

At minimum, a student facilitator could be briefed to explain the purpose of the session, avoid promising that every suggestion will be adopted, invite alternative views and prevent one participant from dominating. They should also make clear that comments will be summarised without names but that the discussion itself is not anonymous.

The report should preserve disagreement rather than manufacture a class view. A simple structure might separate:

1. concerns raised by several participants;
2. different experiences or explanations;
3. isolated views that may still warrant attention;
4. possible changes suggested by participants;
5. questions that remain unresolved.

This would help a course leader distinguish a recurring concern from a memorable comment. It would not make the group representative, but it would make the limits of the evidence more visible.

Close the feedback loop without overpromising

Some interviewed students wanted more communication about how faculty responded to feedback.

After considering a focus-group report alongside survey results and other relevant evidence, a course leader could provide a short response under three headings:

• What we heard: the issue and any important differences in student experience.
• What we will change: a specific adjustment and when it will occur.
• What we will retain: an element that serves an important learning purpose, with a brief explanation.

There may also be issues requiring further investigation rather than an immediate decision. Saying so is more credible than implying that every concern has a simple solution.

Closing the loop does not demonstrate that the eventual change will work. It does, however, show students that their contribution entered a real decision process. A programme can then examine whether the adjustment addressed the original problem in the next delivery.

Treat the first group as a pilot

The medical education study offers favourable perceptions from an early, single-site initiative, not evidence of changes to teaching or student outcomes. Voluntary participation and researcher involvement also limit what can be inferred, and sustained participation remains unknown.

Transfer to accounting education is plausible because both settings need to interpret student feedback and make course decisions. It is not assured. Cohort size, staff-student relationships, evaluation requirements and available resources may produce different results.

A modest accounting pilot could therefore start with one course and one well-defined question. Its evaluation could ask:

• Who attended, and whose perspectives may be missing?
• Did participants feel able to disagree?
• Did minority views survive the reporting process?
• How useful did course leaders find the report?
• How much work did the process require?

The resulting report should sit beside anonymous survey evidence, not above it. A small discussion is not a representative vote. Used selectively, however, it may help a course team move from knowing that students encountered a problem to understanding what kind of problem it might be.