AustLII Home | Databases | WorldLII | Search | Feedback

Legal Education Digest

Legal Education Digest
You are here:  AustLII >> Databases >> Legal Education Digest >> 2012 >> [2012] LegEdDig 34

Database Search | Name Search | Recent Articles | Noteup | LawCite | Author Info | Download | Help

Butler, D --- "Closing the loop 21st century style: providing feedback on written assessment via MP3 recordings" [2012] LegEdDig 34; (2012) 20(3) Legal Education Digest 5


Closing the loop 21st century style: providing feedback on written assessment
via MP3 recordings

D Butler

Journal of the Australasian Law Teachers Association, Vol 4, 2011, pp 99-108

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality.

Technology offers an effective and efficient means by which personalised feedback may be provided to students.

In a much-cited study, John Hattie conducted a comprehensive review of 87 meta-analyses of studies on what makes a difference to student achievement finding that the most powerful single influence was feedback. Feedback enables students to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. It facilitates the development of self-assessment or reflection in learning, can deliver information to students about their learning and encourage positive motivational beliefs and self-esteem. The archetype of personalised feedback on written assessment may be seen as that afforded at Oxford or Cambridge University where the student wrote an essay a week and read it out to his or her tutor in a one-to-one tutorial, who then gave immediate and detailed oral feedback on the students’ understanding as shown in the essay.

However, in most if not all Australian universities the realities of high staff–student ratios and time pressures make this form of assessment and feedback an unattainable dream. Instead, in the modern higher education system, ‘assessment sometimes appears to be, at one and the same time, enormously expensive, disliked by both students and teachers, and largely ineffective in supporting learning’.

Even where the academic makes the effort to provide detailed feedback, studies of what students do with that feedback offer little encouragement. Feedback is frequently not read at all or not properly understood by students even if they do read it.

By contrast, less conscientious teachers may be inclined to spend little time on the marking exercise, perhaps venturing no further than the occasional tick or cross or scant comment, such as ‘good work’. Further, whilst greater use of criterion-referenced assessment (CRA) has enabled an improvement in the extent of feedback being provided to students, the completion of a CRA grid alone is not an adequate substitute for personalised feedback. Similarly, generic feedback on an exercise can reduce the extent of ownership which students take over the feedback they receive, even where that generic feedback is detailed.

Technology offers an effective and efficient means by which personalised feedback may be provided to students. In 2010, the 112 final-year students studying the elective, Media Law, at the Queensland University of Technology were required as part of their assessment to complete a 2,000 word written assignment in groups of two or three. In lieu of written comments, feedback on most of these assignments was provided by way of a CRA sheet and oral comments recorded in MP3 format using free Audacity software. To facilitate a comparison between this methodology and traditional written feedback in terms of workload from an academic’s perspective, a control group of assignments received written comments in lieu of MP3 feedback.

Audacity offers both basic recording features, of the kind that most academics will readily recognise such as record, pause and stop buttons. It also enables more advanced functions such as audio mixing.

In each case, the marking process involved reading the hard copy assignment and making only minimal markings on the pages (such as circling words or placing lines or squiggles besides passages), followed by completion of a CRA sheet. Comments and feedback were then recorded, expanding and explaining the minimal markings on the pages. The recording was then saved to the marker’s hard drive under the joint surnames of the students involved.

Once this process was completed for all assignments, a generic email message was composed. Using the email facility in the unit’s Blackboard website, the generic email message was copy and pasted, personally addressed and sent to each student with the personalised MP3 feedback as an attachment.

Reading the assignments in the control group, making written comments and completing the CRA sheet took on average 22:26 minutes. By comparison, the average time for reading the assignments in the survey group, completing the CRA sheet and recording the MP3 feedback was 18 minutes. The average length of the MP3 feedback was 7:20 minutes. Emailing the MP3 files via the unit’s Blackboard site took on average 90 seconds, making a total of 19:30 minutes for the MP3 feedback methodology.

Part of the generic email message asked those students who received MP3 feedback to follow a link to an online survey instrument. This survey comprised both questions using a 6-point Likert scale (5 representing ‘Strongly Agree’, 1 representing ‘Strongly Disagree’, and 0 representing ‘Not Applicable’) and open-ended questions. There were 38 responses to the survey, representing a 61 per cent response rate for those students who received MP3 feedback.

Students were first asked to address two preliminary points concerning their use of the MP3 feedback. First, students were asked to respond to the statement: ‘I experienced no technical problems in playing the MP3 feedback file.’ A total of 92 per cent of respondents had no difficulty receiving and listening to the feedback provided by MP3 file.

Students were also asked to advise how many times they had listened to the MP3 feedback. Half the students listen to the MP3 feedback more than once, while the other half only listened to it only one time.

Next, students were asked a series of questions regarding their perceptions of MP3 recordings and their value as a means of providing feedback on written assessment. First, they were asked to respond to the statement: ‘A recorded MP3 feedback file was an effective way to receive feedback on my written work.’ One hundred per cent of respondents considered MP3 recording to be an effective way of receiving feedback on their work.

Students were also asked to respond to the statement: ‘The MP3 feedback helped me to understand the mark I received for my work.’ Again, all respondents thought that the MP3 feedback helped them to understand why they got the mark that they were given. Students were then asked to compare this method of receiving feedback to traditional methods by responding to the question: ‘A recorded MP3 feedback file was a more effective way to receive feedback on my written work than written comments.’ A total of 95 per cent of the students preferred MP3 recordings over written comments as a more effective means of receiving feedback.

It would seem self-evident that an academic can speak faster than he or she can type or write. That translates to an ability to provide more substantial feedback in the same amount of time. Email, including that forming part of the group/collaborate tool in a learning management system like Blackboard, enables the audio file to be quickly and efficiently delivered to the student, who not only is spared the challenge of deciphering the academic’s handwriting but also has the advantage of hearing the way the words are spoken: ‘I found the MP3 feedback very helpful as it allows us as students to really find out what the marker thinks of our work as we can hear his tone of voice’.

However, while audio feedback adds a dimension that is more personal and rich in terms of tone, it can also seem more real and potentially upsetting. It is important to remain constructive and supportive, taking care both in terms of the choice of words and the way in which they are expressed.

The audio files can also be replayed to aid comprehension. Students often face interpretive challenges when trying to capitalise on written feedback. For example, if an academic writes the comment ‘this does not follow logically from what goes before’, a student who lacks the tacit knowledge necessary to identify that aspect of their work to which the feedback refers may not be appreciate the problem with the logic and as a consequence take no action to address the failure in future work. A proper explanation of why the logic does not follow may require a paragraph of explanation which the academic may not be able to afford the time to compose nor see as necessary. By comparison, audio feedback affords the academic the opportunity to provide a more detailed explanation of the comment that enables the student to make the necessary connections to properly understand the point being made.

Audio files not only facilitate the provision of more detailed explanations of where students are in error, but also where students have done well. The expanded opportunity to both compliment good work and to seek to inspire improvement can make the marking process a satisfying one for the academic.

There are two sides to feedback being manageable. From an academic’s perspective, designing and delivering feedback can be an exercise that can consume a vast amount of time. But also from the point of view of students, too much feedback can result in an obfuscation of the message, making it difficult to sort out the important feedback from the routine feedback and reducing their opportunity to benefit from the feedback that they need most. In the Media Law trial, there was no necessary correlation between the length of the recording and the size of the mark awarded. For example, the longest audio file was 11:55 minutes long and was for an assignment that was awarded 9 out of 20. By contrast, the second longest recording, which was 11:28 minutes long, was for an assignment which received 19 out of 20, the equal highest mark.

The traditional approach of providing written feedback on written assignments can be an inefficient and ineffective exercise. It can consume a large amount of an academic’s time and, no matter how detailed, may not even be read by students who may have been disappointed by the extent or quality of feedback on past assignments. Even where a student takes an interest in the feedback that has been provided there may be difficulties properly understanding the nuances of the written words and making the connections necessary for an improvement in their performance.

By contrast, providing personalised MP3 feedback using the freely available and free-to use Audacity program is capable of achieving the goals of effective feedback identified in the literature. It allows feedback to be personally directed, complimenting students on the strength of their work while explaining any deficiencies. It enables academics to identify in detail what would have improved their submitted work and what needs to be done next time. It allows the academic to be constructive and supportive, not only by the words used but by the way they are spoken. And since most academics speak faster than they write or type, it can achieve those ends in no more time than traditional approaches.

Technology has transformed, and is transforming, legal education and higher education in the 21st century. Providing comments by MP3 audio file is a further step in this evolution that is capable of delivering personalised and detailed feedback in a more effective and efficient manner than possible through traditional written comments.


AustLII: Copyright Policy | Disclaimers | Privacy Policy | Feedback
URL: http://www.austlii.edu.au/au/journals/LegEdDig/2012/34.html