Butler, Des; Wiseman, Leanne --- "Viva the Viva: Oral Examinations in Contract Law" [1993] LegEdRev 14; (1993) 4(2) Legal Education Review 331

	Home \| Databases \| WorldLII \| Search \| Feedback Legal Education Review

In light of the dependence of lawyers upon the spoken word, whether it be in presenting argument in court, conducting negotiations, or conveying advice to clients, it is perhaps surprising that traditionally law schools assess students primarily on their written rather than their oral expression and presentation. Commonly the emphasis is upon methods of written expression such as examinations and research assignments. Assessment of oral expression, if present at all, is usually through participation in a moot as a barrister or presentations to or participation in class. Oral examinations (“viva voce”) are seldom an option and are reserved, if at all, to the role of being a supplement to some other form of assessment.
At the Queensland University of Technology Law Faculty, assessment of the 300 first year students enrolled in the unit Contract Law prior to 1993 comprised of two examinations (a mid year examination worth 30% of the total marks and an end of year examination worth 50% of the total marks) and a written 2,000 word assignment (worth 20% of the marks). Students could choose to participate in the Faculty’s mooting programme as either a solicitor or barrister in lieu of submission of a written assignment, although the number of moots in Contract Law was restricted to three. Accordingly, there was a limit of 24 places for mooters in Contract Law, of which only 12 were available to students wishing to act as counsel.
In 1993 the four members of the Contract Law teaching team decided to provide more opportunity for assessment of students’ oral expression and presentation. In addition to assessing performance in small group tutorials, a formal programme of oral examinations by panels each comprising two academics was introduced as an alternative to written research assignments and moots. Thus, rather than fulfilling merely a secondary role, vivas were presented as a direct alternative to written assessment by way of assignment, and provided a forum in addition to the limited opportunities available in moots in which students could display oral skills.

It was decided that in order to avoid the viva merely serving as an opportunity to regurgitate ideas and material already expressed by the lecturers in large group lectures, the topics for the vivas would be areas that were yet to be handled in the large group setting. Once that policy was adopted, fairness dictated that the students be provided with some direction for their study. Such direction was also required so that the vivas could have some consistency in their structure. Accordingly, two weeks in advance of their viva, each student was supplied with a personally addressed Viva Information Sheet (“VIS”) which advised not only the topic to be examined, but also prescribed readings from the textbook and “primary” and “secondary” case authorities. Students were advised that they were required to know the “primary” cases in depth by exhibiting knowledge and understanding of the relevant facts, principles and outcomes, and the “secondary” cases to the extent of the applicable principles only. The “secondary” cases were stipulated in order to give direction to the student’s reading of the text book. It was stressed to the students, however, that the prescribed readings and case authorities were only a minimum requirement, and that they could read in more depth if they so desired. Thus, while students were provided with guidance, they retained ultimate responsibility for the depth of study they undertook. Students were further advised that they could take any materials into the viva with them, but that the level of reliance on those materials would be taken into account when determining their marks. This approach was adopted to deter students from merely regurgitating matter from their texts and notes.
The areas chosen for examination in Contract Law were the second semester topics of misrepresentation and mistake. These areas are of comparable size and treatment in the prescribed textbook. The prescribed cases in the areas were specifically chosen not only for their importance but also their length. In this way, it was hoped to ensure some fairness between students doing different topics.
It was planned for questions in the viva to be of five levels:

In formulating the five levels of questions, regard was had to Bloom’s Taxonomy1 as applied to the legal sphere.2 All levels involved components of knowledge and comprehension, and in addition the second and third levels addressed analysis, synthesis and evaluation, the fourth level analysis and the fifth level application and evaluation.
The first four levels entailed compilation of “banks” of questions in the two subject areas of mistake and misrepresentation, using the prescribed text — and case readings as the source of reference. The fifth level required the creation of a sufficient number of problems that were short enough to be quickly read by the student in the viva and yet of such detail to permit discussion of the facts and application of principles. The banks of questions, problems and suggested answers were prepared by two members of the teaching team3 and reviewed by all members participating in the viva programme. This ensured the validity of the process, by confirming that the questions were able to be answered from the materials prescribed, and reliability, as a consequence of a common approach to the given answers.4 The banks were of a size to enable random selection of questions by the viva panels, with a view to ensuring that no two vivas were identical. In this way it was hoped to protect the integrity of each viva by minimising or eliminating the risk of any student communicating the content of the examination to fellow students.

The viva programme was scheduled across three days. A limited number of places (32) was made available at stipulated times, some suited to part time students. These times were determined with regard to the existing teaching and other commitments of the individual panel members. Students registered on a first come-first served basis. Prior to the time set aside for registration, students were issued with copies of a statement outlining the rules governing the vivas and were addressed in class as to the way in which the vivas would be conducted.
Each viva was scheduled to last for up to thirty minutes before panels of two members of the Contract Law teaching team. The members of the panels rotated and determined marks by reference to a list of criteria. These criteria highlighted the following areas:

In each case, participants were measured by a demonstration of their ability to express and explain terms, rules, principles, concepts and theories in their own words and in a clear and coherent manner.
Each area was adjudged on a scale from A (very good) to E (very poor).5 Final marks were awarded in a holistic fashion taking into account the criteria. The criteria also enabled moderation of marks at the completion of the viva programme. The validity and reliability of oral examinations in other disciplines have been questioned because of subjectivity and potential bias,6 although other commentators have suggested means by which these difficulties may be overcome.7 The examiners were conscious of the possible risk of favouritism or bias towards particular students and we believe the process of rotation of panel members, marking by reference to a list of set criteria8 and moderation by all members of the teaching team prevented this from occurring.
In view of the tight schedule adopted, it was important to ensure that vivas did not run over their allotted thirty minutes. It was decided to aim at allowing a maximum of five minutes for questions on each of the five levels, with the expectation that some students might progress through the earlier levels at a faster rate than others. To avoid the risk of students becoming “rattled” or “freezing” in the viva if they felt that they were answering questions incorrectly, it was agreed that the panels would be supportive and encouraging in their responses to answers given but would not directly indicate whether the answers given were correct or incorrect.9 However, students were invited to later approach the relevant panel members for individual feedback on their performances. To facilitate this feedback, notes on the participants’ method, manner, strengths and weaknesses were made by the examining panel members during each viva.
All vivas were conducted in the same venue, the body of the Law Faculty’s Moot Court Room. In this way it was hoped to avoid interruptions and distractions such as telephones and visitors, and to lend a sense of formality to the examination proceedings.
Students undertaking vivas were surveyed on two separate occasions. The first occasion was at the time they collected their Viva Information Sheets, and was with respect to students’ perceptions, motivations, and expected outcomes of their viva. The second survey focused upon the students’ reactions to their vivas. This survey was conducted following the completion of the viva programme and before the release of results in order to minimise the risk of distortion of responses, both positive and negative, that could reasonably be expected in a survey held in the immediate aftermath of each viva or after the students had received their marks. Prior to administering the questionnaires, students were assured that their responses would be confidential and would assist in the assessment and improvement of the viva programme. Unfortunately, as a result of that approach, it was not possible to draw strict parallels between individual responses.

Of the 32 students participating in the viva program, 25 responses were received to the first questionnaire.

Students were questioned as to their motivations by being asked to rank from 1 to 3 their most critical responses to the following question:

The results of the students’ first ranked responses to this question are summarised in Table 1 below.

	Possible Responses	Number of Students	Percentage
1	I think I am better at oral expression than written expression	8	32%
2	I think studying for a Viva will help my study for the end of year exam	4	16%
3	The timing of the Viva suited me	4	16%
4	I think a Viva will be good practice for mooting in later years of my studies	2	8%
5	I was not able to do a moot	2	8%
6	I think a Viva will be easier than a written assignment	2	8%
7	I think I can get a better mark doing a Viva than attempting — a written assignment	1	4%
8	I wanted to try a different type of assessment	1	4%
9	The Viva requires me to do more work earlier than I otherwise would have done	1	4%

One in three students gave as their reason for choosing to do a viva the fact that they thought they were better at oral expression than written expression. However, only two students chose as their primary response that they thought a viva would be easier than a written assignment and only one nominated that they thought they could get a better mark doing a viva than attempting a written assignment. Overall, the response “I think studying for a viva will help my study for the end of year exam” was the most popular reason given for choosing to do a viva, with over half the students ranking it either first, second or third. This would seem to reflect the perception that undertaking a viva produces the double benefit of being a form of assessment in itself as well as preparation for the end of year exam.

Students were requested to rank from 1 to 3 their most critical responses to the following question:

The results of students’ number 1 ranked responses to this question are summarised in Table 2.

	Possible Responses	Number of Students	Percentage
1	Nerves during the Viva affecting my performance	8	32%
2	Not knowing whether I have prepared enough	7	28%
3	Nerves before the Viva affecting my performance	3	12%
4	Other	3	12%
5	Not being able to answer questions asked by the panel	2	8%
6	Forgetting the material I have studied	2	8%

It might be suggested that the top three responses, “nerves during the viva affecting my performance”, “not knowing whether I have prepared enough” and “nerves before the viva affecting my performance” are matters not confined solely to oral examinations and might equally be encountered in written examinations. No respondent nominated “having to answer questions by a panel” as being of primary concern, an interesting result in light of the fact that the vivas were the first time students would have encountered such a face-to-face form of assessment. Indeed, this response received the lowest number of nominations as either the first second or third ranked response. Overall, 20 of the 25 respondents nominated “not knowing whether I have prepared enough” as either their first, second or third choice. To some extent, this might be attributed to the fact that as this was the first time vivas had been held students had no opportunity to speak to students who had experienced vivas in previous years. As previously noted, a number of information sessions were conducted for the benefit of potential viva participants in which the content and format of vivas were discussed. In hindsight, it may have been beneficial to have staged a mock viva, involving staff or later year students, to give participants an insight into oral examinations,10 but for various practical reasons this did not prove to be feasible at the time.

Students were asked to identify what they perceived to be the benefits of doing a viva. Among the responses, the main advantages identified were that it gave an opportunity to improve their confidence and ability in expressing themselves verbally, the development of the ability to “think on their feet”, the chance to gain an in-depth knowledge and understanding of a certain area of the law and the assistance it would provide in studying for the end of year exams. Some typical responses were:

Students were then asked to nominate what they perceived as being the disadvantages in doing a viva. The most common response related to nerves, reflecting the results obtained in relation to students’ main concerns (noted above). Other responses related to the perception that there was little chance to correct a mistake before the next question is asked, that vivas required extensive preparation with a detrimental effect on work in other subjects, and that there was more of a chance of being wrong or of receiving a lower mark than in a written assignment. Examples of the responses are as follows:

The final section of the questionnaire asked students to respond to the statement: “I think that doing a viva will improve my understanding of the area to be examined.” The responses to this statement are set out in Table 3 below.

Table 3: I think that doing a Viva will improve my understanding of the area to be examined

These results show overwhelmingly that students perceived that their understanding of the subject area would be improved as a result of doing a viva.

Were student perceptions and expectations realised? Were their concerns well-founded? The second survey addressed a number of different considerations, namely:

Of the 32 students participating in the programme, 23 students responded to this survey.

Students were asked to respond to the question: “I was comfortable during the viva”. The results are summarised in Table 4 below.

Thirteen students, that is 56% of the respondents, felt comfortable during the viva. Of the 8 students that did not feel comfortable, the main reason cited was not knowing what to expect. Nerves and a lack of preparedness were also given as reasons for discomfort.
Students were also asked whether they felt confident during the viva. A summary of their responses is set out in Table 5 below.

Again, 13 students or 56% of the respondents answered that they agreed or strongly agreed that they were confident during the viva. When asked what could have made the students feel more confident, answers included knowing the work better, having no notes and knowing what degree of detail was required in the answers.
In relation to both comfort and confidence, comments were made that students appreciated the efforts of examiners in trying to put students at ease and in their supportive approach.

Did students feel that they had sufficient direction to properly prepare themselves for their vivas? Did they think that the examination was it too long or of the correct length? These issues were addressed in separate questions. Student responses to the statement: “The prescribed readings and cases provided sufficient information to answer the questions asked by the panel in the viva” are set out in Table 6.

Table 6
The prescribed readings and cases provided sufficient information to answer the questions asked by the panel in the viva

Only 6 students thought that the information set out in the Viva Information Sheet was not sufficient, compared to 16 students or 69% of the respondents who thought otherwise. Of those that found difficulty, the major problem that seems to have been encountered was that students found the prescribed textbook sometimes confusing. Students sought to overcome this problem by resorting to more basic textbooks in the area.
Student responses as to the length of time devoted to each viva (30 minutes) are summarised in Tables 7 and 8.

Eighteen students or 79% of the respondents thought that 30 minutes was not too long for each viva. Indeed, one-quarter of the respondents would have preferred for the examination to have been longer than 30 minutes.

Students were asked to indicate what (if anything) they liked best about doing a viva and what (if anything) they liked least about doing a viva. Many of the responses highlighted similar advantages and disadvantages to those identified in the first survey.
In relation to the question regarding what (if anything) students liked best about the viva, responses were many and varied. Students highlighted the fact that the assessment was early in the semester and therefore reduced pressure on their end-of-year studies. They also stated that they learnt and understood the material very well and that that would assist them in preparing for the end-of-year exam, that the study was enjoyable and interesting, and that they gained confidence speaking about the law. Several students referred to the intimate atmosphere, which was said not to intimidate. Among the responses were the following:

In relation to the question what (if anything) the students liked least about doing vivas, two responses predominated: nerves before or during the viva, and not knowing the depth of answer required beforehand or uncertainty concerning the format. Other responses included remembering the answers to questions afterwards, research done that was not displayed because no questions were asked on that particular material and feeling that there was not enough time to answer each question the way the student best knew how.

Students were asked three questions in order to glean their overall impression of vivas. The first question related to whether students believed that the viva had assisted in improving their understanding of the material examined. The results are set out in Table 9.

Table 9
This mode of assessment assisted in improving my understanding of the material examined

Twenty-two students or 96% of the respondents believed that doing a viva improved their understanding of the material covered. Of these 13 or 57% strongly agreed with the proposition. This contrasts with the response to the statement that an assignment on the area would have provided as broad an understanding, the results of which appear in

An assignment on the area examined by the viva would have provided as broad an understanding of the topic

Only 31% of the respondents believed that an assignment would have provided as broad an understanding of the topic as provided by attempting the oral examination. The final question related to whether the students would choose to do a viva in another subject. A summary of the responses is shown in Table 11 below.

Every student that participated in the viva programme indicated that they would choose to do a viva in other subjects. This in itself may be viewed as a enthusiastic endorsement of the concept.

Overall, the members of the Contract Law teaching team were impressed with the viva programme as a means of assessment. In contrast to written assignments, vivas afforded the opportunity to fully explore the depth of understanding of individual students. It was also felt to be a more accurate reflection of the students’ knowledge and understanding since probing follow-up questions could be asked to pursue a line of enquiry. Students also could ask for clarification if the question was not clear to them, reducing the risk of uncertainty in a way not possible with written assessment. From a marking point of view, each member of the team found the programme to be a very satisfying experience.
The different approaches towards the use of notes was instructive. Some students took in no notes at all, some took in notes but made an obvious effort not to refer to them, some relied on their notes to prompt themselves but no more while some resorted to them at every juncture. Naturally the former students made a better impression and were marked accordingly.
As anticipated, some students progressed through the lower levels of questions at a faster rate than others. We found it to be of assistance to have a notional leader for each panel who would indicate the appropriate time to move onto the next level. It proved important to monitor time to ensure that the vivas did not run over time so that the programme stayed on schedule. We found that one panel member should do no more than four 30 minute vivas without a rest break. This can be achieved within in a tight schedule by the rotation of the panel members. While some students indicated that they would have preferred for vivas to last longer than 30 minutes, panel members felt that this was sufficient time in which to gauge the students’ understanding of the areas to be examined, and in the context of a programme catering for over 30 students vivas of a longer duration would have been too demanding for the panel members given their other commitments.
Although there was no complaint from participants, one issue that ought to be addressed is the gender make-up of the panels. As the Contract Law teaching team consisted of two female and two male academics, there were few instances of all female or all male panels.11 However, wherever possible, all female or all male panels should be avoided to exclude any perception of gender bias and to assist in creating a non-intimidating environment.
It might be suggested that it would be beneficial for a Contract Law student representative to be present on each panel. While this may be superficially attractive from a peer assessment point of view, it presents several serious problems in practice. For example, the first participant in the programme will not have the same opportunity as later participants to observe the conduct of a viva before her/his own. More importantly, including a student representative, whether a participant in the programme or not, creates the risk of that student communicating the content of the examination to other students, which impacts upon the integrity of the programme. Also, a student representative may jeopardise the confidentiality of each participant’s performance. We believe that these difficulties, and the perceptions of unfairness they may create, outweigh the possible advantages of the measure.
While the venue for the viva programme, the Faculty’s Moot Court, achieved the intended objectives of commonality and the avoidance of interruptions, on reflection a more appropriate venue may have been beneficial. While the panels did not sit at the bench and instead sat across a table from participants, there was a possibility that students might have been intimidated by the large formal room. However, only one student commented that they were intimidated by the surroundings and indeed, despite the size of the room, several students commented on the intimate atmosphere of the vivas.
Although the responses to the surveys were confidential, it was clear that some of the responses were coloured by the attitudes of particular individuals. For example, it was evident that students who had not done sufficient preparation or who were inclined to be overly nervous provided negative feedback which may not have been justified.
Despite our invitation to provide individualised feedback to all participants, few students availed themselves of this opportunity. Informal discussion with some students indicated that many had formed their own assessment of their performance which tallied with their results and accordingly had not felt it necessary to further discuss their performances with staff. Nevertheless, we think it would be beneficial, in addition to the open invitation, to provide further feedback, either by means of written comments or a general discussion with all participants, which would highlight common strengths and weaknesses. This would assist the viva programme to be not only a summative exercise but also a formative one.

The members of the Contract Law teaching team are committed to encouraging deep approaches to learning by students. Although student approaches to learning were not the focus of the study, it is worth noting some of the issues that arise in this regard.
Do vivas of themselves encourage deep approaches to learning? Initially it may be thought that the mode of assessment would have no impact on how students approach studying, and that approaches to learning would be determined instead by the content being examined. The focus of the viva on the understanding of legal principles and the relationships between those principles would be expected to significantly encourage deep approaches by students.12 However it may be the case that for some students the viva format itself encouraged the search for understanding and relationships. For example, one student noted the advantages of “learning to talk about a subject which is higher knowledge than writing about it.” We hypothesise that oral expression requires a greater degree of “ownership” of ideas than does written expression and that, in preparing for an oral examination, students will perceive a need to make ideas their own through a thorough understanding of them. This could be tested through research which asks students to describe how they set about preparing for the viva. Such research could parallel Tang’s study of the “Effects of Different Assessment Procedures on Tertiary Students’ Approaches to Studying”13 and would be a useful contribution to our understanding of the determinants of student approaches to learning. 14
Do vivas provide a better forum for assessing understanding than other forms of assessment such as written assignment and examinations? Perhaps the critical contribution of the viva is the examiners’ capacity to probe the limits of students’ understanding through supplementary or follow up questions.15 By posing such questions, we found the oral examination particularly effective for measuring levels of understanding and hence of distinguishing between the good, the very good and the excellent student.
There is no doubt that vivas have drawbacks.16 A large amount of time must be invested in the initial selection of the topic areas and the development of the question banks. Vivas are resource intensive in terms of staff time and monopolisation of the central venue. This problem might be addressed in part by scheduling vivas before single examiners rather than panels of two. However, two perceived disadvantages of this measure would be the greater risk of inconsistency in marking and the increased possibility of subjectivity, bias or favouritism.17 The universal feeling among the members of the Contract Law teaching team, though, was that the present investment in staff time was justified when considering the undoubted benefits in providing a unique setting for students to display the full extent of their knowledge and understanding and to develop their skills of oral expression. In addition, it was clear that the vivas achieved the objective of engendering the reading of cases as a means of learning and understanding the law. Perhaps the full benefits received by students are best summed up in the words of one of the participants:

This checklist is a guide only to assist in determining the ability of a student undertaking an oral examination in the Law of Contract. Space has been left on the right hand side to allow for comments to provide feedback to assist the student in improving their performance in the future.

CANDIDATE:	STUDENT NUMBER:
EXAMINERS:
MATTER:		COMMENTS:
Understanding the meaning of particular legal terms and legal expressions	A B C D E
Understanding of relevant legal principles and any possible relationship between principles:	A B C D E
Understanding of the prescribed cases:	A B C D E
Ability to identify legal issues in given fact situations:	A B C D E
Ability to apply legal principles to given fact situations:	A B C D E
MANNER:
Speaks without undue reliance on written notes:	A B C D E
Speaks in clear, coherent manner:	A B C D E
Ability to explain legal principles clearly:	A B C D E
FINAL MARK =

* Lecturers, Faculty of Law, Queensland University of Technology. We gratefully acknowledge the comments and invaluable assistance of Gordon Joughin in the preparation of this paper, particularly in relation to student approaches to learning.
© 1993. (1993) 4 Legal Educ Rev 331.

1 N Gronlund, Stating Objectives for Classroom Instruction. (New York: Macmillan, 1985) 36–37

2 See also DM Tribe & AJ Tribe, Lawteach: an interactive method for effective large group teaching (1987) 12 Stud in Higher Ed 299 at 302–310.

4 See also J Clift & B Imrie, Assessment: Learning and Performance (in press) to be published by The Open Polytechnic of New Zealand, Wellington at 42 and 101.

6 HJ Hurley, The Part II Certifying Examination in Dermatology: An Assessment of Interpretive Skills in JS Lloyd (ed) Evaluation of Noncognitive Skills and Clinical Performance (Chicago, IL: American Board of Medical Specialties 1982), referred to in ME Lunz, JA Stahl, BD Wright & JM Linacre, Variation Among Examiners and Protocols on Oral Examinations, paper presented at the Annual General Meeting of the American Educational Research Association, San Francisco, Cal, March 1989.

8 See D Newble & R Cannon, A Handbook for Teachers in Universities and Colleges (New York: St Martin’s Press, 1989) at 108–109; M Priestly, Performance Assessment in Education and Training: Alternative Techniques (Englewood Cliffs: Educational Technology Publications, 1982) at 164; JA Green, Teacher Made Tests (New York: Harper and Row, 1975) at 81.

11 Of the 32 vivas conducted, only two were all female and only three were all male.

12 See for example, P Ramsden, Learning to Teach in Higher Education (London: Routledge, 1992) at 41–46 and JB Biggs, Teaching for Better Learning[1991] LegEdRev 8; , (1991) 2 Legal Educ Rev 133–147.

14 Such research would also extend Roger’s consideration of Improving the Quality of Learning in Law Schools by Improving Student Assessment[1993] LegEdRev 5; , (1993) 4 Legal Educ Rev 113.

15 Priestly, supra note 8 at 166; G Gibbs, S Habeshaw & T Habeshaw (1988) 53 Interesting Ways to Assess Your Students (Bristol: Technical and Educational Services, 1988) at 93–94.

16 See also Clift & Imrie, supra note 4 at 41–43; Priestly, supra note 8 at 160–161.

17 Bell suggests that “more than one inquisitor is mandatory ... and usually two will suffice.” RC Bell, Assessment Workshop Package (Perth: University of Western Australia, 1978).

Strongly Agree 5	Agree 4	Neutral 3	Disagree 2	Strongly Disagree 1
16 (64%)	9 (36%)	0 (0%)	0 (0%)	0 (0%)

Strongly Agree 5	Agree 4	Neutral 3	Disagree 2	Strongly Disagree 1
3 (13%)	10 (43%)	2 (9%)	5 (22%)	3 (13%)

Strongly Agree 5	Agree 4	Neutral 3	Disagree 2	Strongly Disagree 1
3 13%)	10 (43%)	2 (9%)	6 (26%)	2 (8%)

Strongly Agree 5	Agree 4	Neutral 3	Disagree 2	Strongly Disagree 1
6 (26%)	10 (43%)	1 (4%)	3 (13%)	3 (13%)

Strongly Agree 5	Agree 4	Neutral 3	Disagree 2	Strongly Disagree 1
0 (0%)	0 (0%)	5 (22%)	13 (57%)	5 (22%)