[Home] [Help] [Databases] [WorldLII] [Feedback]
	Murdoch University Electronic Journal of Law

Assessing Teamwork Skills in Law School: A Window of Opportunity

Author:	Kelley Burton BBus, LLB (Hons) Associate Lecturer, Queensland University of Technology Faculty of Law
Issue:	Volume 10, Number 2 (June 2003)

Contents:

• Introduction
• Traditional Assessment Practices
• Criteria
• Assessment Practices For Teamwork Skills Where There Is A Team Product
• Product summatively assessed and process not assessed
• Product summatively assessed and process formatively assessed
• Product assessed as team mark and process assessed as individual or team mark
• Product assessed as a team mark by educator and peers mark the process
• Product marked as a team and process self marked or assessed
• Product marked as a team and process marked by educator, peers and/or self
• Product not assessed and process assessed using one of above methods
• Assessment Strategy For Law Schools
• Conclusion
• Appendices
• Bibliography
• Notes

Introduction

1. Traditionally, law schools concentrated on teaching students the substantive law, which produced graduates with a sound technical knowledge, but not necessarily skills that are essential to practice law.[1] Generic skills are important to the legal profession and they can be transferred to a non-legal context. An example of a generic skill is the ability to work in a team, that is, teamwork skills.[2] The Queensland University of Technology[3] Law School has embedded teamwork skills into its law degree in response to pressure from employers and students and recommendations from the Australian Law Reform Commission.[4]

2. Christensen and Kift from the QUT Law School identified three levels through which undergraduate law students progress to attain generic skills,[5] such as teamwork skills. The levels are equated to the full-time years of the law degree.[6] Students progress through level one in the first year of their law degree, through level two in the second year and level three in the third and fourth years of their law degree. At level one, students will be instructed on teamwork skills and they may practice teamwork skills under the guidance of a legal educator, who will provide feedback. Assessment of teamwork skills at level one will usually include a critique of teamwork skills.[7] At level two, students are required to be less dependent on the legal educator. The legal educator may provide additional guidance on teamwork skills at an advanced level and students will be expected to practise their teamwork skills "in a real world legal scenario"[8] The legal educator would provide feedback to their students on their teamwork skills. At level three, students should be able to transfer their teamwork skills to any context without guidance from the legal educator.[9] Students should be encouraged to reflect on their teamwork skills at all levels.[10]

3. The QUT law school has embedded teamwork skills into its curriculum at the three levels, but the QUT law school faces a major challenge in determining what is the most effective way of assessing teamwork skills at the three levels so that students develop their teamwork skills as they progress through the law degree and become independent life-long learners. The main aim of this article is to determine how to assess teamwork skills, that is, the process, at the three levels and to generate a strategy for law schools on how to assess teamwork skills. Section 2.0 of this article emphasises that traditional assessment practices of teamwork in law schools have focussed on assessing only what the team produces, that is, the product. Section 3.0 develops criteria that should be met by practices that assess teamwork skills. Section 4.0 considers some practices that assess teamwork skills in terms of the criteria. It also advises law schools where each assessment practice could be encapsulated in an assessment strategy that is designed so that students build up their teamwork skills as they progress through the law degree. Section 5.0 highlights that there is a window of opportunity for assessing teamwork skills and refers to an assessment strategy that law schools may wish to implement.

4. Please note that this article only canvasses assessment practices for teams that produce one product and not individuals who have worked in a group and submitted individual products.

   Traditional Assessment Practices

5. Traditionally, law schools only assessed what a team produced, that is, the product.[11] However, the ability to work in a team, that is, the process, is important for law graduates and law schools should diversify their assessment practices to continually assess the product and the process. Continuous assessment practices promote learning throughout the year, provide a wider sample of a student's knowledge and take into account the variability of student performance.[12] Before law schools can determine the most effective way of assessing teamwork skills, criteria should be established and each assessment practice should be discussed in light of this.

   Criteria

6. In establishing such criteria, it is useful to draw on the generic criteria used by the United Kingdom Quality Assurance Agency for Higher Education[13] and the American Association for Higher Education.[14] The UKQAA recommended that assessment practices should be valid, reliable, transparent, fair, formative, timely, incremental, demanding and efficient.[15] These are discussed below.

7. Assessment practices are valid when they assess what they are supposed to assess, that is, when they are aligned to the unit objectives.[16] Once this happens and students are informed of the unit objectives then transparency occurs.[17] Assessment practices are reliable where, for example, two legal educators mark the same product on different occasions and generate the same result.[18] Validity, reliability and transparency are important at all three levels of the law degree, particularly at levels two and three where students have grasped teamwork skills and want to fine-tune them. These criterions are also more important where the assessment is summative.

8. Assessment practices should be fair, that is, they should not disadvantage an individual or group of students.[19] The AAHE acknowledges that the results of assessment usually indicate which students do best under what conditions and that such knowledge should be used to improve student learning on a continuing basis.[20] Fairness is important at all three levels of the law degree, but is of particular concern where peers mark or assess teamwork skills.

9. Student learning can also be improved by formative assessment, which informs students how they are performing and how they can improve.[21] Formative assessment is analogous to the timely and incremental criterions. The timely and incremental criterions are satisfied if students are provided with the opportunity to practise and build up their skills as they progress through the unit rather than only at the end of the unit. This is consistent with the principles advocated by the AAHE, which supports the use of a diverse range of assessment practices over time to improve the education experience.[22] The formative, timely and incremental criterions are very important at level one of the law degree so that students come to grips with teamwork skills and receive feedback on their teamwork skills before they are summatively assessed on them.

10. Formative assessment should be distinguished from summative assessment, which is designed to satisfy credentialing agencies.[23] The AAHE also supports continuous assessment because it enables legal educators to monitor student progress towards goals.[24] Summative assessment is more appropriate at levels two and three where students have grasped teamwork skills and have already received feedback on how to improve their teamwork skills.

11. The UKQAA asserts that assessment practices are efficient where they use staff time and resources effectively and do not place excessive burdens on students. In contrast, the demanding criterion ensures that assessment practices are not a soft touch and that students should be required to work hard and effectively. However, Barnes suggested that assessment practices should impose a realistic workload on legal educators and students, and minimise the expenditure on resources.[25] There is a correlation between the demanding and efficient criterions because they both consider workloads. The demanding and efficient criterions are important at all levels of the law degree, particularly at level one where there are more students enrolled and students are just appreciating the workload associated with studying law.

12. Each assessment practice will be considered in light of the following criteria:
    • valid
    • reliable
    • transparent
    • fair
    • formative, timely and incremental
    • demanding and efficient

    Finally, each assessment practice will be placed in a law school assessment strategy based on how the assessment practice meets the criteria listed above.

    Assessment Practices For Teamwork Skills Where There Is A Team Product

13. There is a window of opportunity for assessing teamwork. Some of the factors to consider when assessing teamwork skills are the source, mode and whether an individual or team mark are awarded. Appendix A lists the options available for assessment practices in terms of source, mode, individual or team mark and how the total mark is calculated. Appendix A also indicates which opportunities are considered in this article. In light of the ever increasing workloads of legal educators, it is essential to determine who is going to mark or assess the product or the process. The person or people who mark or assess are called the 'source'. The potential sources are the legal educator, peers, the individual student or a combination of them. The source may mark or assess students individually, that is, provide each student with an individual mark. Alternatively, the source may provide one mark to the team, that is, a team mark. The individual or team mark may be summative or formative. The terms 'summative' and 'formative' were briefly addressed under 3.0. Essentially, summative assessment results in a mark that counts toward, for example, a grade for a subject. Formative assessment does not in itself count towards a grade for a subject, but attempts to provide feedback to students on how they can improve their teamwork skills.

14. The total mark for a piece of assessment may be derived by adding the product and the process mark together. Alternatively, the total mark may be derived by distributing the product mark in accordance with the process mark. The following 7 sections of this article consider some of the opportunities available as assessment practices and whether they meet the criteria above. These opportunities have been chosen and discussed in the order below to indicate how students could develop their teamwork skills as they progress through their law degree and become independent life-long learners.

    Product summatively assessed and process not assessed

    Valid

15. This assessment practice involves awarding a team mark for the product and not the process. Traditionally, legal educators have predominantly assessed the product and not the process. This article accepts this as a starting point but focuses on practices that make an effort to assess the process. Some legal educators contend that the process of working in a team is implicit in the product[26] but others argue it is very important for law graduates and should be assessed in its own right. The validity of awarding a team mark to the product and not directly assessing the process should be questioned because it does not assess a student's ability to work in a team.[27]

    Reliable

16. The assessment of the product may be subjective but if a legal educator marks all the products or if all markers understand the content and the criteria then the assessment practice should be reliable.[28]

    Transparent

17. Legal educators should explain the criteria to students at the beginning of the task to satisfy the transparency criterion.

    Fair

18. One of the drawbacks where students in the team receive the same mark is that it encourages some students to freeload off the valuable contributions made by other team members.[29] Consequently, this practice is unfair.

19. Another drawback of awarding all members in the team the same mark is that the low ability members will receive a high mark because of the effort of the high ability students. Lejk tenuously argued that a student's motivation depended on whom they worked with and this impacted on their marks.[30] For example, this caused high ability students working in a mixed-ability team to have lower motivation and receive lower marks than if they had worked in a same ability team.[31] The reverse was true of low ability students, that is, they had higher motivation and received higher marks in mixed-ability teams than low ability students working with students of the same ability.[32]

20. Similarly, Isaacs advocates that low ability students will learn effectively with the assistance of the high ability students.[33] However, Isaacs asserts that students with a high ability will learn effectively when working with students of a low ability because they will clarify their thoughts when assisting the low ability students.[34] This assertion contradicts Lejk's research, but Isaacs recommends that students at the high and low ability extremes should not be placed in a team.[35] He suggests that an intermediate category be introduced so that students are categorised with high, medium or low ability and that medium ability students are placed in teams with high, medium or low ability students.[36] In following this approach, there is one major question that remains unanswered, that is, how do you determine whether a student is of high, medium or low ability? A discussion of this is beyond the scope of this article.

21. Isaacs recommends that if mixed-ability teams are dysfunctional and students complain that the constitution of the team is unfair, legal educators should advise the students to collaborate and provide each other with mutual assistance because this is likely to lead to an increased team mark.[37]

22. Despite the criticisms of awarding a team mark to the product, it would be difficult for legal educators to award students with individual marks based on their contribution to the product because legal educators do not have the inside knowledge on the exact contributions by the team members.

    Formative, timely and incremental

23. Under this assessment practice, the product is marked at the end of the task. It does not inform law students how they can improve their skills as they progress. Consequently, it will not satisfy the formative, timely and incremental criterions, unless for example, the legal educator provides input on drafts of the product. However, this will increase the burden on legal educators.

    Demanding and efficient

24. This practice may save the legal educator's time because they only need to mark one product from a team of students as opposed to marking a product from each student. To counteract this argument, it is expected that a team product would be longer in length and more demanding on students than an individual product. Consequently, this may not reduce the legal educator's marking time by much, if at all. However, marking only the product efficiently uses the legal educator's time compared to those practices that assess the product and the process.

    Strategy for law schools

25. The most compelling reason to employ this practice is that it is less onerous for the legal educator compared to those practices that assess the product and the process. Consequently, it may be more appropriate at level one where there is a high intake of law students.[38] It is also submitted that at level one, it may not be necessary to assess the process of working in a team because students are just coming to grips with the application of the skill.

    Product summatively assessed and process formatively assessed

    Valid

26. In addition to awarding a team mark for the product as outlined in section 4.1 of this article, the process may be formatively assessed. Legal educators, peers or the individual team members may formatively assess the process by awarding a team or individual mark and giving students feedback on how they could be a more effective team member. Legal educators may formatively assess the process by, for example, observing students working in teams or asking students to critique teamwork skills in an essay. Formatively assessing the process increases the validity compared to those assessment practices that do not assess the process. However, there are some criticisms about validity where the legal educator assesses the process. These criticisms also emerge under section 4.3 of this article and are of a greater concern under that section because the assessment practice in section 4.3 is summative. The assessment practices below consider the summative assessment of the process by peers and the individual team members. Similarly, peers or the individual team members could be involved in formatively assessing the process.

    Reliable

27. To ensure that the assessment practice is reliable, the markers must understand the marking criteria.

    Transparent

28. Legal educators should explain the criteria to students at the beginning of the assessment. Legal educators may allocate class time to emphasise to students the importance of assessing the process and discussing the results.[39]

    Fair

29. If the process is only formatively assessed and a team mark was awarded to the product, the freeloading and mixed-ability team drawbacks, discussed at 4.1, would still arise. In this way, this assessment practice is similar to only assessing the product because the process is not summatively assessed. Other fairness issues depend on how the process is assessed and it would be useful to draw on the discussion under the other practices in this regard.

    Formative, timely and incremental

30. Whether these criterions are satisfied depends on how the process is assessed. Once again, the other assessment practices may be a useful reference here.

    Demanding and efficient

31. Whether these criterions are satisfied depends on how the process is assessed. Once again, the other assessment practices may be a useful reference here.

    Strategy for law schools

32. Law schools may decide to formatively assess at level one because students are just coming to grips with the application of the skill and it provides them with the opportunity to improve before they are summatively assessed. Legal educators may assess the process by observing students working in teams or asking students to critique teamwork skills in an essay. Alternatively, formative peer or self assessment or marking may be introduced at level one because the reliability of them improves with practice.[40]

33. The main flaw with this strategy is that students "tend to ignore material or approaches which are not assessed or which they do not think will be assessed"[41] Consequently, students may treat summative assessment more seriously than formative assessment and they may discount the importance of working in a team if it is only formatively assessed at level one. However, legal educators may be able to address this flaw by making the formative assessment a prerequisite[42] of completing the unit to ensure that students value the process of working in a team.

    Product assessed as team mark and process assessed as individual or team mark

    Valid

34. In addition to awarding a team mark for the product as outlined in section 4.1 of this article, the legal educator may summatively assess the process of working in a team in a theoretical manner and award a team or individual mark. An example of this would be assessing a student's ability to critique teamwork skills in an essay. However, this may be criticised on the basis of validity because it does not assess a student's ability to work in a team. It assesses whether a student has developed an appropriate writing genre, whether the student knows how to work in a team[43] and their academic ability[44]

35. Alternatively, the legal educator may observe a student working in a team and on that basis assess the student's ability to work in a team. However, this may be criticised because it would be restricted to teams working in a classroom and the legal educator could not observe every student at all times.[45] Video cameras and audio tape recorders may assist legal educators to observe a student's ability to work in a team, but this may be unfavourable to law schools because it strains resources and it is time-consuming for legal educators who need to replay the video and audio tapes.[46] Perhaps the legal educator could substantiate their marks with the teamwork records, but this places a huge amount of trust in the hands of the students. Examples of teamwork records include agendas, reports, minutes, ground rules, vision statements and statement of outcomes.

36. Due to the criticisms relating to validity, perhaps the weighting of the process should be minimised where the legal educator assesses the process. Also these criticisms relating to validity justify the elimination of distributing the product mark amongst the team members in accordance with the process mark.

    Reliable

37. To ensure that the assessment practice is reliable, the markers must understand the marking criteria.

    Transparent

38. Reliability and transparency would be satisfied if the legal educator develops assessment criteria at the beginning of the task and advises students of it.

    Fair

39. If students were awarded the same mark for the product, the freeloading and mixed-ability team drawbacks, which are discussed at 4.1, would still arise. However, awarding individual marks to team members for the process may go toward addressing these drawbacks. Awarding individual marks may be disparaged because it perpetuates the competitive culture in law schools[47] and discourages collaboration, which is fundamental to teamwork.

    Formative, timely and incremental

40. Similarly to only assessing the product, this assessment practice would not satisfy the formative, timely and incremental criterions because the assessment occurs at the end of the task and does not allow law students to build up their skills.

    Demanding and efficient

41. This assessment practice is more onerous for a legal educator because it requires them to mark the product plus the process. It is also more demanding on students because they need to focus on the product as well as the process.

    Strategy for law schools

42. This assessment practice is more appropriate for level one because the legal educator only summatively assesses the process of working in a team in a theoretical manner or observes the application of teamwork skills at a basic level. It is at a basic level because the legal educator cannot observe every student at all times.

    Product assessed as a team mark by educator and peers mark the process

    Valid

43. In addition to the legal educator awarding a team mark for the product as outlined in section 4.1 of this article, the process may be summatively marked by peers, that is, the other team members. The peer marks may be averaged to arrive at the process mark for each individual in the team. The peers may mark the process by completing a questionnaire set by the legal educator, which is called peer-marking.[48] The questionnaire may also be referred to as a feedback or criteria sheet. Alternatively, the legal educator may ask the students to develop the criteria and mark the process accordingly, which is called peer-assessment.[49] Peer assessment or marking of the process may be more valid than if a legal educator assessed it because the team members have inside knowledge of a member's contribution to the team, particularly if the teamwork is performed outside the classroom or in a virtual environment.[50]

44. Instead of adding the marks for the product and process to determine the total mark, the legal educator may distribute the team product mark in accordance with the individual process mark. This is renowned as a zero-sum game where an increase in marks for one student is accompanied by a decrease in marks for another.[51] It assesses a student's ability to work in a team by comparing it to the ability of the other team members. The validity of this assessment practice may be disparaged because all of the members in one team may be of a higher ability than the members of another team, but this will not be reflected in the mark.

45. Irrespective of how the total mark is derived, peer-assessment better assesses a student's ability to work in a team, but it has been opposed on numerous grounds, most of which relate to reliability.

    Reliable

46. Some legal educators question the ability of students to assess the contributions of their team members.[52] They claim that contextual issues, for example, the nature of the task, impact on the reliability of peer-assessment.[53] To overcome this, legal educators may introduce a moderating mechanism, but this would be time-consuming.[54]

47. Peers usually complete questionnaires to mark the contributions of team members. The development of questionnaires may be considered analogous with legal interviewing because they both involve the careful construction of questions to elicit information. Questionnaire developers should provide clear and specific questions to ensure reliability. They should also avoid value-laden and leading questions.[55]

48. Each team member may mark the contributions of the other team members by nominating, rating, ranking or providing a holistic mark. Nominating involves identifying the members who make the best and worst contributions.[56] This is norm-referenced assessment because it requires a comparison between the contributions of the team members. It generates competition as opposed to co-operation.[57] Rating requires each team member to assess the other members on predetermined criteria.[58] After this, the ratings would be totalled and averaged to arrive at a mark that represents the overall contribution of the team member. Ranking requires team members to identify the best and worst team member for predetermined criteria and there would be a mechanism in place to arrive at an overall contribution mark.[59]

49. Rating and ranking are criterion-referenced assessment, that is, assessment based on competency.[60] It is more effective where there are small differences between the contributions of the team members.[61] Providing a holistic mark requires the students to award one mark to the overall contribution of the team member.[62] It is norm-referenced assessment and is more effective at dealing with outstandingly good and weak contributions[63] but as mentioned above it generates competition.

50. The developers of the peer-marking criteria could use ratings, such as, did not contribute, minimal contribution, below average contribution, average contribution, above average contribution and outstanding contribution[64] or poor, below average, average, above average and outstanding[65] Some researchers have speculated that the type of ratings used may influence students, for example, one to five may produce a different outcome to negative one to three.[66] These arguments are tenuous, but ratings that use behavioural descriptors are likely to provide more informative feedback than numbers or letters.[67]

51. The opponents to peer-marking also claim that students will use different standards for the criteria to evaluate their team members[68] for example, one student may interpret the words 'average contribution' differently to another student and the interpretation may vary from team to team. To overcome this drawback, legal educators should provide students with detailed guidelines on how to mark the process and explain the assessment criteria to them.[69] This would increase reliability.

52. There are other ways to increase the reliability of peer marking and assessment. There is a strong correlation between increased reliability of peer-marking and student experience at peer-marking.[70] Reliability is also improved where an individual process mark is derived from the averaging of multiple peer-markings.[71] The literature also indicates that peers tend to award marks within a narrower range compared to the legal educator.[72] This benefits the lower ability students at the expense of the higher ability students.[73] However, where peer-marking is conducted secretly under exam conditions, there is likely to be a greater spread of individual marks than if the peer-marking was conducted openly between the members. The reason for this may be that students fear that by openly providing lower peer marks they may damage their relationships with the other team members and may receive lower peer marks in the future as revenge. Therefore, secret peer-marking better reflects a student's contribution to the team.[74]

53. Reliability would also be increased if legal educators requested students to develop the criteria, so that they can peer-assess rather than peer-mark. Some variations on this may involve the legal educator negotiating the criteria with students or the legal educator providing some core criteria and the students adding some criterions.[75] In doing this, they develop a sense of ownership of the criteria because they determine the meaning of it and its worth in terms of marks.[76] Race argues that this enables the students to apply the criteria much more objectively than one devised by the legal educator.[77] Orsmond contends that first year students who are quiet in class discussions may benefit from constructing marking criteria in teams because it will develop their assessment skills.[78] However, this may be disparaged because some first year students would not have enough knowledge, experience or confidence to develop marking criteria. This comment may be criticised because many students enrolled in the QUT law degree in 2002 are non-school leavers and have already completed a university degree. Orsmond asserts that third year students may find it more challenging to satisfy marking criteria given by the legal educator.[79] However, it is submitted that some students may find it more challenging to develop marking criteria than to satisfy one designed by the legal educator.

    Transparent

54. Transparency would be increased in peer marking or assessing if the legal educator explains to the students how to complete the questionnaires or if the legal educator encouraged students to be involved in designing the questionnaires.

    Fair

55. Irrespective of whether the marking criteria is developed by the legal educator or the students, there is a huge amount of trust[80] being placed in the hands of students when the peer assessment or marking forms part of the summative assessment. Students should be honest and objective, but some of the literature indicates that some students are suspicious about the strategies of other teams or team members to inflate their marks.[81] The literature indicates that if a student marks their peers generously compared to what their peers mark them, they penalise themself.[82]

56. To counteract this, self assessment or marking may be introduced because students that mark their peers generously tend to generously mark their own contribution.[83] Where peer assessment or marking occurs in conjunction with self assessment or marking, the student who provides the greatest contribution to the team is usually modest about their contribution in their self assessment or marking and is rated very highly by their team members in the peer assessment or marking.[84]

57. The reverse of this is also true, that is, the student who contributes the least to the team would usually over-rate their contribution and the team members rate them lowly.[85] One exception to this would be students with low self-esteem, who would probably under-rate themselves regardless of who they were comparing themselves.[86] Some students have expressed concern about being too critical or generous in peer assessing or marking,[87] but the fairness and reliability increases when multiple peer assessments or marks are totalled and averaged to determine the summative individual process mark.

58. To address these issues, perhaps the peer mark or assessment could be substantiated by the team records. This still places a huge amount of trust in the hands of the students who may fabricate the team records.[88] Fabrication may be reduced if students had to submit the team records to the legal educator on a regular basis, but this would increase the workload of the legal educator. Alternatively, the legal educator could provide a moderating mechanism in the case of an injustice, but this increases the workload of the legal educator, particularly if large numbers of students are enrolled in the unit.[89]

59. Another issue relating to fairness is that international students are more willing to contribute to teamwork because of the "allegiance to Confucian principles as they relate to the organisational unit".[90] This may lead to international students being awarded higher peer-marks for their contribution compared to Australian students. To minimise the gap between international and Australian students, legal educators should advise Australian students of the benefits of teamwork skills. However, this will increase the workloads of legal educators.

    Formative, timely and incremental

60. Peer assessment and marking could satisfy the formative, timely and incremental criterions if, for example, they were carried out each time the team met, rather than only once at the end of the task. However, the drawback of this is that it is more demanding of students.

    Demanding and efficient

61. Questionnaire developers should appreciate that open questions as opposed to closed or narrow questions, are more onerous for the peers completing the questionnaire.

62. Peer assessment and marking should increase student learning because it teaches students how to think critically and reflect, without more input from legal educators.[91] In contrast, Cheng and Warren contend that administering peer assessment and marking to determine the final mark is much more complicated than if a legal educator awarded the mark. Purchase recognised that the legal educator's time is not reduced in the first few years because the assessment practice needs to be set up and refined in accordance with student feedback.[92] In the long term, this assessment practice may be less onerous for legal educators, but it would still place a heavier burden on legal educators and students compared to those practices that do not assess the process.

    Strategy for law schools

63. Students learn to peer-assess at an early age[93] and consequently, first year law students would be capable of peer-assessing, but this may be more effective at level two where students have grasped teamwork skills and are applying them to real world legal scenarios. Law students at level three may be more competent at peer- assessing, that is marking according to criteria set by themselves, because they have more knowledge and experience with assessment criteria. In comparison to the other assessment practices, peer marking and assessing places a heavier administrative burden on legal educators because they have to total and average the peer-marks to arrive at the total process mark. Consequently, it should be implemented at levels two and three where less students are usually enrolled.

    Product marked as a team and process self marked or assessed

    Valid

64. In addition to the legal educator awarding a mark for the product as outlined in section 4.1 of this article, the individual may be required to award themselves with an individual mark for the process. This practice is valid because the team member self assesses or marks their ability to work in a team. This could be achieved by using open-ended self reports, semi-structured reports, ratings or checklists.[94] The students may mark themselves against a criteria set by themselves or the legal educator, or against their previous performance or the performance of others in their team.[95] It is submitted that if students are marking themselves against their previous performance, the process should be formatively assessed.

    Reliable

65. Some of the commentators contend that student generated marks are consistent with legal educator marks, whereas others claim that lower ability students over-rate their ability and higher ability students under-rate their ability.[96] In contrast, some commentators have argued that all students under-rate their abilities.[97] It is difficult to reconcile these comments, but reliability is increased where legal educators explain the criteria to students and where students are involved in developing the criteria.

66. Distributing the product mark amongst the team members in accordance with the process marks may be unfavourable given the problems with reliability.

    Transparent

67. If the legal educator explains the criteria to students or encourages them to be involved in developing the critieria, the transparency of the assessment practice would be increased.

    Fair

68. If some students over-rate their ability, it would be unfair to the other students. Consequently, the weighting of summative self assessment or marking may be minimal or legal educators may request students to complete a formative reflection sheet. The assessment of reflections has been criticised because students who developed an appropriate writing genre would receive high marks, irrespective of whether they are above average at reflection.[98] Reflections are personal and it would be difficult to assess them with consistency, unless clear criteria are provided, but this may encourage superficial learning habits.[99] Reflection encourages students to become independent learners and should be incorporated at all levels of the law degree.

69. Distributing the product mark amongst the team members in accordance with the process marks may be unfavourable given the problems with fairness.

    Formative, timely and incremental

70. Self assessment or marking could be formative, timely and incremental if it was conducted at regular intervals, for example, each time the team met.

    Demanding and efficient

71. Self assessment or marking is more demanding for students because they are required to assess their teamwork skills, but it may also make students less reliant on legal educators and encourage them to be independent life-long learners. Self assessment or marking may be more efficient for legal educators because they are not required to award a mark for the process. However, from an administrative perspective, it may still be onerous for legal educators, particularly, if they need to add the process mark to the product mark.

    Strategy for law schools

72. It is recommended that summative self assessment or marking be introduced in the law degree at level three because level three students have developed their teamwork skills, received feedback from other sources on how they can improve their teamwork skills and be more experienced to make such judgments. The reliability of self assessment and marking is increased if students have formatively practised this at the previous levels. Some commentators recommend that self assessment or marking be introduced in the first year because students are more likely to be receptive to new assessment practice,[100] but this may be rebutted by arguing that as the legal profession offers a dynamic environment, students should be open to changes in assessment practices at all levels.

    Product marked as a team and process marked by educator, peers and/or self

    Valid

73. This practice involves totalling and averaging the marks awarded by the legal educator, peers and self to arrive at an individual mark for the process. It is renowned as triangulation.[101] Alternatively, the marks awarded by peers and self might be averaged. This process mark could be added to the product. The product mark could be determined in accordance with section 4.1 of this article. Alternatively, the product mark could be distributed in accordance with the process mark.

    Reliable

74. This practice results in increased reliability because multiple marks from different sources, that is, the legal educator, peers and self are totalled and averaged.

    Transparent

75. For the assessment to be transparent all sources must be advised of the criteria upfront and understand it.

    Fair

76. Once again, averaging multiple marks from different sources is a method of ensuring fairness because the weighting of any outriders would be minimised.

    Formative, timely and incremental

77. These criterions would be satisfied if the process was assessed throughout the assignment, rather than in a one-off manner.

    Demanding and efficient

78. The main drawback with this practice is that it is very time consuming for students and legal educators. Students are required to peer and self mark or assess and legal educators need to total and average three marks for each student.

    Strategy for law schools

79. As this assessment practice is very time consuming, law schools may be reluctant to implement it. Alternatively, it may be employed at level three where less students are usually enrolled. In addition to considering the number of students enrolled, law schools should consider the staff to student ratio, before implementing this assessment practice.

    Product not assessed and process assessed using one of above methods

    Valid

80. Law schools may award a team or individual mark to the process using one of the practices discussed in sections 4.2 to 4.6 of this article, but not assess the product. Therefore, the product and the process are not added together and the mark for the product is not distributed in accordance with the process mark. The assessment of only the process is the opposite to what has traditionally been assessed, that is, only the product. This assessment practice is likely to be disparaged by credentialing agencies and legal educators because students may not take the content seriously if it is not assessed. There needs to be a balance between product and process because a student cannot make a valuable contribution to a team unless he/she has a sufficient knowledge of the content.

    Reliable

81. Whether this assessment practice satisfies these criteria depends on how the process is assessed. Reference to the above practices may be useful.

    Transparent

82. Whether this assessment practice satisfies these criteria depends on how the process is assessed. Reference to the above practices may be useful.

    Fair

83. Whether this assessment practice satisfies these criteria depends on how the process is assessed. Reference to the above practices may be useful.

    Formative, timely and incremental

84. Whether this assessment practice satisfies these criteria depends on how the process is assessed. Reference to the above practices may be useful.

    Demanding and efficient

85. Whether this assessment practice satisfies these criteria depends on how the process is assessed. Reference to the above practices may be useful.

    Strategy for law schools

86. Law schools may avoid this practice because it is extreme. Alternatively, they may incorporate it at level three where students may understand the importance of the content and may be fine-tuning their teamwork skills.

    Assessment Strategy For Law Schools

87. As discussed above, law schools have traditionally assessed the product of a team rather than the process of working in a team. Teamwork skills are important to the legal profession and this has inspired legal educators to embed and assess teamwork skills. There is a window of opportunity for assessing the process of working in a team and some of the opportunities have been discussed in sections 4.2 to 4.7. Some of these opportunities are more appropriate for levels one, two or three of the law degree as discussed at (g) above. Appendix B lists the opportunities discussed above and provides the main reasons for placing the assessment practices where they are in the assessment strategy. Ultimately, appendix B provides an assessment strategy for law schools on how to assess the process of working in a team at the three levels of the law degree.

    Conclusion

88. Law schools have traditionally only assessed what is produced by the team, that is, the product. They have neglected to assess the process of working in a team, that is, teamwork skills. The QUT Law School has recognised the importance of teamwork in the legal profession and has embedded teamwork skills across the three levels of the law degree. It is determining the most effective way of assessing teamwork skills so that students develop their teamwork skills as they progress through the law degree. If other law schools are revising how they asses teamwork skills, they should consider whether their assessment practices are valid, reliable, transparent, fair, formative, timely, demanding and efficient. There is a window of opportunity for the assessment of teamwork skills and law schools should approach it strategically so as to produce law graduates that have well developed teamwork skills and are independent life-long learners.

Appendices

Bibliography

Notes

[1] S Christensen and S Kift, 'Graduate Attributes and Legal Skills' (1997) 11 Legal Education Review 207 at 211.

[2] S Kift, 'Lawyering Skills: Finding their Place in Legal Education' (1997) 8 Legal Education Review 43 at 53.

[3] Hereinafter referred to as QUT.

[4] Para 2.89 of the Australian Law Reform Commission, Managing Justice - A Review of the Federal Civil Justice System Report No 89 available at <http://www.austlii.edu.au/au/other/alrc/publications/reports/89/ch2.html#Heading3>

[5] Christensen and Kift, above n 1, 219.

[6] ibid.

[7] ibid.

[8] ibid.

[9] ibid.

[10] ibid.

[11] Christensen and Kift, above n 1, 231.

[12] R Freeman and R Lewis, Planning and Implementing Assessment, Kogan Page, London, 1998 at 33.

[13] Hereinafter referred to as UKQAA.

[14] Hereinafter referred to as AAHE.

[15] P Race, The Lecturer's Toolkit: A Practical Guide to Learning and Teaching Assessment, 2nd ed, Kogan Page, London, 2001 at 33.

[16] ibid.

[17] ibid.

[18] ibid.

[19] Race, above n 15, 34.

[20] AAHE, Assessment Forum - 9 Principles of Good Practice for Assessing Student Learning available at <http://www.aahe.org/assessment/principl.htm>

[21] M Le Brun and R Johnstone, The Quiet Revolution: Improving Student Learning in Law, The Law Book Company Limited, Sydney, 1994 at 181.

[22] ibid.

[23] Le Brun and Johnstone, above n 21, 182.

[24] AAHE, above n 20, <http://www.aahe.org/assessment/principl.htm>.

[25] J Barnes, 'The Functions of Assessment: A Re-examination' (1991) 2 Legal Education Review 177 at 215.

[26] D Magin, 'A Novel Technique for Comparing the Reliability of Multiple Peer Assessments with that of Single Teacher Assessments of Group Process Work' (2001)

[26] Assessment & Evaluation in Higher Education 139 at 141.

[27] J Goldfinch, P Laybourn, L MacLeod, S Stewart, 'Improving Groupworking Skills in Undergraduates Through Employer Involvement' (1999) 24 Assessment & Evaluation in Higher Education 41at 47.

[28] Le Brun and Johnstone, above n 21, 185.

[29] G Isaacs, "Group Assessment" - Assessment of Students on Group-Based Tasks - Issues and Options - A Report for the University of Queensland Teaching and Learning Committee available at <http://http://www.tedi.uq.edu.au/teaching/assessment/downloads.html> at 3.

[30] M Lejk, M Wyvill, S Farrow 'Group Assessment in Systems Analysis and Design: A Comparison of the Performance of Streamed and Mixed-ability Groups' (1999) 24 Assessment & Evaluation in Higher Education 5 at 10.

[31] ibid.

[32] ibid.

[33] Isaacs, above n 29, 8.

[34] ibid.

[35] ibid.

[36] ibid.

[37] ibid.

[38] After searching the number of students enrolled in the core units in the QUT law degree on QUT Virtual in 2002, it appeared that approximately 600 students were enrolled at level one. This number decreased at levels two and three, to approximately 430 and 350, respectively.

[39] Freeman and Lewis, above n 12, 123.

[40] K MacPherson, 'The Development of Critical Thinking Skills in Undergraduate Supervisory Management Units: Efficacy of Student Peer Assessment' (1999) 24 Assessment & Evaluation in Higher Education 273 at 275.

[41] F Martin, 'The Integration of Legal Skills into the Curriculum of the Undergraduate Law Degree: The Queensland University of Technology Perspective' (1995) 13 Journal of Professional Legal Education 45 at 57.

[42] D Boud, R Cohen and J Sampson, 'Peer Learning and Assessment' (1999) 24 Assessment & Evaluation in Higher Education 413 at 422 and 424.

[43] S Stewart and B Richardson, 'Reflection and its Place in the Curriculum on an Undergraduate Course: should it be assessed?' (2000) 25 Assessment & Evaluation in Higher Education 369.

[44] P Laybourn, J Goldfinch, J Graham, L MacLeod and S Stewart, 'Measuring Changes in Groupworking Skills in Undergraduate Students After Employer Involvement in Group Skill Development' (2001) 26 Assessment & Evaluation in Higher Education 367 at 378.

[45] Freeman and Lewis, above n 12, 291.

[46] K Kovach, 'Virtual Reality Testing: The Use of Video for Evaluation in Legal Education' (1996)

[46] Journal of Legal Education 233 at 249.

[47] G Hess and S Friedland, Techniques for teaching law, Carolina Academic Press, North Carolina, 999 at 131.

[48] Le Brun and Johnstone, above n 21, 192.

[49] ibid.

[50] T Gatfield, 'Examining Student Satisfaction with Group Projects and Peer Assessment' 24 Assessment & Evaluation in Higher Education 365 at 374; and Magin, above n 26, 140.

[51] M Lejk and M Wyvill, 'Peer Assessment of Contributions to a Group Project: a Comparison of Holistic and Category-based Approaches' (2001) 26 Assessment & Evaluation in Higher Education 61 at 62.

[52] M Goos and K Moni, 'Modelling Professional Practice: a Collaborative Approach to Developing Criteria and Standards-based Assessment in Pre-service Teacher Education Courses' (2001) 26 Assessment & Evaluation in Higher Education 73 at 83.

[53] N Pope, 'An Examination of the Use of Peer Rating for Formative Assessment in the Context of the Theory of Consumption Values' (2001) 26 Assessment & Evaluation in Higher Education 235 at 237.

[54] Magin, above n 26, 146.

[55] A Chay and J Smith, Legal Interviewing in Practice, Law Book Company Ltd, North Ryde, 1996 at 177 and 178.

[56] Pope, above n 53, 236.

[57] Boud, Cohen and Sampson, above n 42, 418.

[58] Pope, above n 53, 236.

[59] ibid.

[60] Boud, Cohen and Sampson, above n 42, 416.

[61] Lejk and Wyvill, above n 51, 69.

[62] Lejk and Wyvill, above n 51, 62.

[63] ibid.

[64] Gatfield, above n 50, 374.

[65] L Li, 'Some Refinements on Peer Assessment of Group Projects' (2001) 26 Assessment & Evaluation in Higher Education 5 at 13.

[66] Lejk and Wyvill, above n 51, 70.

[67] Freeman and Lewis, above n 12, 43.

[68] Li, above n 65, 8 and F Lopez-Real and Y Chan, 'Peer Assessment of a Group Project in a Primary Mathematics Education Course' (1999) 24 Assessment & Evaluation in Higher Education 67 at 75.

[69] MacPherson, above n 40, 275.

[70] M Lejk and M Wyvill, 'The Effect of the Inclusion of Self-assessment with Peer Assessment of Contributions to a Group Project: a Quantitative Study of Secret and Agreed Assessments' (2001) 26 Assessment & Evaluation in Higher Education 551 at 558 and MacPherson, above n 42, 275.

[71] Magin, above n 26, 146.

[72] MacPherson, above n 40, 275.

[73] J MacAlpine, 'Improving and Encouraging Peer Assessment of Student Presentations' (1999) 24 Assessment & Evaluation in Higher Education 15 at 20 and W Cheng and M Warren, 'Peer and Teacher Assessment of the Oral and Written Tasks of a Group Project' (1999) 24 Assessment & Evaluation in Higher Education 301 at 305.

[74] Lejk and Wyvill, above n 70, at 560.

[75] Freeman and Lewis, above n 12, 40.

[76] P Orsmond, S Merry and K Reiling, 'The Use of Student Derived Marking Criteria in Peer and Self-assessment' (2000) 25 Assessment & Evaluation in Higher Education 23 at 32.

[77] Race, above n 15, 95.

[78] Orsmond, above n 76, 34.

[79] ibid.

[80] Lopez-Real and Chan, above n 68 at 78.

[81] Lejk and Wyvill, above n 51, 63.

[82] Lejk and Wyvill, above n 70, 558.

[83] Li, above n 65, 9.

[84] Lejk and Wyvill, above n 70, 557.

[85] ibid.

[86] Li, above n 65, 9.

[87] MacPherson, above n 40, 275.

[88] Lejk, Wyvill and Farrow, above n 30, 7 and A Matthew, 'Assessing Teamwork Skills in Undergraduate Law Students: A Case Study' presented at the Evaluation and Assessment Conference 2002 in Brisbane.

[89] Magin, above n 26, 146.

[90] Gatfield, above n 50, 371.

[91] ibid.

[92] H Purchase, 'Learning about Interface Design through Peer Assessment' (2000) 25 Assessment & Evaluation in Higher Education 341 at 350.

[93] Freeman and Lewis, above n 12, 125.

[94] Freeman and Lewis, above n 12, 119.

[95] ibid.

[96] Freeman and Lewis, above n 12, 120 and 121.

[97] Freeman and Lewis, above n 12, 121.

[98] Stewart and Richardson, above n 43, 372.

[99] Stewart and Richardson, above n 43, 374.

[100] Freeman and Lewis, above n 12, 123 and M Taras, 'The Use of Tutor Feedback and Student Self-assessment in Summative Assessment Tasks: Towards Transparency for Students and for Tutors' (2001) 26 Assessment & Evaluation in Higher Education 605 at 611.

[101] K Topping, E Smith, I Swanson and A Elliott, 'Formative Peer Assessment of Academic Writing between Postgraduate Students' (2000) 25 Assessment & Evaluation in Higher Education 149 at 149.