Hughes, C --- "The modification of task dimensions for the development and assessment of student progress in legal skills development" [2010] LegEdDig 2; (2010) 18(1) Legal Education Digest 5

• The modification of assessment task dimensions in support of student progression in legal skills development

---

The modification of assessment task dimensions in support of student progression in legal skills development

C Hughes

Legal Education Review, Vol 19, No. 1, 2009, pp133-148

Coordinators and teachers now have access to an extensive body of literature, offering both theoretical and practical guidelines for those engaged in preparing students to profit from the work-based or work-integrated learning components of their programs. This literature has been developed from a range of sources including formal research, reflection on practice, and various disciplinary and interdisciplinary intervention and scoping projects.

This article owes its origins to one such project, one with an assessment focus led by the Council of Australian Law Deans (CALD) and funded by the Australian Learning and Teaching Council (ALTC). At a symposium conducted as part of this project, group discussion centred on issues relating to the systematic assessment of law students’ abilities to undertake a variety of professional roles.

Assessment tasks are comprised of a number of individual and inter-related elements, each of which requires decision-making at the design stage. An assessment task design (ATD) framework developed from systemic functional linguistics illustrates task elements and associated task design decisions and is based on the following assumptions: (1) A text is defined as ‘any meaning-producing event, be it a book, a film, an advertisement, a phone conversation and so on’; (2) All assessment involves students in the production of ‘texts’. The essays, reports, oral presentations, interviews, posters or blogs that students produce when engaging in assessment tasks are all accepted as lying within this definition of ‘text’.

Texts are shaped by the cultural and social contexts in which they are produced. It is helpful to our understanding of cultural context to consider the legal profession as a cultural group – referred to as a ‘community of practice’ or ‘discourse community’. Cultural groups share specialised knowledge and terminology, and communication practices which employ particular text types. Students may simultaneously participate in a number of communities, including peripheral membership in a professional community of practice, while striving for ‘full enculturation’. Full membership in a community of practice is demonstrated by being able to participate in the discourses or cultural practices that distinguish that community, including producing the texts characteristically produced by its members.

Within any particular social context or field of practice, the texts produced will be shaped by the role assumed by the text producer, the presumed audience of the text, its subject matter, and the mode or medium of communication.

The legal profession, like other professional communities, has its own collection of traditional and valued text types which form the basis of students’ text production during their years of university study. The independent production of many of the ‘signature’ texts of a professional community is the ultimate purpose of many professional law programs and an essential condition for attaining full membership of this particular cultural group.

The priority of the academics who participated in the role discussion at the ALTC/CALD symposium was quite specific – to design assessment tasks that provided appropriate opportunities for students to demonstrate learning and understanding at different levels of the professional program, while encouraging them to periodically revisit legal knowledge, skills and ways of disciplinary thinking.

Assessment task design requires an assessor to develop specifications for the texts that students are to produce to demonstrate their learning. Although, in practice, assessment task descriptions commonly explicate some specifications – usually subject matter and text type – but only imply others, the elements of the framework can be identified in all assessment tasks. For example, a law student working in the context or situation of a legal practice and assuming the role of solicitor may be required to conduct a face-to- face (medium) oral (mode) interview (text type) to elicit relevant information (purpose) from a client (audience) wishing to add a codicil to a will (subject matter).

To address the assessment priority of the academic participants at the ALTC/CALD symposium, a set of task ‘dimensions’ was identified. These dimensions would constitute variables which could be used to determine or adjust the challenge or demand of individual assessment tasks. Following the symposium, the initial set of dimensions that had been generated was further developed and links established with relevant assessment literature. Some resources, such as the ‘United Kingdom Quality Assurance Agency Subject Benchmark Statements’ and graduate attribute maps developed by individual institutions and disciplines address progression to some extent, but are too general or holistic to inform the design of individual tasks. However, the dimensions of oral assessment identified by Gordon Joughin, and the dimensions of authenticity articulated by Judith Gulikers, Theo Bastiaens and Paul Kirschner suggest a way of advancing the ideas participants generated at the symposium when drawing on their own assessment design experience. There are, of course, a number of other variables, such as text length or reference specifications, that assessors can manipulate in order to vary the challenge of assessment tasks. However, as the object of this exercise is identifying dimensions capable of adjustment for the design of tasks that suit different stages of student development, only those dimensions that can be represented on a continuum have been considered for inclusion in this paper.

Authenticity of task context, purpose as related to cognitive complexity and selection of text type all present the assessor with possibilities for manipulating elements of the cultural context in the design of assessment tasks.

Role-related learning can be decontextualised when students learn about role as a component of course subject matter. Students become aware of the expectations, regulations, accountability and ethical codes of roles associated with the legal profession. They also develop a knowledge and appreciation of the roles of people with whom they are likely to work collaboratively or whom they are otherwise likely to encounter in professional practice. However, they are distanced from practice or are ‘standing outside’ it, when the assumption of these roles is not integral to teaching and learning activities. Decontextualised assessment would reflect this type of learning in that students would be required to demonstrate learning about these roles through the production of texts such as essays, multiple choice tests or oral presentations within traditional university assessment contexts. Experiential learning pedagogies involve learning through role (role-play or simulations) or in role (practicums, internships, work-based/ integrated or service learning). Simulated contexts can be provided through role-play with peers (solicitor and client), with academics or actors as clients, or through the use of technology that facilitates replication of client interactions. Suggestions provided by Lee Andresen, David Boud and Ruth Cohen, illustrate types of assessment tasks congruent with experience-based learning and include group projects, critical essays located in the learner’s own experience, reading logs and learning journals. Case-based or scenario type examination items can also address the need to connect assessment to authentic contexts. There are, for example, instances where a legal practice has been invited to contribute to the design of assessment tasks to boost authenticity.

In authentic, real-life contexts, students undertake actual tasks with varying levels of supervision depending on ethical, legal and safety factors.

Though the level of cognitive demand required by the student in producing any particular text should directly correlate with the learning objectives or purposes of the course, resources such as Bloom’s (revised) Taxonomy, can support the development of progressive assessment tasks that are mainly formative or developmental in orientation. Activities requiring the reproduction or identification of information represent the lowest level of demand on the taxonomy and provide a foundation for further progression through other stages of understanding, applying, analysing, evaluating and synthesising to creating (the highest level). Thus, development of a text type such as an interview can be undertaken initially as an observation exercise using a video tape with students required only to recognise or identify examples of specified behaviours and to record them on a checklist. Students can then progress to assessment tasks requiring application of interview prompts in a formulaic way to analysing and evaluating own and others’ interview effectiveness through role-play before conducting interviews in more authentic contexts.

A related concept, ‘solution space’, is used by Gulikers et al in referring to the number of possible correct responses students can give to an assessment task. Real-life tasks or problems can be ‘closed’ – there is only one acceptable correct response – or ‘open’ – problems are open to interpretation and the generation of multiple responses that will be accepted as ‘correct’ providing task criteria are met. It is also essential that students learn to recognise whether one or multiple responses are required by an assessment task and that they gain experience in producing texts as appropriate responses to problems with differing levels of cognitive complexity and ‘solution space’.

It has been observed that assessment tasks in higher education exist on a continuum between traditional genres (text types) with strict norms, and less structured problem- or project-based assignments. Law students, for example, are sometimes required to produce texts that have clearly defined, formulaic or rigid structures such as case notes. Other text types more suited to students in later years – a research essay or memorandum – may require the student to devise an original way of organising and relating information.

Within each cultural context, texts are shaped by the role assumed by text producers; their knowledge of, relationships and interactions with others involved in the production process and with the audience of the text; the subject matter of the text; and the mode and medium in and through which it is expressed. To some extent, in a field such as law, mode and medium are determined by the selection of text type – a will (text type) is a written document (mode) stored in a paper-based or electronic form (medium). Consequently, mode and medium will be considered as inextricably bound to the selection of text type and therefore beyond significant independent variation.

Legal practitioners undertake a variety of professional roles in developing and maintaining society’s legal, industry, political and other systems. This provides a rich opportunity for the development of assessment tasks in which students assume a variety of roles in simulated or authentic contexts. Assigned roles can be professional – advisor, advocate, policy developer – or related to those with whom legal practitioners are likely to interact in professional practice – client, industry advocate, or environmentalist. It is therefore suggested that those who design tasks to assess role development consider a comprehensive range of roles which reflect different fields of potential professional practice and the role possibilities within those fields. While role-play in non-professional roles may not in itself offer direct assessment opportunities, it can provide a valuable learning experience to be assessed in other ways. Letters to the editor, informational pamphlets, posters, ministerial briefings or speeches to community groups are only some of the texts that can be produced in simulated contexts, while field or service learning experiences provide rich contexts for the production of a variety of more authentic texts.

Predictability can be manipulated by deciding whether, first, the audience will be one with which the student is familiar and comfortable, or whether audience dispositions and positions will be either unknown to a student (for example, an actor ‘primed’ to be hostile or otherwise difficult) or known to be antagonistic to an argument or perspective to be presented. Second, the task could allow texts to be prepared in advance and presented as intended (a prepared talk on a familiar topic delivered with no interruptions) or it could require interaction with others (interviewee, client, audience), which would call for spontaneity of response and an ability to improvise or ‘think on one’s feet’.

The subject matter of assessment tasks can most easily be adapted for students at different stages of a professional program by the extent to which relevant legal principles are specified or otherwise indicated in task instructions. Some tasks restrict, prescribe, or identify the subject matter to be included in a text. Other tasks merely state a problem or issue and leave it to students to draw on their expanding knowledge of the disciplinary field to identify the appropriate legal principles or subject matter to bring to bear in investigating the problem and in proposing or justifying a solution or argument.

Therefore, students in the early years of a program can be given quite specific parameters regarding the subject matter to be included (or excluded) when responding to assessment tasks and guidelines for its treatment. More advanced students may be provided with a case or problem for which the relevant legal principle may be open to various interpretations and therefore the quality of response will be determined by students’ breadth and depth of legal knowledge, their ability to discuss and evaluate a range of perspectives and to present a defensible conclusion or opinion.

Two further task dimensions relate to the conditions under which assessment is conducted. The level of scaffolding provided to students and the weighting attached to individual assessment tasks offer possibilities for adjustment.

Scaffolding refers to the type and level of support provided to students completing learning and assessment tasks and it is therefore an essential component of a teacher’s pedagogical repertoire. Assessment tasks can be heavily scaffolded for students in the early years of a program or the early stages of development of a particular skill or attribute and then gradually reduced or ‘faded’ as students acquire greater independence. In general, the level of scaffolding or guidance provided will depend on ‘the student’s stage of development in the field and the complexity of the material’. This by no means absolves the teacher from any teaching responsibility as students advance through a program, but rather changes the nature of teaching to one more commensurate with the shift of responsibility for learning from teacher to student. Scaffolding can take many forms, including providing detailed criteria and standards, prompts, templates or one or more opportunities for feedback on draft texts prior to final submission or performance.

In addition, text types with clearly distinguishable components suggest activities where the task can be scaffolded by requiring only partial text completion.

Weighting is also obviously easy to modify. The advantages of assigning a low weighting to an assessment task are that it can minimise the level of stress induced by other dimensions of the task and, in giving the task a mainly formative or developmental orientation, it can provide a supportive learning environment. However, as students often equate low weighting with low importance, it is essential to clarify the intent behind such assessment decisions and to apply rigorous minimum standards.

A further caution relates to the level and type of scaffolding provided to students at any level – there is growing concern in the assessment community that many current scaffolding practices are resulting in inappropriate levels of student dependency and a general reluctance by students to accept responsibility for their own learning or develop either the disposition or attributes required for lifelong learning.

The identification of the key dimensions of assessment tasks that can be manipulated to prepare students for professional practice is intended to serve as a resource that supports the work of those with assessment responsibilities in professional programs. The continua suggest ways of supporting skill development and assessment over a single course or over a number of years, of bridging course-based and work-based elements of professional programs, and of preparing students to ‘perform’ under varying levels of supervision. The continua in no way suggest the fixed or staged development approaches that have been subject to critique because of their inflexibility and narrowness of perspective. Rather, the independence with which each of the dimensions can be manipulated provides assessors with a flexible means of manipulating assessment design in response to diverse and changing student characteristics and assessment contexts.

The development of the legal skills that are fundamental to professional practice is too important to be left to chance. It is improbable that work-based or integrated learning experiences will be able to offer the structured skill development that can be provided through formal course-based activities. Skills such as those required for professional communication and interaction therefore need to be introduced during the formal education phase that generally precedes work-based learning.