Collins, Pauline; Brackin, Toni; Hart, Caroline --- "The Rocky Rhetoric and Hard Reality: The Academic's Dilemma Surrounding Assessment" [2010] LegEdRev 9; (2010) 20(1&2) Legal Education Review 157

ROCKY RHETORIC AND HARD REALITY: THE ACADEMIC’S DILEMMA SURROUNDING ASSESSMENT

PAULINE COLLINS,^* TONI BRACKIN^** AND CAROLINE HART^**

I INTRODUCTION

The Bradley Review of Higher Education ¹ states that both more money and more students from diverse backgrounds ² will be necessary for Australia to effectively compete in the global market. Assessment is a factor which, as a learning driver for students, ³ must figure strongly in any consideration of the agenda to widen participation in higher education. In this regard, we have to closely consider the role of assessment within the institutional context. David Boud sounds a timely warning: ‘[i]f, as teachers, we want to exert maximum leverage over change in higher education we must confront the ways in which assessment tends to undermine learning’. ⁴ A focus on improving educational outcomes surely needs to seriously consider the role of assessment and factors that may prevent the outcomes hoped for in the Bradley Review. The Government response has been the establishment of the Tertiary Education Quality and Standards Agency (‘TESQA’) ⁵ and budgetary responses focusing on equity, access, sustainability, research and quality teaching. Funding of higher education is to be driven by the market as determined by students’ choice, without a cap on domestic undergraduate enrolments, and this is to be linked with exacting targets set by the government who will attempt to apply rigorous quality assurance standards to universities. ⁶

The drive to increase numbers of students aged 25–34 to 40 per cent by 2020 includes encouraging students who would previously not have considered university education. Therefore, students from diverse backgrounds, including students who are the first in their family to study at university (‘first in family’), from regional and remote areas, Indigenous communities and low socioeconomic groups (up to 20 per cent) become the focus in terms of student recruitment. ⁷ With this comes the correlated need to retain and progress these students to produce ‘quality’ and engaged students who are ‘work ready’ at the end of the education process. For law schools, this requires adopting a legal practice orientation and a broader curriculum, to engage a more diverse student body catering for differing career options for an uncertain future. This is all in a climate in which government spending over the past decades has lagged behind many other Western countries, with a lot of catch-up spending needed to claw back international credibility. ⁸ The education system has become a commodity by which government accountability to its taxpaying constituents is used to argue a right to expect greater value for each dollar spent ⁹ — a very seductive and, on the face of it, reasonable argument. However, it becomes the duty of academics and universities to ensure that the substantive role of genuine learning, education and research is not buried under a mountain of numbers and questionable accountability evaluations.

This raises the question of the very business of learning, at the heart of which is the lecturer–student relationship. It is well known that student learning is driven by assessment ¹⁰ and is the most important factor when it comes to determining whether the student has reached a satisfactory standard. ¹¹ More specifically, ‘[a]ssessment is the most powerful lever teachers have to influence the way students respond to courses and behave as learners’. ¹² However, David Boud has stressed that ‘[t]here is probably more bad practice and ignorance of significant issues in the area of assessment than in any other aspect of higher education’. ¹³ Student experience of assessment and thus their attitude to learning is also a vital component in students’ evaluation of both academics and the institution. Recent research that has analysed existing qualitative data from the national Course Experience Questionnaire (‘CEQ’) on students’ experience of higher education found that assessment was a key factor in determining the quality of students’ learning experience. ¹⁴ However, the drive for assessment of learning, as part of the public accountability requirements, can lose sight of the need to balance assessment for learning which is essential if students are to improve their skills and graduate as successful contributors to their chosen field. ¹⁵ Table A below indicates the preponderance of law assessment types students believe occur. This discloses that across Australian law schools students’ experience of assessment is that of traditional assessment types such as exams and essays which are focused on assessment of content, rather than skills and graduate attributes where assessment is more likely to be for learning outcomes.

Table A Frequency of Experience of Assessment Methods (overall percentage of students that thought each of the following occurred regularly) ¹⁶

Deputy Prime Minister Gillard, as she then was, acknowledged that the market will be driven by student choice in the ‘new student centred system and that this will impact on institutional behaviour’ in a new form of popularism. ¹⁷ This must be closely linked to questions concerning assessment, the academics that set assessment and the institutional and professional requirements that dictate the skills and graduate attributes that students must achieve in the climate of higher education. It is suggested that the factors outlined in this paper may well have a major impact on assessment and thus student learning.

Corporatisation of education across the globe pressures universities to meet many stakeholder demands and thus exerts pressure on academics to be many things to many people. ¹⁸ The move from education for the intellectual elite to education for the masses has created increasing student numbers in times when Australia is lagging far behind other OECD countries in its expenditure on education. ¹⁹ This, together with the demand for quality graduates and student retention, when students are often ‘first in family’ and from diverse, minority and low socioeconomic backgrounds, places an almost overwhelming burden on the ageing and decreasing number of academics. ²⁰ With around 60 per cent of academics over 45 years old, and many in Australia expected to retire over the next 10 years, there are concerns about how adaptable academics will be in the new competitive education environment. ²¹ Graeme Hugo suggests ‘that the lack of opportunities in the Australian universities over the last two decades and/or a decrease in the attractiveness of academic jobs’ ²² will lead to recruitment problems. Most significantly, the time pressure caused by

ongoing bureaucratisation of universities (and the attendant needs for accountability …); and the rise in the consumerist ethos of students … has meant that the paperwork alone (such as new course proposals, course outlines, reading guides, assessment guides and course evaluations) has increased dramatically. ²³

The impact on academics carries across to the students, many of whom no longer have the benefit of being known individually by staff members. For many students, the on-campus experience is a mass one, with lectures for many hundreds of students and, if they are fortunate, tutorials for 30 students. Even the usual two hour a week lecture and tutorial is now considered a Rolls Royce model. In the law school (a post-2007 law school) in which this research was undertaken, it is not uncommon to have 60 students in a classroom. As a consequence, the school has moved to three-hour seminars with larger classes and in most cases no longer holds tutorial groups. For the large number of off-campus students for whom the school caters, the education process can be alienating and isolating. Academics are therefore tasked with overcoming this problem by engaging students through the use of ever-changing new technologies. ²⁴

Various employer and higher education studies decry the lack of skills preparation of graduates — in particular, in the communication skills area. ²⁵ The Council of Australian Law Deans (‘CALD’) Report, Learning and Teaching in the Discipline of Law, notes that

[o]nly a few law schools had really addressed the key issue of Graduate Attributes in relation to assessment, with graduate attributes being systematically mapped against topics and assessment tasks developed with clear and relevant criteria to indicate that students had demonstrated that they had achieved competence in the generic skills areas. ²⁶

Students’ perceptions of assessment ²⁷ are supported in the research, and confirm the approach taken by academics when setting assessment tasks. ²⁸ This approach is out of alignment with the intended goals of bodies such as CALD ²⁹ — that

[d]idactic and other traditional approaches will be supplemented by experiential learning placements, clinical opportunities and simulations, and e-learning. Space will be found for formative education that provides practice and feedback opportunities, a methodology more suited to the development and progressive enhancement of skills and attitudes than the summative approach characteristic of content delivery. By engaging students in critical thinking, guided group-work processes, building reflection skills and journal writing, using role plays and experiencing ethical decision-making dilemmas, not only will the law curriculum be re-invigorated, but deeper learning in the areas of knowledge, skills and values will be promoted. ³⁰

As Table A above shows, assessment tasks set in most Australian law schools are largely not skill-intensive. In this context, it is important to consider the impact of academics’ perceptions on higher education — in particular, the training of law graduates. Since ‘[t]he complex interrelationships amongst students, lecturers, institutions and professions draw together at the point of assessment’, ³¹ the focus of this paper is on the critical juncture of assessment. The research aim was to gather and analyse data in the formative years (for learners, teachers and program designers) of an undergraduate law program. The research sought to discover the key factors that influence academics when it comes to designing, setting and assessing students in the current higher education, institutional and professional context. The question of what these factors are and how they influence assessment needs to be understood if quality education and, therefore, quality graduates are to be produced. Using a grounded research approach, ³² evidence was sought that enables reflection on how assessment is designed and implemented, and what should be considered to improve assessment both of and for learning. A principal objective of the project was to illuminate understanding of what factors (internal and external) drive choice of assessment type and how different assessment practices engage and motivate law students, leading to improved retention/progression and acquisition of graduate attributes. This paper outlines the research methodology; discusses some of the key factors that influence academics’ perceptions of assessment; and finally reflects on concerns academics have about assessment and on future research that should be undertaken.

II  METHODOLOGY

The study focused on assessment practice: how academics approach assessment from their personal perspective; what motivates the creation of particular assessments; and issues surrounding the actual assessment of students. It looked at internal factors such as academics’ perceptions of how students approach assessment and the role of skills and graduate attributes in the assessment process, along with external considerations such as the impact on assessment of the academics’ work environment. Ethical clearance from the University Office of Research and Higher Degrees was approved prior to undertaking the research, and all 13 participants voluntarily consented to be involved in the research. The academics surveyed are all course leaders in first- and second-year core undergraduate curriculum courses in the LLB programme and they work across two campuses. The three authors/researchers were included as instruments in their own research, as they are also core course leaders, along with the other 10 academics involved.

A grounded theory approach was adopted using a qualitative research method which involved 1) a pre-interview survey of the academics; 2) a half-hour semi-structured interview; 3) qualitative information obtained from course evaluations; and 4) a student focus group. The pre-interview survey addressed key aspects around assessment of graduate attributes and skills, motivation for types of assessment instruments and learning objectives that drove academics’ assessment design (see below Table B). The semi-structured interview was then conducted with all 13 participants to follow up on aspects highlighted by the completed survey. Areas addressed included the purpose of assessment and what factors influence assessment design. Student evaluation data was also mined; in particular, for qualitative feedback in the form of open comments addressing issues relating to assessment practices and a student focus group dealing with student perceptions of their assessment experience was held. Sample questions from these instruments are included in Table B below.

Table B Sample of Research Questions

Academics: written survey How important do you believe it to be that the following identified graduate qualities are encouraged through the development of assessment items? a) discipline expertise; b) professional practice; c) global citizenship; d) scholarship; e) lifelong learning. What specific ways do you use to develop students’ awareness of these graduate qualities as part of their assessment practices? Are there other qualities not covered? How would you rank a list of 10 key skills, in order of importance? What specific ways are they assessed? At what level are they assessed? Do you believe that students are aware of how the assessment items develop graduate skills and/or attributes and which of these are being developed? Do the assessment instruments used enable students to adequately determine that they can demonstrate the stated learning outcomes? What factors are taken into account and what ultimately motivates academics in determining assessment items? How authentic do you consider the assessment to be, and to what extent do you believe it engages students in their learning?

Academics: follow up interview What do you believe is the most important purpose of assessment? To what extent do you believe that assessment in the law program should prepare students for professional practice? How important do you think graduate attributes and skills and in particular disciplinary knowledge are to your development of assessment items? How confident are you in your capacity to assess graduate attributes and skills? What do you believe are the roles of 1) the lecturer and 2) the student in assisting students to successfully complete their assessment tasks? What are the factors that most influence you when assessing students — that is, Faculty goals in pass rates; a duty to the profession to ensure students have an appropriate skill level; student reaction; other? continued

Students: focus group questions What are some of the most important skills you learn in your assessment? Are the learning objectives the assessment is addressing made clear? How do assessments within courses across the program relate directly to course learning objectives described in each course specification and the materials? How much do you feel the following are assessed, and how important are each? a) fundamental knowledge related to the law discipline; b) critical thinking; c) communication skills including written and oral; d) problem-solving skills; e) research skill; f) professional/ethical practices; g) cultural awareness. Can you describe an assessment item you most enjoyed or found most stimulating? And why was this?

As mentioned above, the research method adopted a grounded theory approach by seeking exploratory data, focusing on linking emergent data from which links with concepts and theory from the current literature could be made and conclusions for further research could be drawn. ³³ This approach employs the ability to observe and compare participant’s data, to determine what is happening for the person in their domain and to draw tentative conclusions where possible rather than testing a specific hypothesis. A research assistant was employed to oversee the collection of the data from all 13 participants and to conduct the student focus group, so as not to contaminate any of the data. The methodology and the review of the data are based on qualitative research principles such as reflexivity and concern for the consideration of human behaviour in a contextualised manner. A limitation of the study is the small number of respondents and the fact that it is located within only one law school. However, this exploratory research is aimed to share qualitative data which provides insights into law academics’ design and implementation of assessment in an Australian law school.

III RESULTS OF THE RESEARCH

A How is Assessment Perceived by Academics?

The survey and interview demonstrated that factors impacting on academics — such as, workload demands, research output requirements and other demands including student retention and progression — affect their assessment design and thus, ultimately, the production of quality graduates. To move students towards being genuine quality graduates, the literature has demonstrated that academics have to set engaging, skill-rich assessment. ³⁴ However, the environment in which this is to occur is often not conducive to quality, because quality assessment may demand more of the academics’ time and skills. With many of the traditional forms of content-based assessment — such as, exams and essays based on standard-form problems, which are predictable and easily administered — a superficial learning known as subject-based learning occurs. ³⁵ These assessments are in the main assessments of learning, which ensure students have understood the material they have been presented with, rather than assessments for learning. Biggs argues that the standard lecture presentations focused on content not only fail to engage learners but also focus on recall and repetition at a superficial learning level. ³⁶ Higher-order learning outcomes do not result from this teaching focus and deep learning, in which the student not only understands and comprehends the content but can apply and evaluate it creatively, is not achieved. ³⁷ However, the demand, institutionally and professionally, is that students not only be able to demonstrate that they possess skills that employers value, but that they are able to identify where in their undergraduate studies they learnt those skills. Richard Johnstone and Simitra Vignaendra together with Gary Davis et al ³⁸ indicate that most assessment types are content-based while few are in areas such as group, oral, mooting, reflective, peer and self-assessment. ³⁹

So what are academics’ perceptions of the reality — as opposed to the rhetoric — of higher education environments, and of the institutional and government demands regarding quality education? How do they see their role in the new competitive environment and, most importantly, how does this impact on key activities such as assessment? The following are some quotes from academic participants in the research that indicate time and resourcing — the practical realities of academics’ work environments — inhibit setting assessments that may be richer in learning outcomes:

Resourcing: what will address the [assessment needs] in the most efficient way, given constraints on academic time, ie to get the most effect with the lightest hand.

Assessment is one of those things, it’s usually always done under a tight timeframe, like especially at the end of the year when you’ve got to turn them all around and have them back in. So you want to do the right thing by students and give them enough time to read their work fairly, but when you’re running to a tight timeframe you almost think, I’m not marking this as best as I could.

I could think of this really good assessment for … law, but it’s totally labour intensive and there’s no recognition given to that in the workload over someone who sets multiple choice tests. So I think the system totally works against having the best assessment outcome.

Perceptions that research is of greater importance to the institution than teaching will need considerable reframing if Bradley and Gillard’s visions for teaching and graduates are to be achieved. ⁴⁰ This is a theme supported by comments in our research:

[I]n developing assessment items one also has to be mindful of student numbers and the reality that the time spent writing articles would be rewarded and the time spent marking assessment [sic] isn’t within the system. And I think that develops, that drives the assessment items more than graduate attributes and skills.

… because we’ve got other commitments as academics called research and which unfortunately I do believe takes precedence over anything else that we do at university so as a time resource the more engaging you try and make the assessment for the students, the more resource intensive it is, for both the academics and the students.

The other thing that has been a factor in lightening assessment loads has been workload and that has been notoriously the case … where the effect of demands for research on academics has actually encouraged those places to reduce the amount of assessment. Which, you know, the students generally like. But as to whether it brings them to the point where you want to — I don’t know.

… it’s difficult to achieve recognition as good assessment because … it’s relying on the student’s feedback and you’ll get feedback from the most happy ones and the most disgruntled ones, but you know objectively speaking, how can the system really reward good assessment practices? I don’t think it does, whereas if you spend that same time writing six journal articles people say — ‘Wow, what a productive member of staff, look at their publication list’. I think the time spent in this, even though it’s better for student outcomes, is not valued in the system.

As well as driving student engagement, assessment also measures and verifies required forms of student learning for courses and degree programs. ⁴¹ These required forms of learning include the development of specific disciplinary knowledge as well as competencies, attributes and skills identified by university policy and university stakeholders such as employer groups and professional accreditation bodies. ⁴² This raises the question of the different perspectives academics take on what assessment should cover.

B What Should Assessment Be Assessing: Content/Skills/Graduate Attributes?

It is well known that assessment has a direct impact on student study habits. ⁴³ However, in higher education there can often be a disjunction between how academics see assessment and how students see it. Richard James, Craig McInnis and Marcia Devlin suggest that academics first consider the content that has to be taught, then how they should do that, and lastly how they can prove, through assessment, that they have succeeded in having students understand the content. ⁴⁴ So, predominately, academics consider assessment as a way of proving that students have digested the course content and are focused on assessment of rather than for learning. This perspective was well supported by our research and summarised by law academics within the school as follows:

it’s really to see whether the students have fully grasped the principles of the course.

… to test that they actually get the information and that we can say — this student understands this subject and they are ready to move to the next subject of their degree.

… to see that what you have taught has been absorbed and processed, that they’ve understood.

… assessment is the way by which we judge whether a student has understood the important principles in the course …

However, some academics go further and require assessment to be for learning, where the assessment task itself is part of the learning process and students are required to demonstrate skills needed for later use in the workforce as well as discipline knowledge.

I was very driven by what skills can this course give these students and how can I move the assessment towards that.

… make sure that the students have absorbed everything that they’re supposed to have learned during the semester. But not only that, but also to equip them for further on when they are out in practice and just to see that not only have they learned but if so have they picked up skills to be able to research and for correct critical thinking …

… it’s better that assessment[s] prepare them, well give them all the skills. So we teach them how to research, we teach them how to write, we teach them how to think.

… they will need to be able to access the relevant law and interpret that law appropriately. So those research skills are, in a way, just as vital as understanding — if you learn a lot of black letter law, by the time you get through it could be redundant.

The skills identified collectively by the law school as appropriate for students include discipline-based knowledge and key skills for lawyers — such as ethical research and inquiry; problem-solving ability; academic and professional literacy; written and oral communication; interpersonal skills; teamwork; cultural literacy; management, planning and organisational skills; creativity, initiative and enterprise; sustainable practice; and doctrinal knowledge. As shown in Table C below, the research highlighted skills which academics ranked as most important and which are the areas in which the academics tended to focus assessment and thus skill development. Table C shows academics’ combined ranking of skills in this study as broadly reflected in Table A above — the type of assessment students consider they receive in Australian law schools. This correlation confirms that assessments which are creative, interactive and involve teamwork tend to fare less favourably when it comes to assessment practices, possibly as they are seen as more resource-intensive by academics.

Table C Relative Importance of Skills as Determined by 13 Academics in One Law School

This leads to the question of exactly how assessment assists in producing a ‘quality’ law graduate. In an undergraduate law degree the emphasis has been on learning the theory, with the skills to come later in professional practice training. With professional degrees, this can be the cause of some lack of clarity, particularly in a context where higher education is placing greater emphasis on generic skill attainment in quality graduates. ⁴⁵ Some academics acknowledged that assessment should develop the graduate attributes that link with the ideal skills for a student in their professional practice; in fact, giving assessment a ‘double duty’. ⁴⁶

So the purpose for me of assessment, that is most important, is helping the students expand their knowledge, their engagement with the world around them, the lifelong learning process and you know obviously showing that they understand the content to a sufficient level to progress to the next stage.

I think that we should deliver certain practical elements to reassure people that they are learning a degree that is useful, that will prepare them at least to some extent for professional life afterwards.

I think the idea of what they will be doing in practice should drive the assessment.

The assessment or certainly a large chunk of it should be designed around what you are likely to experience in practice.

They should be writing a letter of advice, drafting documents that make them ready for professional practice.

Other academics, however, question how much graduate attributes do, in reality, drive assessment:

To be honest I am not sure about the extent to which examiners think about the attributes when developing assessment. I think the assessment flows from the way in which they prepare the course. It may or may not be that attributes are taken into account in preparation of the course. I think they should be taken into account, but I suspect that people choose different assessment items for all sorts of reasons that may not be attributable to attributes, because their teaching style is the way in which the course has been written.

Interestingly, one academic saw assessment as deflecting students from engaging in deep learning:

Well personally if I was really teaching students I wouldn’t have any assessment at all. I actually hate assessment because students are so focused on their marks rather than learning and so focused on what’s the assessment in this course, what do I have to do to get through it — minimum — that they totally miss engaging with the course content.

This view is perhaps an acknowledgement that many students, as strategic learners, ⁴⁷ will first look at what the assessment tasks are, and so what content will be assessed. Once they know what they have to learn they decide on the best study approach to achieve this — rather than having the wider curriculum engage them, thereby taking their learning to a deeper level. However, such an approach supports the notion that assessment drives student learning and therefore has the task of ensuring that ‘quality’ graduates are the outcome. ⁴⁸ The following answers provide some insight into the qualities some law academics wish to develop in their course objectives, to provide engaging assessments leading to quality graduate outcomes:

Students are required to deliver their oral submissions in front of other students. Student observers are encouraged to value any contributions given with the understanding that each student is informed by their personal environment (home, work etc…); level of education; age; gender; cultural background etc … and that we can learn from each other’s perspectives.

Self reflection and an ability to understand their own personality and its impact on conflict … Their own values for similar reasons and to overcome overwhelming amounts of depression suffered by lawyers.

Too often, creativity and innovation are discouraged by the focus on workloads and meeting end results. Where there is constant oversight and evaluation, fear of failure tends to quash these qualities. The overarching graduate attributes (as opposed to specific graduate skills) at the institutional level where this research was undertaken are identified as: discipline expertise, professional practice, global citizenship, scholarship and lifelong learning skills. The degree of importance accorded to each of these by the law academics indicated a continuing focus on discipline content at the expense of others — such as, global citizenship and scholarship — as shown in Table D below.

Table D Relative Importance of Graduate Attributes as Determined by 13 Academics in One Law School

The implementation of graduate attributes can be problematic when some academics seem to feel a lack of ownership or affinity with the types of graduate attributes promoted by the institution. This factor was noted by Davis et al:

A strong theme emerging from the consultations within the regional meetings was not to discount the importance of the ‘ownership’ factor being present among the academics affected by, and charged with, implementing a Graduate Attributes process. ⁴⁹

Our research uncovered similar results where academics felt the graduate attributes were something being imposed on their course:

But I think there is a certain pattern that your course is supposed to conform to have these graduate attributes it doesn’t matter whether it is suitable or not suitable as long as your course has these … graduate attributes.

Graduate attributes are like signposting they are very important to make sure students learn properly and come out in the real world with the knowledge. Certainly sometimes the ones they use are not always the best they could be.

Well the problem with graduate attributes is that they are high-minded, very general statements that are bandied about, and … students have little time to really deeply engage with them. And probably, you know, I feel that I need to more fully understand them — I understand them in a very general sense.

… graduate attributes and skills, some of them that I think that we have, seem to be a little bit far-fetched or difficult to relate back. I understand where they are going, but to actually reach that, I’m not sure that we’ve got there.

This uncertainty when it comes to graduate attributes, their applicability to the specifics of a law degree and the difference between skills and attributes can lead to a lack of clarity for students. Academics in this study reflected a perception that students may lack an understanding of graduate attributes:

They just focus on the assessment, and aren’t discerning about graduate skills, etc.

Some are, but most are not.

Some are; and have raised them. Most are not.

Varies — depending on student & stage of their degree.

It is not essential that students have a sense of how their skills and knowledge are being progressively developed, but it would be nice.

It is more important that students actually develop the skill; awareness of … the specific skills … and … how they are particularly mapped out in the LLB or JD, is secondary and would even be impressive. It would be rare even for most of the academic staff in any given university to have this awareness.

Students I believe see the courses as providing content rather than skills — they may see that they ‘know’ an area without understanding that knowledge comes with certain skills/abilities.

This perception was confirmed to a degree by the students in the focus group; although some students claimed that this lack of understanding was due to lack of explicit information from academics on graduate attributes:

there are learning objectives in the study guide but we are not taken through those.

… there is no explicit talk about them.

[X has] taken us through them, so I am aware of them … and no one else has taken me through them.

The research indicates that, while statements about graduate attributes at the institutional level are clear at the outset, this clarity is somewhat muddied in the implementation.

C Satisfying the Profession v Keeping the Institution Happy

The academic sits at the centre of the varying demands from the profession, educational institution and government. In Australia, the Legal Practitioners Admission Boards accredit university courses for the purpose of qualifying people to be admitted to practice in a paper-based process that assesses against national rules for prerequisites, or core subjects in law courses. These were set up by the Council of Chief Justices in 1994 under mutual recognition arrangements that specify 12 academic content areas — the ‘Priestley 12’ — that must be covered by a law student. The profession also nominates people to serve on Faculty Advisory Boards or review teams when invited by universities. ⁵⁰ With lower entry requirements and increased competition between institutions, less money and more pressure, academics are being expected to progress a greater number of students suitable for professions that have external professional degree requirements in the pervasive belief that everyone can be taught, if only the academic knows how to bring out the best in the student. This mismatch is borne out by an academic comment in the research:

[P]reparation for professional practice in a law program is a significant feature of the pedagogy of learning … but it’s also important for institutional purposes because without accreditation this program doesn’t run. The universities just do not run law programs that aren’t accredited for professional admission.

Mark Barrow’s research has highlighted a problem when it comes to university degrees with professional qualification hurdles such as law. ⁵¹ In such a mix where the degree is not the ‘work-ready’ entry entitlement to a profession, but a further clinical professional hurdle exists, ⁵² the importance of the ‘theory’ lecturer at the institutional level is perceived to be diminished by the student. Barrow’s research shows that, in this context, assessment regimes tend to demonstrate only that students have ‘absorbed, and could restate, the discourse of the discipline’ ⁵³ and ‘this leads to limited expectations of the expertise of the academic’. ⁵⁴ This could have major impacts on a discipline such as law, where academics may still see themselves (quite correctly) as ‘experts’ in their field. The fact that there is an external, professional admitting authority ⁵⁵ that limits the power of the educational institution and the academic to award the student the immediate status of ‘practising lawyer’ may lead to unacknowledged tensions and uncertainties in the role and purpose of the legal academic.

The problem needs to be considered in the context of the CALD submission to the Bradley Review concerning the ability to demonstrate quality learning outcomes in a measured way:

Through this process, the focus for lecturers will shift from content knowledge and transmission roles to a responsibility for ensuring the achievement of broad and transparent outcomes consistent with graduate attributes. Learning activities will be scaffolded to engage students in well-structured and varied tasks. ⁵⁶

While the differences between demands by the profession, the institution and the government remain largely in the unconscious, the aim of bodies such as CALD ⁵⁷ to change assessment practices of academics will remain difficult. The CALD Report, Learning and Teaching in the Discipline of Law, indicated:

[i]n terms of driving curriculum and teaching via Graduate Attributes, a typical response pattern was had from academics: one-third of staff enthusiastically embracing change; one-third willing to go along; and one-third involved in rejection, active resistance or spoiling behaviour. ⁵⁸

No doubt a disjunction in perceptions can lead to stressors for both the academic and the student. Academic perceptions in our research aligned with Barrow’s findings ⁵⁹ that professional skills training is seen as separate, something that comes after the degree with the clinical legal education course:

[P]ractical skills are something to be acquired in the professional practice course undertaken after the law degree: I didn’t feel that my law degree prepared me for professional practice. It provided a lot of theory … but much of it wasn’t practical. But then its purpose wasn’t to be practical, it was to deliver the theory and general knowledge in relation to the areas of law and the practical element came with either practical training, legal practical training later, or professional practice at its junior stages.

… but not everyone who does a law degree obviously ends up in professional practice. So what I think is it’s better that assessment prepare them, well give them all the skills. So we teach them how to research, we teach them how to write, we teach them how to think. Then they can adapt to professional practice and learn what they might need to do to you know go down to court. We can’t teach them all that because it changes all the time.

Student participants confirmed this clear distinction:

I think assessment is more content driven in law … I don’t know that they test our skills.

… to me legal skills are taught in the legal professional year. What we are doing now is just having our heads crammed with knowledge.

In any law subject you are applying your basic knowledge, concepts and principles to a practical life situation and in exams it’s the same thing. You know, here’s a case study, what’s the issue, identify the issue … what’s the law, how would you apply it and what’s your conclusion.

However, student participants did positively recognise academics whose assessment developed the ‘know how’ and ‘problematise[d] their relationship with the ways of knowing of the discipline’. ⁶⁰ This reflects the claim that student engagement comes through collaborative integrated learning. ⁶¹ Studies indicate that, to engage students, assessment must provide some or all of the following: opportunities for ‘active’ learning; collaboration between students; tasks that are perceived by students to be authentic; opportunities for student reflection; and opportunities for students to manage their own learning processes. ⁶² Barrow’s study suggests that, where there is preparation for work through both skill and discipline attainment, students become engaged. ⁶³ The following range of responses to questions on what students found most useful in assessment practice supports this:

Doing the exercises in class helped a lot …

The lectures … in the computer labs where we were taught to track down cases and legislation which directly related to our assessment at the time.

… to relate the course material to real life.

Excellent use of online interaction via Second Life very beneficial.

Some academics reported seeing the need for ‘work-ready’ skills to be incorporated in learning and assessment:

I think we do need to make sure that students are prepared for professional practice. The other issue is once they’ve graduated from their degree they are expected to do some professional training, whether that be with a law firm or with a legal practice course. So to some extent students get practical training there, but I think we should have some practical training within the law degree.

I think it [professional practice] is a significant component. I think it’s very important that they learn where it can be used in professional practice — not just in a legal practice, but also if they are going to work for a government agency, or for an academic institution, or for a company, you know the broader definition of professional practice.

… there’s obviously difference between theoretical and academic law, and practice law … any sort of assessment should have a theoretical application and also a practical application.

The following comments from student participants in regard to a course assessment clearly identified these skills:

We used team work, interpersonal skills, academic and professional literacy, oral and written communication … problem solving … creativity and enterprise … cultural literacy … ethics … Management, planning and organisational skills.

Authentic assessment tends to satisfy the CALD and professional bodies’ demands by duplicating experiences students are likely to face in their profession, and such assessments overcome some of the issues presented by Barrow. ⁶⁴ Engaging assessments that include skills often require team collaboration; oral presentations; and use of portfolios or journals of reflection on the problem-solving relating to course content, involving application of theory to practical examples: ⁶⁵

An inquiring mind — curiosity. A desire to improve oneself and the world. I put the smorgasbord of topics on the table and invite students to engage with the topics that most engage them — to explore these further in their assessments. I try to maintain enthusiasm that will challenge and infect the students to want to know more — to move them beyond their comfort zone — sometimes this requires throwing out challenging topics or statements for students to respond to. I encourage student participation in the classroom or on discussion boards by putting up weekly discussion topics concerning the area we are learning at the time.

So when I’m shaping a piece of assessment I try and … have it so it’s relevant to the workplace and that’s what students want — they are wanting to know, how am I going to use this.

However, these types of assessment require more commitment of staff time and resources. In a climate in which academics are being asked to restrict their face-to-face contact time and cut down on casual tutors and marking contracts, the needs of the student and the demands of the institution leave the academic who cares between a rock and a hard place:

Well, I’m in a bit of a quandary at the moment, because this year I’ve set one particular piece of assessment for students to have as an option. You know, they can make a choice as to whether they sit at home and read through cases and write a case note on it or they can go and visit a court and I would have thought students would have jumped at the chance of doing a piece of assessment that actually took them into a courtroom environment and got them to sit down and comprehend what was going on … I was really disappointed at the number of students who took up that option. I think it comes back to the point of what is motivating students to do what they are doing at this point in time, in their undergraduate degree studies and unfortunately to the bulk of them it’s a means to an end.

My own enthusiasm and desire for students is to learn something useful and to, in particular, learn how to question and think critically. Also to learn lifelong learning skills by being self directed, sufficient and motivated. This makes more work for me which is not what I first think about in setting assessment … but is becoming more the case.

With universities in the current climate moving to more restrictions on academics, the ability to ‘play’ and access the thinking and creative time needed for genuine engagement is diminishing. Paula Baron notes a heavy burden can be placed on an academics’ time with ‘the necessity to teach skills as well as content in courses; and the tendency to move away from doctrinal scholarship to the comparative, theoretical and sociological exploration of law’. ⁶⁶ Overall, the academics in this research reflected the broader Australian experience of assessment in law schools, with an indicated preference to assess content rather than skills (consider Table A and Table C above). Despite the fact that teaching and assessing skills requires perhaps greater resources and time commitments from academics, many still acknowledge that it is an important aspect of teaching in a professional degree qualification:

Graduate attributes and skills — it’s very helpful to have an idea of the kinds of skills and sort of qualities you are trying to develop and trying to encourage when you’re preparing assessment. So having those in mind beforehand and during the preparation of assessment, as well as during its marking and generally overall for the subject is quite important.

Assignments give the opportunity to assess written communication skills and problem-solving and literacy generally, as do exams; some of the other skills are not so easily assessed.

I think assessing some of them like oral presentations are left to a few courses because of the difficulty of assessing that in the context of the great number of external students that we have.

I think there is a tendency in our law programs to adopt methods of assessment somewhat blindly and a number of courses that have a 60% exam and a 40% assignment in it is too high.

Many academics find that the demands placed on them do not align and so create uncertainties that, with a lack of support, are encouraging some to retire or leave the profession. ⁶⁷ In many cases, incomes do not maintain parity with other professions and the things that made that acceptable to academics are being eroded — such as not being tied to a desk, instead being treated as a professional independent worker who is trusted to get the job done. In a professional degree qualification such as the LLB, there is an obligation on academics not only to the institution of the university but also as officers of the court. Law academics have a professional obligation to uphold the standards of the profession and make sure that graduates will be able to carry out their work to a professional standard. This creates a tension when academics are being required to achieve a certain level of progression and retention of students, taking into account the demand for an increase of 40 per cent in student numbers, including students from diverse backgrounds. Academics interviewed indicated that this disjunction in demand is a cause for concern:

[T]he faculty goals and the pass rate is [sic] totally relevant, a duty to the profession to ensure students have an appropriate skill level should be relevant, but the faculty goals work directly in opposition to that.

There is duty to the profession and to a number of entities; there is a duty to yourself and the student, but to the university and to the legal profession.

Well I’d like to say it’s a duty to the profession to ensure students have an appropriate skill level, but I’m very much aware of the pressure from the faculty that … if we go over a 20% fail rate or pass rate then we’ve got to provide some sort of explanation. I’m not happy with that, I don’t think that’s academically sound but that’s the way it is.

There’s always faculty goals that … you are suppose [sic] to get a certain percentage that are passes and a certain percentage of failures. Nobody denies … those pressures to have those sort of things, but for a lawyer, so far as I am concerned if you have got good students they come out as good students, but you should not just be churning out second or third rate students and passing them off as lawyers, the whole institution gets a mickey mouse image that is something I do feel strongly about.

I’m still most influenced by looking at what the desirable learning outcomes are, but you never actually develop those without an eye on the other things like professional accreditation and institutional demands. The institutional demands here quite clearly are faculty goals and pass rates. Those are necessarily incompatible with criterion referenced assessment, but we’re not unique in having those.

In terms of faculty goals and pass rates … it is something that drives me it’s just a fact of life that … I’ve got to meet those pass rates.

Faculty goals and pass rates, I’m quite resistant to that, but because there’s a big stick and I can get called before an examiners meeting to ‘please explain’, there is that pressure there to make it the easiest for myself. As a teacher we’re very nervous about appeals, we’re very nervous about complaints and these things will drive our assessment.

I am concerned at the idea of faculty … I don’t like the idea that you’re supposed to have a certain amount of high distinction students and a certain number of students in each of the categories thereafter.

Some academics interviewed indicated they took their obligation to the profession as a higher standard to follow:

A duty to the profession, absolutely, I don’t want to have students that are incapable of carrying out the skills that are going to be necessary to be a legal practitioner and those are a whole range of duties to the profession.

I’m very conscious of a duty to the profession to ensure that the students have an appropriate skill level … we want to be quite careful to do a good job and to turn out competent students …

The degree to which academics can meet the new demands in quality learning outcomes for graduate attributes and skills, the demands of the profession and the demands of the higher education institution for retention and progression can be seen to be placing conflicting loyalties on academics. It is clear that these institutional constraints and directions are external factors which have an impact on how an academic assesses students.

D Other Factors Influencing Assessment

Our research identified some other areas of the assessment process that lack clarity, leading to uncertainty for academics when assessing and indicating a need for further research and refinement. These factors include the academic skills needed, and the process that occurs, at the micro level of actual assessment; the contribution and significance of technology in assisting, or otherwise, with assessment; and the role evaluations have in compelling certain assessment outcomes.

1 The Actual Assessment Process

Implementing and assessing skills can be seen as an uncertain art, particularly in a higher education environment that imposes ‘retention and progression’ requirements on academics, which can set up conflicting demands:

I sometimes worry that I’m undermining the graduate attribute skills in completing assessment, because one of the graduate attribute skills is about research and also about initiative, and if I then enter in to moral discussions and email exchanges I am undermining the very attributes that we’re trying to create. And I think that you want to have students that are able to be proactive, who are able to be a bit smart about carrying out an assignment. … that’s a bit of an art in balancing those two needs.

How an academic actually assesses the level of skill attainment seems heavily reliant on the academic’s own judgement, which is in turn based on their experience, as the following quote demonstrates:

I think in general there’s going to be an element of judgement rather than of scientific calculation in assessing whether you’ve actually promoted say the development of critical thinking skills. … but you have to rely on judgement as much as on hard calculation.

Responses about how confident one is in assessing graduate attributes indicate it tends to be related by some academics to the number of years spent either teaching or in professional practice:

Quite confident in that, because I have been a lawyer for umpteen years. I had my own law firm … I have been here for about [x] years. I think I am pretty good at assessing students and applying the graduate attributes.

Confident because I’ve been teaching for [x] years and I know I’m quite pedantic about written work.

I’ve got reasonably confident because I’ve been in the workforce for twenty-odd years and I’m still a practising solicitor.

While the art of assessing skills and graduate attributes would appear, from both our research and from the literature, to be uncertain, the need for academics to grapple with the professional, institutional and government demands to retain and progress an increasing number of diversified students to achieve ‘quality’ graduates remains. This has led to the use of tools such as technology — which some consider a panacea and others an added cause for stress, ultimately affecting assessment and student learning. ⁶⁸

2 Technology in Assessment

With the higher education sector competing in an open globalised market, together with encouragement for increased attendance from diversified student populations, many universities have responded in a similar vein in their organisational changes and uptake of IT. ⁶⁹ With a diversified market, including mature-aged students looking for flexible lifelong learning opportunities to prepare them for uncertain futures, there is a drive for virtual education to prepare quality students for a digitalised world. ⁷⁰ Students are given assessments incorporating digital literacy skills requiring development of self-directed learning styles with the teaching style changing to a facilitative mentoring mode. Collaborative student-focused learning has often been touted as the cure for growing class numbers and workload demands for academics. ⁷¹ This, unfortunately, is not often the experience of academics, who are now expected to become expert mentors, available to assist students one-on-one in a non-time-constrained technological environment: ⁷²

I find that I can spend an enormous amount of time dealing with one or two students, which I feel is not fair to the other students. It also detracts from my engagement with the substantive content and the discipline knowledge. Time that I could be spending on that; I am spending with a few students.

With distance education, students’ evaluation of the academic and their course often occurs through their perception of the teacher’s engagement with the student online and the provision of interactive and Internet-based resources. This is particularly of concern when the evidence shows that 60 per cent of academics in Australia are over the age of 45 ⁷³ and, further, that engagement with and use of technology tends to decline with age. ⁷⁴ For older academics, this is an extra burden as they have to adapt and learn the new technologies, becoming an expert in order to provide interactive and educational virtual environments that can incorporate knowledge and skill training to produce the best graduates. ⁷⁵ Terry Anderson and others note that ‘[s]uch perceptions underline the importance of providing all teachers with effective professional development so that they can confidently establish teacher presence online’. ⁷⁶

There is a need for structural changes such as adaptation to different student cohorts, in particular off-campus students, to adjust to the competitive pressure brought about by globalisation and technology. ⁷⁷ However, in the drive to incorporate technology into the learning environment, academics still aim to create an engaging learning environment that adapts the available technology tools to the teaching needs: ⁷⁸

I’ve become a little more savvy … in the next course I’m writing, about my ability to survive assessment and you know, making it a little easier on myself. I still probably haven’t gone to the full extent with that because I still have to value my teaching, that I’m actually teaching the student something so that is what drives me, but it will be to make sure it doesn’t kill me.

The saviour of technology is one which is problematic and adds its own challenges to effective teaching and design of assessment items.

3 Evaluations

Another concern raised in the research and closely linked to the pressures from the current higher education environment is the evaluation of teaching and its impact on assessment practices. As mentioned at the outset, the push for ‘quality’ graduates means that the systemised language of accountability and number crunching comes into play with a plethora of testing instruments being introduced. Trust is no longer placed in the academic to be professional in their assessment and grading of the student. More quality assurance testing has to occur to justify government expenditure of tax payers’ funds and to work as a carrot type incentive to inspire academics to ensure they produce ‘quality’ graduates. ⁷⁹ Further, the student is often placed in the role of evaluator, and it is questionable just how well-placed they are to make this judgement. ⁸⁰ Stress factors increase when the student evaluations are perceived to arise from poorly designed evaluation systems with perceptions that the highest ratings are based on ‘the entertainment value and ease of the course’. ⁸¹ Much of the ‘quality’ assurance is largely dependent on the capability of the testing instrument to genuinely assess the graduates’ ‘quality’ or the teaching and learning outcomes.

As Vicki Rosser, Linda Johnsrud and Ronald Heck note:

[i]ncreases in competition for scarce resources and a decrease in the public’s trust in higher education practices have resulted in demands for campuses to demonstrate their productivity, effectiveness, and efficiency. Institutions have responded with a variety of data about student enrolment trends, student retention and graduation rates, job and career placement, and faculty workload studies. In turn, this frenzy of measurement is passed on to individual academics who face a range of accountability measures for all aspects of their performance: teaching evaluations. ⁸²

It seems that, while higher education is being induced to diversify its intake and accommodate difference, little is being done to acknowledge these differences in the evaluative tests, such as looking at cultural and socioeconomic factors. ⁸³ Progression rates of students to graduations seemed to have improved. However, understanding the reasons for this is not so clear: it could be new and improved teaching; better or more motivated students; or lower standards and pass rates. Whichever it is, ‘[i]t would seem important that the institutions themselves have the means to understand what is happening in these new circumstances’. ⁸⁴ The following views clearly demonstrate the need for this information to be used wisely:

I have concerns about assignments as to whether they are done by students. Students give you a brilliant assignment and then they sit the exam and they can’t even write English, it is obvious the same person hasn’t written the assignment … I don’t think that is going to achieve anything.

Bradney has expressed concern with the impact of testing on the future of legal education, stating that ‘[a]udit invites academics to treat students as future employees and seeks to treat academics as workers on an assembly line’. ⁸⁵ The terms ‘inputs’ and ‘outcomes’ are frequently used in discussion, and there seems to be a growing move towards assessing outcomes rather than inputs, which understandably are seen as less valid indicators of standards. ⁸⁶ The frustration with the demands is voiced in the following comment:

I think the whole problem is the University is just wanting student retention, wanting a churning out, x number of students … not the quality. I don’t think that should be …

Richard Johnstone and Sumitra Vignaendra’s thorough study reflects an attitude by law academics that student evaluations are often used in staff reviews ‘as a “stick” to “manage” teaching quality in schools’ which only ‘encouraged teachers to stick to tried and trusted methods such as straight lecturing and other forms of “spoon-feeding” in order to ensure that their student “ratings” were high enough to ensure their tenure or promotion’. ⁸⁷ Just how this monitoring and response to feedback is to occur would seem to be largely left to individual institutions ⁸⁸ and, in many cases, the individual academic. Significantly, it is to be noted that, as an academic, interpreting the feedback in a useful way can be an uncertain art:

The other thing which is an institutional demand is student reaction. I don’t think it’s just a perception that universities often respond more favourably to student complaints than they do to academic demands. I’ve seen it too often and every academic is quite sensitive to student reaction … The only thing is that student reaction is the Tower of Babel and you get a lot of different voices, in a lot of different languages … So student reaction is taken seriously, the question is, how do we respond to that and that’s a difficult thing because different students say different things — better students say they don’t like it, most students say that’s okay, that suits me. You know, what do you do?

The Course Experience Questionnaire (CEQ) is widely used in Australia; however, it is administered only at the end of a degree and thus any data obtained suffers from a time-lag effect as far as opportunities to respond, particularly at a course level. ⁸⁹ The different models of evaluation used at the course level, program level, institutional and national level can have variations in their theoretical underpinnings and design and ultimately impact on the approach an academic takes to assessment. Johnstone and Vignaendra conclude that any evaluation of teaching should include a wider focus than just the academic. ⁹⁰ It should take a holistic approach, taking into account the environment within which the teaching occurs,

including the approaches taken by colleagues within the same school, the attitudes of senior managers within the school, the university’s policies towards teaching and the school, and the signals and resources from governments. ⁹¹

IV REFLECTIONS

This research has revealed a number of factors (internal and external) weighing on academics when it comes to their perceptions of assessment which may influence how assessment is set and managed, and thus the types of learning outcomes for students. The research highlights some key areas of tension in assessment, which are supported by the literature. Internal factors, such as how individual academics perceive assessment, what they should assess and how they should assess, and external factors such as conflicting institutional, governmental and professional demands, along with the use of IT and regular evaluations, need to be addressed in a holistic manner.

The perceptions of the group of 13 academics from one Australian law school aid in understanding where the tensions exist for academics, and thus assessment and student learning. The perceptions held, while reflecting a range of positions, highlight the need for further research to assess whether an ability to overcome some of the concerns can be achieved in order to genuinely improve graduate quality through assessment. The implications for student-engaged learning and quality graduates, if they are not met, are considerable.

The solution to these interconnected dilemmas — namely, the institutional demands such as workload, student retention/progression and research output; professional skill requirements and admission hurdles; and government demands for ‘first in family’, low socioeconomic, rural and remote students to become quality graduates — may well be to focus on changes to the environment and context in which academics construct assessment. Taking just one aspect, namely the changes in teaching practice with the introduction of virtual teaching platforms, this encompasses a whole new approach to teaching, with some students expecting academics to be continually available. ⁹² The range of skills now extends across many capabilities; such as, facilitator; technology expert; designer; manager/administrator; advisor/counselor; mentor; assessor; and researcher. ⁹³ The latter aspect does not mean pursuing one’s research interests and obligations but rather researching new and engaging ways to deliver discipline content and skills learning in a virtual environment. Nicole Gillespie, Meagan Walsh, Anthony Winefield, Jagdish Dua and Con Stough conclude, in their study of workplace stress, that ‘the introduction of new technologies (eg, internet communication, web-based and on-line teaching) and software packages increased their workload and contributed to stress’. ⁹⁴ Cownie indicated that another downside of technology is the role it plays in removing secretarial support, as academics carry more of these ‘self-servicing’ duties, including writing and editing their own course materials on sophisticated publishing software platforms. ⁹⁵

Baron also suggests that there is a significant difference between mere survival as a law academic and thriving. ⁹⁶ She suggests that universities should focus on helping law academics thrive and proposes some practical suggestions to move towards this, such as promoting collegiality; practising time management strategies, including factoring in free ‘thinking’ time; having realistic expectations; utilising mentoring; having a supportive management; maximising professional autonomy; and development of the individual academic. ⁹⁷ This is all made problematic by the vision of increasing numbers of students needing greater teaching support, with an added consumer attitude that demands greater academic availability, while all the time faced with a ‘big brother is watching you’ institutional context. ⁹⁸

Studies on academic satisfaction ⁹⁹ indicate clearly what is needed to improve alarming imbalances in work–life experience and stress and depression levels. However, little response seems to have occurred. In fact, the demands and the rhetoric of the higher education ‘agenda-setters’’ ideological commitment to a goal encompassing a free market seems to be driving in the opposite direction to all of the clues for improvement. Baron discusses the need for a creative life to bring about a ‘thriving’ person, while a compliant life requires ‘adaptation and fitting in’, leading to a feeling of futility and that nothing really matters. ¹⁰⁰

Baron links a feeling of wellbeing to a safe social environment.¹⁰¹ This is an environment in which leadership provides a warm, empathetic and attending aspect, encouraging academics to feel part of a collegial group that can communicate and participate in a ‘non-threatening’ atmosphere. ¹⁰² Arguably, if more attention was paid to law academics’ wellbeing, this could have a positive effect on law students and, ultimately, practitioners. How academics deliver the teaching of law can influence this outcome. ¹⁰³ To avoid overwhelming amounts of depression in lawyers and law students, in particular, teaching needs to allow for the expression of personal values and opinions. It needs to put ‘thinking like a lawyer’, dominated by the positive law paradigm, into an overall perspective so that it can be used appropriately and not dominate the curriculum. ¹⁰⁴ Graduate attributes, such as global citizenship, invite more of this in the law curriculum but have a tendency to be undervalued by academics still focused on discipline knowledge of positive law content. ¹⁰⁵

V CONCLUSION

The research undertaken identified issues such as different values impacting the perceived purpose of assessment; what graduate attributes and skills, if any, should be assessed; the impact of institutional and professional demands on what assessments academics will set; and how these determine the types of assessments utilised. All of the factors demand teasing out through further research to more clearly discern the degree of influence each has, in a holistic manner, on assessment and therefore learning outcomes for graduates. This study has demonstrated that there is a variety of approaches in response to the higher education context based on academics’ individual values. However, these responses are within a consistent range being presented in the literature. This reinforces the need to address these concerns to improve assessment and thus the quality of graduates.

Despite the hope, there is still a risk of widening the divide that exists between the rhetoric and the reality. Much of the bridging of this divide has been dependent on the good will of committed academics who value their professional teaching and research enough to keep batting against the odds. However, the brink would seem to be looming closer for many, as the research indicates. ¹⁰⁶ Cownie advocates following the advice of Socrates and examining our lives as academics to determine what we want as individuals. ¹⁰⁷ Do we want to be consumed by ‘the greedy university’ ¹⁰⁸ or do we reject the long hours and settle for less promotion and job satisfaction? These are real questions academics are now being forced to face as they are caught between the rocky rhetoric and the hard reality. The impact of these concerns on the focal point of assessment highlights the need for further consideration of the very real way in which the ‘rhetoric’ of higher education and the ‘reality’ at the coal face is perceived by those in it, thus effecting the ultimate product of higher education — the ‘quality’ graduate.

---

^[*] Pauline Collins LLB (Adel), B Vis Arts (USQ), Grad Dip in Prof Com (USQ), LLM (UQ), Solicitor & Barrister (SA), Lecturer, School of Law, University of Southern Queensland.

^[**] Toni Brackin B Com (USQ), B Ed (Sec) (QUT), M Tax (UNSW), FTIA, Registered Teacher QCT, Lecturer, School of Law, University of Southern Queensland.

^[***] Caroline Hart LLB, BA (UQ), LLM (QUT), Solicitor, Lecturer, School of Law, University of Southern Queensland. The authors would like to thank Professor Michael Robertson and unknown reviewers for their comments on earlier drafts. We would also like to acknowledge our Faculty’s research funding grant which supported the research.

^[1] Denise Bradley et al, Review of Australian Higher Education: Final Report (Commonwealth of Australia Government, 2008) (‘Bradley Review’).

^[2] Ibid; Julia Gillard, Address to the Universities Australia Annual Higher Education Conference (3 March 2010) Department of Employment, Education and Workplace Relations (‘DEEWR’) <http://www.deewr.gov.au/Ministers/Gillard/Media/Speeches/Pages/Article_100303_102842.aspx> :

In January, 2010 the number of low SES applicants increased by 9.8 per cent on January 2009, compared to a 8.2 per cent increase in the number of medium SES applicants and 5.4 per cent in high SES applicants. While this shift can be explained by poor employment outcomes among the young, one of the tragic impacts of the global recession, this shift will be reinforced by our substantial low SES loading, which DEEWR estimates will be worth $540 per student this year rising to $1500 in 2012.

^[3] Margot McNeill, Maree Gosper and John Hedberg, ‘Engaging Students with Higher Order Learning (or not): Insights into Academic Practice’ (Paper presented at the ATN Assessment 08: Engaging Students with Assessment, University of South Australia, Adelaide, 20–21 November 2008) <http://www.ojs.unisa.edu.au/index.php/atna/article/viewFile/385/244> .

^[4] David Boud, ‘Assessment and Learning: Contradictory or Complementary?’ in Peter Knight (ed), Assessment for Learning in Higher Education (Kogan, 1995) 35, 35.

^[5] TEQSA was established by the government in response to the Bradley Review, above n 1, Recommendation 20. See, eg, Gillard, above n 2:

an independent national regulatory body will be responsible for regulating all types of tertiary education. TEQSA will register providers, carry out evaluations of standards and performance, protect and quality assure international education and streamline current regulatory arrangements. It will join together the regulatory activity currently undertaken in the states and territories with the quality assurance activities currently undertaken by the Australian Universities Quality Agency (AUQA). In so doing it will reduce the number of regulatory bodies from 9 (all states and territories plus AUQA) to one.

^[6] Don Anderson, Richard Johnson and Bruce Milligan, Quality Assurance and Accreditation in Australian Higher Education: An Assessment of Australian and International Practice (Commonwealth of Australia, 2000) <http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/archives/quality_assurance_in_australian_higher_education.htm> :

[a regulatory body’s] purpose would be to ensure, for the institutions themselves, for the Australian Government and the general public, and for students, that degrees are all of a sound standard in which the Australian people may have confidence and pride. Its methods would include audits of institutions’ quality practices and for this purpose an independent agency would be needed.

^[7] Bradley Review, above n 1, Recommendations 2 and 4.

^[8] Ibid.

^[9] Denise Chalmers, A Review of Australian and International Quality Systems and Indicators of Learning and Teaching (Carrick Institute, v 1.2, 2007) 69:

Higher education is now more than ever seen as an economic commodity, with increased interest in linking employment outcomes to higher education (employment and graduate destinations). This in turn has led to interest from governments and funding agencies in measuring the employability of students through measures of learning and their employment outcomes …

^[10] See, eg, McNeill, Gosper and Hedberg, above n 3.

^[11] See, eg, John Biggs, Teaching for Quality Learning at University (Open University Press, 2^nd ed, 2003); see also John D Bransford, Ann L Brown and Rodney R Cooking (eds), National Research Council: How People Learn: Brain, Mind, Experience, and School (National Academy Press, revised ed, 2000).

^[12] Graham Gibbs, ‘Using Assessment Strategically to Change the Way Students Learn’ in Sally Brown and Angela Glasner (eds), Assessment Matters in Higher Education (Open University Press, 1999) 44, 44.

^[13] David Boud, ‘Assessment and Learning — Unlearning Bad Habits of Assessment’ (Paper presented at the Effective Assessment at University, University of Queensland, 4–5 November 1998) 1.

^[14] Geoff Scott, Accessing the Student Voice: Using CEQuery to Identify What Retains Students and Promotes Engagement in Productive Learning in Australian Higher Education (Department of Science Education and Training, 2005).

^[15] Richard J Stiggins, ‘Assessment Crisis: The Absence of Assessment FOR Learning’ (2002) 83(10) Phi Delta Kappan 758.

^[16] Richard Johnstone and Sumitra Vignaendra, Learning Outcomes and Curriculum Development in Law: A Report Commissioned by the Australian Universities Teaching Committee (AUTC, 2003) <http://www.cald.asn.au/docs/AUTC_2003_Johnstone-Vignaendra.pdf> 257.

^[17] Gillard, above n 2.

^[18] See, eg, Sir Ron Dearing, Report of the National Committee of Inquiry into Higher Education (NCIHE Publications, 1997) <http://www.leeds.ac.uk/educol/ncihe/> 69:

The health of higher education depends entirely on its staff, whether academic, professional or administrative. There is concern among staff that they have received neither the recognition, opportunities for personal development, nor the rewards which their contribution over the last decade merits. Over the next 20 years, the roles of staff are likely to change, as they undertake different combinations of functions at different stages of their careers. To support and prepare staff for these new working patterns, more focused and appropriate training and staff development activities will be needed.

See also Cath Lambert, Andrew Parker and Michael Neary, ‘Entrepreneurialism and Critical Pedagogy: Reinventing the Higher Education Curriculum’ (2007) 12(4) Teaching in Higher Education 525.

^[19] Bradley Review, above n 1.

^[20] Christine Broughan (ed), Academic Futures: Inquiries into Higher Education and Pedagogy (Cambridge Scholars Publishing, 2009).

^[21] See also Don Anderson, Richard Johnson and Lawrence Saha, ‘Changes in Academic Work: Implications for Universities of the Changing Age Distribution and Work Roles of Academic Staff: An inquiry into the Implications for Universities of Changes in the Academic Workforce and Work Conditions’ (DEEWR, 2002) <http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/changes_in_academic_work.htm> see also Piävi Tynjälä, Jussi Välimaa and Anneli Sarja, ‘Pedagogical Perspectives on the Relationship between Higher Education and Working Life’ (2003) 46(2) Higher Education 147.

^[22] Graeme Hugo, ‘The Demography of Australia’s Academic Workforce: Patterns, Problems and Policy Implications’ (Paper presented at the Monash Seminars on Higher Education, Monash University, 7 September 2004) 23.

^[23] Paula Baron, ‘Thriving in the Legal Academy’ (2007) 17 Legal Education Review, 27–52, 37; see, eg, Gary Watt, ‘The Soul of Legal Education’ (2006) 3(2) Web Journal of Current Legal Issues <http://webjcli.ncl.ac.uk/2006/issue3/watt3.html> .

^[24] See, eg, Alfred Rovai, ‘Building Sense of Community at a Distance’ (2002) 3(1) International Review of Research in Open and Distance Learning 1. An off-campus law student gave the following feedback in 2008 regarding lecture recordings:

The opportunity to listen does make external students feel more like part of the ‘team’. I appreciate any efforts you make to include external students …

^[25] Dennis Pearce, Enid Campbell and Don Harding, Australian Law Schools: A Discipline Assessment for the Commonwealth Tertiary Education Commission (Australian Government Publishing Service, 1987) vol 1, 1 (‘Pearce Report 1987’). Key skills the report identified as important for law schools to focus on included oral expression, advocacy, drafting, negotiation and interpersonal skills. See also Craig McInnis and Simon Marginson, ‘Australian Law Schools after the 1987 Pearce Report’ (Australian Government Publishing Service, 1994); Anderson, Johnson and Milligan, above n 6, who report the need for training in teamwork skills; Dearing, above n 18: white paper on the future of higher education in the UK that indicated a desire to increase student numbers, widen participation and produce lifelong learners.

^[26] Davis et al, above n 16, 58.

^[27] See Table A above.

^[28] See Table C below.

^[29] Johnstone and Vignaendra, above n 16, Table A.

^[30] CALD, Submission No 275 to DEEWR, Review of Australian Higher Education, (2008) 4–5.

^[31] Mark Barrow, ‘Student Assessment and Knowing in Contemporary Western Societies’ (Paper presented at the International Conference of the Higher Education Research and Development Society of Australasia, Miri, Sarawak, 4–7 July 2004).

^[32] See generally Anselm L Strauss and Juliet M Corbin, Grounded Theory in Practice (Sage Publications, 1997); Anselm L Strauss, Qualitative Analysis for Social Scientists (Cambridge University Press, 1987); Barney G Glaser, Doing Grounded Theory: Issues and Discussions (Sociology Press, 1998).

^[33] Strauss and Corbin, above n 32; Strauss, above n 32; Glaser, above n 32.

^[34] See, eg, Jennifer M Brill and Yeonjeong Park, ‘Facilitating Engaged Learning in the Interaction Age Taking a Pedagogically Disciplined Approach to Innovation with Emergent Technologies’ (2008) 20(1) International Journal of Teaching and Learning in Higher Education 70.

^[35] Diana H J M Dolmans, ‘The Relationship between Student Generated Learning Issues and Self-Study in Problem-Based Learning’ (1992) 22(4) Instructional Science 251.

^[36] Biggs, above n 11.

^[37] Diana Laurillard, Rethinking University Teaching — A Conversational Framework for the Effective Use of Educational Technology (Routledge Farmer, 2002).

^[38] Johnstone and Vignaendra, above n 16; Gary Davis, Susanne Owen, Michael Coper, William Ford and Jill McKeough, Learning and Teaching in the Descipline of Law: Achieving and Sustaining Excellence in a Changed and Changing Environment (Council of Australian Law Deans (‘CALD’), 2009) <http://www.altc.edu.au/resource-learning-teaching-law-flinders-2009> .

^[39] Johnstone and Vignaendra, above n 16; Davis et al, above n 38. Both reports suggest a movement away from prescription of content and towards broader skills related to professional and personal attributes, with collaborative professional learning processes used to achieve and sustain long-term curriculum change.

^[40] Bradley Review, above n 1; Gillard, above n 2.

^[41] Kerri-Lee Harris and Richard James, ‘Facilitating Reflection on Assessment Policies and Practices: A Planning Framework for Educative Review of Assessment’ (2006) 3(2) Studies in Learning, Evaluation, Innovation and Development 23, 24.

^[42] Christopher K Morgan et al, ‘Scholarship Neglected? How Levels are Assigned for Units of Study in Australian Undergraduate Courses’ (2004) 29(3) Assessment & Evaluation in Higher Education 283.

^[43] Boud, ‘Assessment and Learning: Contradictory or Complementary?’, above n 4; Timothy J Curnow and Anthony J Liddicoat, ‘Assessment as Learning: Engaging Students in Academic Literacy in their First Semester’ (Paper presented at the ATN Assessment Conference: Engaging Students in Assessment, University of South Australia, 20–21 November 2008).

^[44] Richard James, Craig McInnis and Marcia Devlin, Assessing Learning in Australian Universities (Centre for the Study of Higher Education, 2002).

^[45] CALD, Submission to DEEWR, above n 30, 4–5:

Learning Outcomes: Legal education can be better developed through measuring learning outcomes that have been derived from Graduate Attributes statements and processes that encompass the full range of qualities that will lead to the production of law graduates who not only know how to think like lawyers, but who also know how to perform like lawyers and conduct themselves as lawyers should.

^[46] Boud, above n 13.

^[47] See, eg, McNeill, Gosper and Hedberg, above n 3.

^[48] Ibid.

^[49] Davis et al, above n 38, 60.

^[50] CALD, Submission to DEEWR, above n 30, 93; Davis et al, above n 38, 54:

Beyond broader university-specified Graduate Attributes, in relation to legal education in Australia, the curriculum is required to meet the accreditation standards of the profession. That means that law schools are required to place a strong focus on knowledge of the Priestley 11 content areas …

CALD (with the assistance of a report compiled by Dr Chris Roper as consultant also available on the CALD website: http://www.cald.asn.au/legal_educ.html) moved toward the adoption of standards aimed at enhancing the quality of Australian law schools in all of their diverse endeavours, and to do so by assisting all Australian law schools to strive for and reach a clearly articulated set of standards. Formal adoption took place at the CALD Meeting held on 17 November 2009: CALD, The Standards for Australian Law Schools (2009).

^[51] Barrow, above n 31.

^[52] Anderson, Johnson and Milligan, above n 6. So, for instance, the College of Law course (providing training in practical aspects of a legal office, as a substitute for articles) remains the principal means of satisfying the final registration requirements in NSW. The alternative is the UTS course, which incorporates these matters into the undergraduate course: above note 6, 93.

^[53] Barrow, above n 31, 6.

^[54] Ibid 7.

^[55] Anderson, Johnson and Milligan, above n 6, 6:

Professional associations loom large in the operations of many professional faculties, and in accrediting for the purposes of registration, most appear to focus more on inputs — curriculum content, student staff ratios, contact hours, resources and equipment and so on — than outcomes.

^[56] CALD, Submission to DEEWR, above n 30, 4 (emphasis added).

^[57] Ibid 43.

^[58] Davis et al, above n 38, 61.

^[59] Barrow, above n 31, 6: ‘This disjunction between the institution and the world of work is recognised by the students who diminish the role of the lecturer as a result’.

^[60] Ibid.

^[61] Thomas J Shuell, ‘Cognitive Conceptions of Learning’ (1986) 56 Review of Educational Research 411.

^[62] Biggs, above n 11; Scott, above n 14.

^[63] Barrow, above n 31; see also Stuart Palmer, ‘Authenticity in Assessment: Reflecting Undergraduate Study and Professional Practice’ (2004) 29(2) European Journal of Engineering Education 193.

^[64] Barrow, above n 31.

^[65] Greg Kearsley and Ben Shneiderman, ‘Engagement Theory: A Framework for Technology-Based Teaching and Learning’ (1999) <http://home.sprynet.com/~gkearsley/engage.htm> : ‘Engagement theory is based upon the idea of creating successful collaborative teams that work on ambitious projects that are meaningful to someone outside the classroom.’

^[66] Baron, above n 23, 37, citing Fiona Cownie, ‘Two Jobs, Two Lives and a Funeral: Legal Academics and Work–Life Balance’ (2004) 5 Web Journal of Current Legal Issues <http://webjcli.ncl.ac.uk/2004/issue5/cownie5.html> .

^[67] See Anderson, Johnson and Milligan, above n 6.

^[68] See, eg, John O’Donoghue, Gurmak Singh and Lisa Dorward, ‘Virtual Education in Universities: A Technological Imperative’ (2001) 32(5) British Journal of Educational Technology 511; see also Ron Oliver, ‘Engaging First Year Students Using a Web-Supported Inquiry-Based Learning Setting’ (2008) 55 Higher Education 285.

^[69] John Barnard, ‘The World Wide Web and Higher Education: Promise of Virtual Universities and On-Line Libraries’ (1997) 37(3) Educational Technologies 30; Richard Cornell, ‘The Onrush of Technology in Education: The Professor’s New Dilemma’ (1999) 39(3) Educational Technology Research and Development 60.

^[70] See also Peter Goodyear et al, ‘Networked Learning in Higher Education: Students’ Expectations and Experiences’ (2005) 50 Higher Education 473; Beverley Oliver and Veronica Goerke, ‘Australian Undergraduates’ Use and Ownership of Emerging Technologies: Implications and Opportunities for Creating Engaging Learning Experiences for the Net Generation’ (2007) 23(2) Australasian Journal of Educational Technology 171, 180–1: ‘Additionally, the teaching and learning styles that middle aged university teachers find intellectually stimulating are unlikely to maintain the interest of today’s “Net Genners” …’.

^[71] See also Martina A Doolan and Trevor Barker, ‘Measuring the Effectiveness of StudyNet in the Context of Online Learning Environments’ (Paper presented at the Computer and Learning Conference, Belfast, 8–10 April 2003).

^[72] See also Karen M Fitzgibbon and Norah Jones, ‘Jumping the Hurdles: Challenges of Staff Development Delivered in a Blended Learning Environment’ (2004) 29(1) Journal of Education Media 25.

^[73] Hugo, above n 22.

^[74] David White, Results of the ‘Online Tool Use Survey’ Undertaken by the JISC Funded SPIRE Project (16 March 2007) Tall Blog <http://tallblog.conted.ox.ac.uk/wp-content/uploads/2007/03/survey-summary.pdf> , cited in Oliver and Goerke, above n 70.

^[75] Richard Smith and Pamela Curtin, ‘Children, Computers and Life Online: Education and the Cyber-World’ in Ilana Snyder (ed), Page to Screen: Taking Literacy into the Electronic Era (Allen and Unwin, 1997) 211.

^[76] Terry Anderson et al, ‘Assessing Teaching Presence in a Computer Conferencing Context’ (2001) 5(2) Journal of Asynchronous Learning Networks 1, cited in Gail Wilson and Elizabeth Stacey, ‘Online Interaction Impacts on Learning: Teaching the Teachers to Teach Online’ (2004) 20(1) Australasian Journal of Educational Technology 33, 35.

^[77] O’Donoghue, Singh and Dorward, above n 68, 512.

^[78] Heinz Ulrich Hoppe et al, ‘Guest Editorial: Wireless and Mobile Technologies in Education’ (2003) 19(3) Journal of Computer Assisted Learning 255.

^[79] Chalmers, above n 9, 69:

There are concerns expressed by researchers and higher education institutions about the impact of national/sector performance indicators on the autonomy and diversity of institutions. While there are clear trends emerging of greater oversight and desire for standardised measures of learning and effectiveness at the national level, this trend should be interpreted cautiously. The more promising measures and indicators are those that are situated in institutional practice.

^[80] Johnstone and Vignaendra, above n 16, 335.

^[81] Nicole Gillespie, Meaghan Walsh, Anthony Winefield, Jagdish Dua and Con Stough, ‘Occupational Stress in Universities: Staff Perceptions of the Causes, Consequences and Moderators of Stress’ (2001) 15(1) Work and Stress 53, 65.

^[82] Vicki Rosser, Linda Johnsrud and Ronald Heck, ‘Academic Deans and Directors: Assessing Their Effectiveness from Individual and Institutional Perspectives’ (2003) 74(1) Journal of Higher Education 1, 1.

^[83] Anderson, Johnson and Milligan, above n 6.

^[84] Ibid 3.

^[85] Anthony Bradney, ‘The Quality Assurance Agenda and the Politics of Audit’ (2001) 28 Journal of Law and Society 430, 441–2.

^[86] Anderson, Johnson and Milligan, above n 6.

^[87] Johnstone and Vignaendra, above n 16, 335.

^[88] Ibid; CALD, The Standards for Australian Law Schools, above n 50, [7]:

7. Course evaluation

7.1 The law school has course evaluation procedures that regularly monitor the curriculum, quality of teaching and student progress, and identify and address concerns.

7.2 Measures of, and information about, Graduate Attributes are used as feedback to course development.

^[89] Simon Barrie and Paul Ginns, ‘The Linking of National Teaching Performance Indicators to Improvements in Teaching and Learning in Classrooms’ 13(3) Quality in Higher Education 275.

^[90] Johnstone and Vignaendra, above n 16.

^[91] Ibid 344.

^[92] Mark Bullen, ‘Participation and Critical Thinking in Online University Distance Education’ (1998) 13(2) Journal of Distance Education 1.

^[93] Gail Wilson and Elizabeth Stacey, ‘Online Interaction Impacts on Learning: Teaching the Teachers to Teach Online’ (2004) 20(1) Australasian Journal of Educational Technology 33, 38.

^[94] Gillespie et al, above n 81.

^[95] Cownie, above n 66.

^[96] Baron, above n 23.

^[97] Ibid 47–52.

^[98] Gillespie et al, above n 81, 62.

^[99] Ibid.

^[100] Donald Winnicott, Playing and Reality (Routledge revised ed, 2005), cited in Baron, above n 23, 30.

^[101] Baron, above n 23, 31.

^[102] Thomas Gordon, ‘Group-Centred Leadership and Administration’ in Carl R Rogers (ed), Client-Centred Therapy (Houghton Mifflin Company,1951) 320, cited in Baron, above n 23, 31.

^[103] Baron, above n 23, 35.

^[104] Lawrence S Krieger, ‘Institutional Denial about the Dark Side of Law School, and Fresh Empirical Guidance for Constructively Breaking the Silence’ (2002) 52 Journal of Legal Education 112.

^[105] See above Table D.

^[106] Gillespie et al, above n 81; Cownie, above n 66.

^[107] Cownie, above n 66.

^[108] Jan Currie, Patricia Harris and Bev Thiele, ‘Sacrifices in Greedy Universities: Are They Gendered?’ (2000) 12(3) Gender and Education 269.