McNamara, Lawrence --- "Why Teaching Matters and Technology Doesn't: An Evaluation and Review of Web-base Lectures" [2000] LegEdRev 7; (2000) 11(2) Legal Education Review 175

Why Teaching Matters and Technology Doesn’t: An Evaluation and Review of Web-based Lectures

	Home \| Databases \| WorldLII \| Search \| Feedback Legal Education Review

INTRODUCTION

The development of Australian law teaching in recent years has been characterised by critical and reflective approaches to educational theory and practice. These developments have transformed teaching through a critical consideration of what happens in law school classrooms. As institutional moves towards flexible delivery alter the landscape of higher education, the classroom-based focus of law teaching is under challenge; it has become increasingly necessary for law teachers to explore non-traditional modes of course delivery, including teaching using the Internet.

In a 1999 project called Starting Out: An Introduction to Law and An Introduction to Flexible Delivery, I sought to explore the ways in which law teaching might be carried from the classroom to the World Wide Web.¹ The project replaced the weekly face-to-face lecture in a first year Bachelor of Laws subject with a ‘web-lecture’ delivered over the Internet. A core facet of the change was the development of a web- based teaching strategy from the ground up, inspired by the developments in traditional law teaching and looking to fundamental educational objectives as the starting point. The strategy employed in the web-lectures used questions to guide students through the readings, rather than transposing lecture notes from a face-to-face context onto a web page. It relied upon a combination of explanation and questioning, through which it was hoped that students would be both compelled and inspired to engage with the course materials and themes in a meaningful way. The rationale for this strategy was grounded in educational objectives of deep, holistic and student-centred learning. In an earlier article, I documented in detail the development of the teaching strategy.²

This paper – a companion to the first – is an evaluation and review of the web-lectures and the teaching strategy which was applied. Student responses to the web-based format and to the specific teaching strategy are addressed in sections two and three. In section four, the project review turns to the educational objectives and examines how the student responses might inform course design with a view to enhancing student learning approaches and outcomes.

My aim in this article is not only to describe and analyse the experience during autumn semester 1999, but also to extrapolate from the project and the literature to identify some problems and potentials in the shift to web-based flexible learning. The evaluation is placed in the context of the theoretical and evaluative literature on teaching, learning and flexible delivery with a view to offering a constructive critique of teaching practices, new technologies and the relationships between these and the contemporary administration of higher education.

The arguments advanced are both specific and general. With regard to the former, it will be argued that where web-lectures are employed the key implication for course design and structure is the need to reconceptualise (rather than replace) the lecture and establish the web as a teaching and learning resource within a broader student-centred framework for inquiry. There is, however, a more significant and more broadly conceived argument which drives the paper: in the push for increased flexible delivery, the core concern should not be technology but the objectives and practices of teaching. The point, it will be suggested, is to make the technology work for teaching, not the other way around. The project review and the conclusion offer some critical and cautionary thoughts on the possible relationships between pedagogy and the Internet in a climate of higher education management where good teaching all too frequently appears to occur in spite of – rather than because of – the drive towards flexible delivery.³

Evaluation Strategies

The main evaluation mechanism was a survey completed by students in the last tutorial class of semester where 15 to 20 minutes were set aside specifically for this purpose. A total of 133 surveys were completed; this constituted 73.9% of the total cohort of 180 students in Introduction to Law. Additional strategies for evaluation included student focus groups4 and the lecturer’s journal. The student surveys are the principal source of data in the analysis which follows. The focus group discussions are less frequently referred to because the comments made in those groups often related to organisational issues and service-provision specific to the university.

Identifying the Issues

No significant themes emerged from the first three of these categories.⁵ Student comments were for the most part concerned with the web-based format of teaching⁶ and the content of the lectures.⁷ In discussing the web-based format I have also addressed the issues of Internet access and use which from a teaching and administrative viewpoint were crucial in ensuring that students were able to access and use the teaching medium, and important for course design with regard to how they used the medium. The latter category is addressed in section four. The comments on lecture content raised issues that were directly relevant to the teaching strategy, offering the most useful material for insight into student learning experiences in a self-directed and web-based context.⁸

TEACHING AND LEARNING WITH A WEB-BASED FORMAT

The level of student satisfaction was on the whole quite high. Approximately one third of the respondents thought the web-based lecture delivery was very good or excellent, while 60.9% of students thought it was good, very good or excellent. Just 5% of students were not satisfied, with another 12% approving but finding significant drawbacks.

	% of responses
Table 1 Overall Satisfaction
Under the heading “overall”, students were asked to circle one of the following responses to the question, “How satisfied were you with the web-based lectures in Introduction to Law?” 133 respondents answered this question.
Not satisfied at all	2.3
Not satisfied in important respects	3.0
OK, but there were significant drawbacks	12.0
Generally it was OK	16.5
Generally it was good	24.1
Generally it was very good	22.5
Generally it was excellent	14.3
No answer/incorrect response	5.3

The degree and distribution of positive and negative comments, and the comparison of the web-based lectures against face-to-face lectures in other subjects are consistent with the overall ratings. Approximately two-thirds of the total 464 comments provided were positive. The distribution of those comments reveals 57% of surveys to have offered three or more positive comments, while only 15.8% of surveys offered three or more negative comments. 67.7% of surveys offered either no negative comments or just one negative comment.

Table 2 All Positive and Negative Comments: By Distribution
The percentage of the 133 surveys which carried 0, 1, 2 or 3 comments (positive and negative):
No of positive comments	%	%	No of negative comments
0	10.5	28.6	0
1	17.3	39.1	1
2	15.1	16.5	2
3	57.1	15.8	3

Comparison with Face-to-face Lectures

By way of comparison with other subjects using face-to-face lectures, the student feedback was very positive: of the 181 comparative comments offered, 72.4% were positive with regard to the web-based program in Introduction to Law in comparison to the face-to-face lectures in other subjects. Overall, the web-based lectures compared very favourably: 69.1% of the 55 respondents who made a direct comparison indicated the lectures were either good and compared well, or very good and better than face-to-face lectures in other subjects.⁹

Table 3 Comparative Comments – Direct Comparison
	Number of comments	% (of 55 direct comparisons)
Not as good	4	7.3
No different	1	1.8
Not much different	1	1.8
OK	10	18.2
Good/compared well	15	27.3
Very good/better	23	41.8
Excellent/far superior	2	3.6

Why Did Students Like or Dislike the Web-based Format?

Table 4, shows student opinions on the web-based lecture format. The figures in the first three columns are drawn from responses to questions asking students to identify positive and negative features of the web-based lectures in Introduction to Law. The column on the far right denotes responses to the question asking students to compare the web-lectures in Introduction to Law with face-to-face lectures in other subjects; where responses offered meaningful parallels, they have been included, though the right-hand column is not exhaustive.

Table 4 Web-based delivery of lectures: positives and negatives

	Number of comments	% of total positive comments	% of total negative comments	Number of comparative comments
*Positive comments*
Convenience (include work at own pace & access from home)	66	22.5		52
Can refer back to notes anytime	12	4.1		8
‘Ready made notes’	12	4.1		28
Allows concentration on content and avoids preoccupation with note-taking	9	3.1
Ability to get lecture in advance	9	3.1
Modern style of teaching	5	1.7
Equality for slow note takers	4	1.4
*Negative comments*
Too impersonal / prefer face-to-face	16		10.0	27
Notes not available far enough in advance	14		8.7
Improvement is necessary	6		3.7
All material not on one page	6		3.7
Does not facilitate student-student interaction	2		1.2
Lecture notes only available over the web	1		0.6

The positive comments overwhelmingly expressed a liking for the convenience of the web-based lecture, including the ability to work at one’s own pace and the ability to access the lectures from home. This was reflected in the focus groups where, for instance, the following view expressed by one student was strongly supported by others: “I loved the idea of the lectures on the Internet because, to be honest, I didn’t have to stay back for a lecture and I could do it whenever I liked at home.”

The negative comments showed a substantial number of responses – 10% of the total negative comments – favouring face-to-face teaching and finding web-based delivery too impersonal. This was emphasised more in the comparative questions, which revealed a range of concerns on this issue. The far right-hand column shows that the main comments regarding the strengths and weaknesses of the lecture program were generally consistent with those which emerged when students were asked to compare the web- lectures with face-to-face teaching in other subjects.

Face-to-face Teaching Versus Web-based Teaching

The value of face-to-face teaching in the subject was clear from student responses which overwhelmingly indicated that the cohort found the weekly seminar classes to be the most helpful part of the course.

Table 5 Lectures, Seminars and Materials
Students were asked: “In undertaking the subject Introduction to Law, how helpful were the following?” 133 students responded. All figures are in percentages.
	Web- lectures	Seminars	Materials
Not very helpful at all	5.3	0	0
A bit helpful	6.0	1.5	3.7
OK	25.5	9.0	27.8
Helpful	33.1	30.1	39.9
Very helpful	29.3	58.6	27.8

The table should not be seen as indicating a simple preference for face-to-face teaching; the open-ended and comparative comments make it clear that the absence of face-to- face teaching was clearly picked up by students for a range of reasons. These issues are discussed further below.

Internet Access

Student Computing Competency

Student computing competency was both a pre-requisite and an objective of the web-based lectures. As a pre-requisite, almost all students commenced the course with adequate skills; less than a dozen students attended the instructional classes at the beginning of semester.¹⁰ By the end of semester, computing competency had increased: half the cohort could use the Internet with confidence at the start of semester, and two- thirds were confident by the end of semester. 93.2% of respondents stated that they were at least comfortable with basic use of the Internet by the end of semester, up from 77.4% at the beginning of the year.

It is perhaps worth noting that some students had difficulty in accessing Adobe PDF files if the Adobe Acrobat reader was not already on their computer. This was raised as an issue in focus groups where students working from home said they had difficulty downloading the Adobe Reader software and had to use the university’s computing labs in order to access the few PDF files that were used on the web site.

Accessing the Web Page

Access questions are relevant not only to the viability and utility of teaching across the web but also to equity concerns.

Table 6 Point of Access
Point of access to subject web page. Figures represent the percentage of survey respondents.
	University computing labs	Law library computing labs	Home
0-2 times during semester	44.3	38.1	26.3
3-7 times during semester	20.3	25.5	10.5
8 times or more during semester	33.1	33.1	60.2

The results show that just over 60% of students were accessing the site predominantly from home, with a further 10% accessing the site from home on a regular basis.¹¹ While this no doubt prompted a deal of positive feedback concerning the convenience of web-based lectures, it cannot be permitted to overshadow the significant use of university computing facilities. Any shift towards more comprehensive web-based teaching cannot occur without adequate resourcing and monitoring of institutional computing facilities lest web-based components of courses become a resource only for those who can afford computing and Internet access from home.¹²

Using the Lectures

Students should ideally have accessed the lectures at least once weekly, or perhaps more often if the lectures were read and printed at different times. The surveys indicated that 91.4% of respondents accessed the lectures at least once weekly. This is in contrast to approximately a 70-80% lecture attendance for Introduction to Law in previous years.¹³ It appeared from tutorial participation that students had generally read the lectures prior to their seminar classes; some of the focus group participants indicated that students in Introduction to Law seminars appeared slightly better prepared than in other subjects.

Printing and On-line Use

The substitution of face-to-face lectures with a web-based lecture was not intended to be interactive but was at least intended to encourage students to make use of the HTML format and move between links within the Introduction to Law site as well as to explore sites beyond that by providing links in web-lectures.

The pattern of web use shown in table 7 suggests that students did not work on-line but tended to consider the Introduction to Law site as a medium for convenient access to a static course component. The lecture format lent itself to this and so the result is not surprising. On the other hand, the lectures often contained links to other pages which students would most likely wish to print (eg, assignment questions, or a self-assessment marking guide); it would have made the most sense for students to at least read through the lecture prior to printing so that they could ascertain whether other pages needed to be accessed. Nonetheless, 24% of students accessed the lectures more than once weekly, possibly indicating that students would print the lectures out and then return to them later to move through links and other sites. The improvement in computing competencies also indicates that students may have utilised the web-format more than the printing figures suggest.

It will be suggested below that in light of the teaching strategy in Introduction to Law, the student preference for printing the notes immediately does not at all detract from the value of the lectures as an educational tool.

A Success?

The web-based format appears to have been for the most part a success with regard to student responses. For a pilot program with a teaching strategy and delivery format that departed substantially from traditional teaching methods, the results suggest a program that has proved to be a viable and successful mode of delivery for the lecture component of the subject. While the “convenience factor” appears to be the key motivator for student satisfaction, the responses to the teaching strategy are indicative of a more complex reaction to the project.

STUDENT RESPONSES TO THE TEACHING STRATEGY

Comments on the Web-lectures

The most striking feature of the comments on the lecture content is the polarisation of student responses to the teaching strategy. Of 185 comments, 110 (59.5%) were positive and 75 (40%) were negative. The breakdown of the types of comments made shows the divisions more clearly (table 8 on next page).

Of the comments regarding the lectures as a source of explanation, 54 were positive and 36 negative (60% to 40%). The comments concerning the use of questions showed approximately the same ratio: 28 to 22 (56% to 44%). The polarisation of student opinion is apparent from the comparison of conflicting views on the quantity and quality of explanation. For instance, 15 respondents regarded the notes as comprehensive and 13 respondents considered the notes to be a good summary, while 12 expressed the view that more content or quantity was needed; 24 students indicated that the lectures provided a clear explanation, while 13 thought there needed to be either more or clearer explanation.

From a teaching perspective, it is pleasing that a reasonable number of students found the lectures to be a good source of explanation – particularly since in many cases students were required to reach the core of the explanation by themselves. It indicates at least a degree of success with regard to the self-direction students needed to employ and the assistance they derived from the web-lectures in doing so.

The use of questions and references to the readings prompted an even more significant division of opinion, but supports the success of the teaching strategy in a manner consistent with the above analysis. Once again, approximately half of the students who addressed this issue considered the questions to have been useful. In the focus groups the use of questions was generally well received, though students noted that when there were too many questions they tended to ignore them and pass over the points. These comments highlight the need for careful structuring of the material.

Table 8 Lecture Content: Positive and Negative Comments (breakdown)
	Number of comments	Total positive comments	Total negative comments
• Content and Explanation		54	36
(+) Clear explanation of material	24
(+) Each lecture a good summary	13
(+) Comprehensive notes	15
(+) Pleased with content	2
(-) Clearer/more explanation needed	13
(-) More content/quantity needed	12
(-) Notes too lengthy/too specific	3
(-) Too much legal jargon	3
(-) Notes unhelpful	2
(-) Did not highlight important points	3
• Use of Questions and References		28	22
(+) References to readings helpful/ useful; it put the readings in context	18
(+) Questions encourage meaningful thought	10
(-) Use fewer questions – more statements and information needed	18
(-) Supply more information rather than references to readings	4
• Appearance of Lectures		18	0
(+) positive aesthetics of notes (headings, font, etc)	18

Explaining the Polarisation

The surveys do not offer any immediately apparent explanation of the polarisation of student opinion with regard to the lecture content, but there are numerous comments in the comparison with face-to-face lectures in other subjects which suggest that the negative responses regarding the lectures may perhaps be explained at least in part by the demands which arise from the self-learning emphasis which underpinned the teaching strategy.

Learning and Self-learning

A number of comments concerned the degree of self-learning required of students. Self-learning for some students was generally perceived as a negative aspect of Introduction to Law, its perceived absence presumably being a positive feature of other subjects.¹⁴ The negative attitudes with regard to the self-learning process supported by the web-lectures and students’ expectation or desire for information alone is perhaps the most likely catalyst for the negative comments with regard to the lecture content.

Table 9 Comparative Comments: Learning and Self-learning
The question of comparison yielded 69 negative comments. From these, the following opinions can be discerned:
	No of comments	% (of 69)
With web-based lectures the onus is on the student to work. This can lead to laziness. Face-to-face “forces” a student to work.	9	13.0
Web-based lectures made learning more difficult – too many questions without answers.	5	7.2
Web-based lectures involve more self-learning [the comments indicate this was a negative notion]	2	2.9
Web-based lectures cannot explain concepts as well as face-to-face. This encompasses both basic and complex issues.	13	18.8

The positive and negative comments on the lecture content, combined with the comparative comments, are suggestive of the following explanation of the survey results. The web-lectures guided students through the materials quite closely; in order to understand the lectures, students were required to read the materials in more depth than they otherwise might have – that is, to engage in a self-learning process to a greater degree than in traditional lectures. The benefits were that they should have attained a more thorough grasp of the readings, as well as building analytical and critical reading skills. The positive responses in the surveys might then have been provided from students who read the materials thoroughly (and who would possibly read the materials thoroughly whatever the format of teaching). The lectures would be of most benefit to these students. On the other hand, the high degree of close reading and self-directed learning required would mitigate against those who rely primarily on lectures (rather than readings) as their source of knowledge in the course. For these students the lectures would be lacking in explanation or content. This may prompt the negative responses to the survey questions regarding the content of the lectures.¹⁵

Insights from the Literature

It appears from the literature that for a range of reasons an emphasis on self-learning does not always prompt the enthusiastic student response which might be desired by teaching staff; Le Brun and Johnstone note that

Three samples of student responses to self-learning strategies are documented by Clark,¹⁷ Jones and Jones¹⁸ and Scott, Buchanan and Haigh.¹⁹ A polarisation not dissimilar to that found in the Introduction to Law evaluation is apparent in these reports: “nearly half” the students in the Jones’ study preferred conventional lecturing²⁰ and while Clark does not provide a figure, he clearly identifies streams of satisfaction and dissatisfaction with a self-directed approach.²¹

Clark’s study points to several reasons for negative feedback regarding self learning. First, the reasons for dislike of student-centred self-learning approaches lie most frequently in the area of learner confidence, with many students uncertain as to whether they are “learning the right thing” and even enthusiastic students feeling that they had not accomplished what they might have in a conventional mode.²² Learner confidence also related to the flexibility which students feared would allow them to miss out on important information.²³ In the evaluation of the Introduction to Law project, these concerns arose in one of the focus groups when a student expressed a desire for at least some face-to-face lecturing. When asked why, the response was an emphatic, “We want assurance from the lecturer himself. That we’ve understood the right thing, what he means us to understand.”²⁴

Secondly, Clark found that the context of the self-learning approach mattered; students’ reactions depended on their experiences of teaching and learning in higher education.²⁵ This is consistent with the conclusions of Scott, Buchanan and Haigh with regard to student experience in both prior and parallel courses compared with the student-centred focus in the researchers’ course at the University of Waikato:

The third factor identified by Clark was that the impact of the change itself was seen by some students as too demanding; “plunging into the deep end” is not appropriate and it appears students need not only guidance but also to be made aware of the motives and strategies for new forms of teaching and learning.²⁷ Fourth, there was a perception by some students that they were being asked to do the work; students perceived it not as empowerment but as “a denigration of their educational experience”.²⁸ Finally, the issue of student priorities appeared to underpin some of the negative opinion; if students are empowered in their learning, they may have numerous reasons (including the demands of other subjects) for rejecting a self-learning emphasis.²⁹

Jones and Jones found that students who preferred conventional learning “enjoyed the lecturer providing them with the information and felt that the presence of a lecturer stimulated their interest to learn”.³⁰ Such students also stated that the explanation from a lecturer encouraged students to think critically, and lecturer enthusiasm stimulated learning interest, though many acknowledged that lectures could be uninteresting.³¹ External commitments such as part-time work also affected the ability of students to devote adequate time to a self-learning strategy.

Scott, Buchanan and Haigh also identified the vocational orientation of younger students as dominating their understanding of what employers seek and do not seek in graduates. As a result, they value generic attitudes and skills rather than processes of learning.³²

The results of these reports are not used here to explain the survey results in Starting Out, but do suggest that the polarisation is not out of the ordinary. They go some way towards explaining possible reasons for student dissatisfaction and are suggestive of issues which might be considered in remedying the concerns about negative attitudes. At the very least, they indicate concerns which might be expressly addressed in future projects. A significant aspect of the project review turns on the ways in which course structure and materials might be altered so as to enable students to better manage a self-directed unit; this theme is taken up in part four.

Feedback

The web site included a ‘feedback’ link which could be accessed from any screen on the site. It allowed students to make comments or ask questions directly of myself as subject coordinator. They could do so anonymously if they wished, and could request that I respond or not respond. The feedback link represents one of the most troublesome issues in the evaluation. I discuss it under the heading of student responses to the teaching strategy because, like the reflections on the lecture content, it offers some insight into the extent to which students actively engaged with the course itself. Accessing and use of the link was generally very poor, but the reasons for this are difficult to establish.

Table 10 Feedback Link: Access and Use
Figures represent percentage of total survey respondents.
	Regularly (once a fortnight or more)	1-4 times during semester	Never
Looked at feedback link	15.5	33.8	50.3
Contacted lecturer using feedback link	9.5	28.6	63.1

The feedback link was at least to some extent a positive feature of the web site. Approximately 50% of respondents looked at the link, and 36% of respondents used the feedback link at least once during semester, though only about six comments offered feedback on the substance or style of teaching. The use of the feedback link is more problematic, however, in the lack of access and use.

The Absence of Feedback

A negative spin on the statistics shows that the feedback link on the web page was never even accessed by 50% of the survey respondents. This seems a very high figure, especially in light of the fact that 77% of students at the beginning of semester and 93% by the conclusion of semester were comfortable with at least basic use of the web, meaning that the lack of accessing the feedback link cannot be attributed to a lack of ability to do so.

The lack of interest remained even when the following was included in the week six lecture with a view to eliciting at least some feedback from students:

Evaluation of Web Lectures

The shift into web-based delivery of classes is occurring across the university and will play an increasing role in higher education. This is the first time that a law subject at Macarthur has used the Internet to replace the delivery of face-to-face lectures, but it will certainly not be the last. One of the aims of the project evaluation is to find out what students think of the lectures.

• What is good?

• What is not so good?

• Are there things that are good and should not be taken out?

• Are there things that are good but need to be improved?

• Are there things that are no use at all, or perhaps something that is not there at all but you think would be helpful?

• How does it compare with other subjects which are using the web to offer some form of lecture notes to support the face-to-face lectures?

• Would it be better to have a face-to-face lecture? Why or why not?

• Does it matter in any case? (“I don’t care either way” is a response we are still interested in if that is what you think!)

Your input into the evaluation process is important because the feedback from this subject will be written up to offer a guide as to how web-based delivery might be used in other subjects (which you will have to do in future years).

Please use the feedback link to let me know what you think.

Thanks,

Lawrence

Explaining the Lack of Comment

While it was a positive sign that there were not streams of negative feedback flowing in, it was disappointing that there was not more interest in it. How might this be explained? I will advance six possible factors which may have contributed, though I am reluctant to claim that any of the following hypotheses are adequate explanations for the absence of feedback. The issue is important because it is suggestive of lack of student involvement in the constitution of their learning context.³³

First, the notion that the feedback link is on-line is a little illusory given that approximately half the students just printed the notes and read them later, thus when reading the request for feedback generally not being on-line at the time. Nevertheless, around 40% of students read the notes briefly on line, and around 10% read them more thoroughly on-line. In addition, the high number of students with Internet access at home (roughly 60%) means that for many students it is not demanding to convey feedback through the web page even if they have printed the pages earlier. The printing of notes may be one factor which contributes to the lack of feedback but would not of itself appear to be a satisfactory explanation.

A second potential explanation is that the web-based nature of the lectures de-personalises the subject coordinator. The survey responses indicating a dislike of the impersonal nature of web-based teaching may also indicate a reluctance to engage with it. Perhaps it seemed that there was no-one to respond to. This, however, also seems an inadequate explanation. I was on campus regularly, taught one seminar group, and my office was directly opposite the teaching room where four of the other seven seminars were held. In addition, I had been to all seminar groups twice in the first two weeks of semester.

A third explanation might be that these are students in their first year at UWS Macarthur and they may have the impression that this is simply how the subjects run. This, however, is inconsistent with the enrolment structure under which at least 60 students were also completing three other LLB subjects, some of which had no web materials, while others (in addition to face-to-face lectures) had lecture notes on a subject web page. That is, at least 60 students were aware that there was a great difference between subjects with regard to the delivery of web-based materials, but still no students responded.

A fourth possible explanation is that students are simply too busy getting themselves through the degree to engage in the construction of teaching and learning. The issue of external commitments was identified in the Jones and Jones study as a factor contributing to negative views of self-directed learning.³⁴

Fifth, it is possible that students simply did not care how the web page ran, at least insofar as if it was adequate, they had nothing to say. While this may appear inconsistent with the degree of effort that students put into responding to the survey, the survey was administered in the final seminar classes with 10 to 15 minutes set aside specifically for that purpose.

A sixth explanation might be that students, for whatever reason, saw no point in responding to the requests for feedback. There is nothing to indicate that this is the reason, and on the contrary the wording of the request suggests there is indeed a point to responding, but the possibility needs nonetheless to be raised.³⁵

PROJECT REVIEW: RE-THINKING THE FRAMEWORK FOR STUDENT INQUIRY

Reflections on Internet Use and the Teaching Strategy

The evaluation of the Starting Out project suggests that on the whole it was viewed by students as a very successful move into flexible web-based delivery of the lecture component of Introduction to Law with regard to technology as well as teaching and learning. It is also clear that the technology used for on-line delivery needs to be simple and straightforward, easily accessible and economically viable for students. For all this, the results of the student survey are frequently suggestive not of great shifts in student learning or a newfound enthusiasm for Internet-based teaching, but rather of a desire for little more than the convenience of accessing lectures at the time and place of one’s own choosing. In this light, can it be said that the project was successful in terms of teaching and learning?

The measure of success is difficult to define. I have no evidence that students either individually or as a cohort performed better when the lectures were web-based than in previous years of face-to-face lecturing, but there was certainly nothing to suggest that performance was worse. It was my impression at times that students seemed to have a better grasp of the readings with the web-based lectures, and it was suggested in the focus groups that students in Introduction to Law were better prepared and engaged in more meaningful discussion of the readings than was the case in other subjects with face-to-face lectures. On the other hand, this did not necessarily translate into improved examination results over the entire cohort.

The measure of success I would like to employ is whether students engaged in deep learning, or at least whether Introduction to Law impacted positively on students’ approaches to learning which will develop further as they progress through their degrees. Ultimately, I do not know. The evaluation suggests, however, that the use of the web in no way detracts from that possibility and, more positively, the Internet appears to be a medium which holds the potential to impact significantly on student approaches to learning. But this impact will not be the result of technology: if the key themes of the evaluation were to be drawn together in one sentence, the Starting Out project points to the core conclusion that the most fundamental questions of web-based instruction concern not the technology but the practice of teaching.³⁶ It is not the medium that matters, but how one teaches within both the opportunities for adventurous teaching that the web provides and the constraints of the technology which removes us from the classroom.³⁷

With this in mind, the somewhat polarised student responses to the teaching strategy are reasons to further develop and rework the project, not reasons for dismantling it or retreating from the challenges of teaching an LLB program which fosters deep and lifelong learning while using the Internet as a medium for partial flexible delivery of law subjects. There should not be a rejection or watering down of a self-learning emphasis, but nor can resistance and dissatisfaction simply be ignored – there were, it seems, a number of survey respondents who did not consider that they had learned effectively through the web-lectures in Introduction to Law.

How might all students in the course – those satisfied and those dissatisfied – be able to learn more effectively using the web-based program, and to more successfully undertake a course where they engage in a meaningful and rich introduction to the law? Moreover, might it be the case that a web-based component in a course could facilitate – more effectively than conventional teaching formats – a process of deep learning? Drawing on the experience of the Starting Out project, this review both identifies and represents a fundamental shift in the orientation of flexible delivery in Introduction to Law. Where the project aims at the outset consisted of the substitution of face-to-face lectures with web-based lectures, the review argues for the complete discarding (or reconceiving) of the concept of a lecture in the subject.

A Self-learning Orientation

A self-learning orientation needs to remain at the heart of any subject which endeavours to have students engaging in deep learning and to instill in students a desire and capacity for lifelong learning. This is axiomatic in a constructivist understanding of learning: that the most meaningful forms of learning – learning as the abstraction of meaning and learning as an interpretive process aimed at understanding an outside reality – can be undertaken by the learner her or himself.³⁸ Candy explains that in the constructivist paradigm knowledge is not “something external to be ‘mastered’” but is “an internal construction or an attempt to impose meaning and significance on events and ideas.”³⁹ Hence, learners

Deep learning will accordingly occur best through strategies which involve student-centred learning. The point is to:

In this light, two revisions to Introduction to Law can be envisaged, which might go some way towards countering the negative reaction and simultaneously improve the pedagogical aspects of the subject.⁴³

Teaching Students about Learning

First, the self-learning emphasis could be made more explicit. That is, there needs to be a clearer and more frequent explanation to students why there “are too many questions without answers,”⁴⁴ and why the web-lectures differ from the lecture process in other subjects. This could be done by compiling a short package of materials consisting of extracts from the teaching literature in order to make clear the learning process which students are expected to undertake and to offer them guidance as to how they can go about it. Such a package would ideally be referable to the degree program as a whole so that students might, in the words of Le Brun and Johnstone, “understand what is (or could be) happening in their classrooms.”⁴⁵ Taylor argues that

The learning materials then become a part of the process by which the course content is learned, making clear the rationale for different processes of teaching and assessment. In short, an express statement of self-learning strategies may make the process more reflective for the participating students.⁴⁷ This reflectivity may enhance and enrich their learning processes.

Restructuring Course Design for Mixed Mode Teaching

Second, if the self-learning emphasis is shifted from the periphery to the centre of course design, Introduction to Law becomes as much about the learning process as about the course content. Further, the content of the course can possibly be driven by and derived from the learning process itself.

In order to shift deep learning and self-learning emphases to the centre of course design, the entire course structure and emphasis needs to change, not just the web component. The question then becomes not how to replace face-to-face lectures with web-lectures, but what one does with all of the teaching and learning processes in the subject. The first step in a restructure is to dispose of the lecture as a teaching process. The substance of what was called in the Starting Out project a “web-lecture” should be renamed as, for instance, a web-guide. The web-guide would not take up administrative functions, but act as a guide to the course readings, interspersed with questions. In part it would function as a map, assisting students to navigate their way through the readings, in part as a set of roads, requiring students to put them together so as to form a map or framework within which they can locate each of the course themes and readings.

There would need to be some explanatory content within the web-guide to enable students to make sense of the readings and to emphasise the key themes and issues. But this would not make it a set of lecture notes. The web-guide, like the web-lectures in 1999, would remain a weekly instalment, enabling the teaching dialogue to build on queries, issues and current developments as they arise. It would not be a pre-written study guide. It would enable the subject coordinator (presumably the author) to monitor students’ grasp of issues and engage with students on a regular basis.

The removal of the lecture would occur in both name and concept. The nomenclature of components is important because it aims to remove a sense of loss, which may be associated with web-based teaching. That is, the seminar would be expressly re-established as the point of face-to-face shared experience and human community in teaching and learning. The second change to the course, an express restatement and re-structuring of the learning project itself, would make this more than an illusory re-naming of a teaching component.

Achieving a Unity of Teacher and Student Perceptions

The re-worked model turns on a framework for student inquiry, which pulls student-centred learning from the periphery to the centre of course design, structure and content. The teaching components have previously been a guide to the readings with a view to understanding the course themes. The reliance by students on the teaching components for their knowledge rather than on the readings has been problematic. The web-lecture sought on the one hand to avoid any further shift toward this tendency and on the other to develop self and deep learning.

The most significant aim of the suggested re-structure would be to provide a course which would give students a reason to perceive learning approaches and requirements differently: to perceive the course as a process which facilitates and requires deep, holistic and active student-centred learning. Ramsden identifies the perceptions of students as being crucial in the process of learning:

To this end, the framework revised as suggested here would aim to achieve a mutual understanding of what constitutes learning with regard to the content of the course and the development of foundational legal and study skills.

A Proposed Framework for Inquiry

The proposed framework would shift the course content towards student inquiry by stating the entire course not as a set of topics about which questions will be asked at the end, but as a set of four to six broad and thematic questions stated at the outset. The project for students over the 13 weeks of semester would be to be able to answer these questions. The final examination, worth 50% of the subject mark, would consist of questions derived directly and closely from each of the thematic questions.

Students would be told that it is their responsibility to work out how to answer the questions, but that to assist them in their own endeavours, the teaching staff will provide a selection of resources for teaching and learning: course readings, a weekly two-hour seminar, a weekly Internet web-guide, threaded discussion groups through the web page,⁴⁹ and a set of materials on higher education learning and teaching.

There is concern that confronted with a set of questions, students may engage in surface rather than deep learning.⁵⁰ The holistic approach and the express requirements of the course would hopefully address this possibility. In addition, the thematic course questions would be framed so that they would not allow for a surface learning of the materials. The use in the seminar classes of reflective exercises on the relationship between the different questions would also hopefully avoid a tendency toward surface approaches.

Assessment

It is well established that the driving force behind learning is assessment to the extent that “the form and nature of assessment often swamps the effect of any other aspect of the curriculum.”⁵¹ Assessment methods – both what is assessed and how it is assessed – “tell … students what we think is important.”⁵²

The close, if somewhat unorthodox, relationship between the course structure and the main assessment task is directly aimed at integrating the processes of teaching, learning and assessment, removing the idea of assessment as a threat. Instead, a constructive and open assessment task which is inherently linked to the course objectives sets out to make positive use of student perceptions of the significance of assessment. In this revised course design, rather than being merely a method of grading and certification, the course structure and assessment task together form a strategy for teaching.⁵³

How the Inquiry Framework Addresses the Evaluation Concerns

Such a revised framework for student inquiry would address the evaluation concerns in several respects.⁵⁴ First, the loss associated with the replacement of face-to-face teaching would at least to some extent be alleviated by re-centering the seminar class as the contact point for students. The re-naming of the web-guide and concomitant restructuring of the inquiry framework would not place the absence of a lecture as a loss. Rather, the lecture would be extinct as an entity.

Second, while students might still be reluctant to engage in self-learning, the process of learning in this way should become more manageable as a result of being more expressly stated and more reflective throughout. With students able to see the point of the subject structure and increasing their ability to answer the course questions as they proceed through the teaching components, the self-learning emphasis would hopefully have noticeable gains for them.

Third, the cohesiveness of the course as a whole and the links between materials, seminars and the web-guide should become more apparent as students focus their attention on the various aspects and draw them together themselves. This more directed focus should hopefully address student concerns that the web-lectures were not sufficiently explanatory; in effect, it would remove the point of the question because of the shift in emphasis towards student-centred learning. The more appropriate survey questions to ask would be along the following lines:

• did the web-guide effectively assist you in understanding the readings and answering the course questions?⁵⁵

• after working through the course materials with the web- guide, were you able to answer the seminar questions for each week?

Fourth, while the discussion groups would be threaded by the teacher to ensure that a question is clearly identified, the discussion between students would enhance the student- centred nature of the learning tasks. The weekly web-guide would review the discussion of these questions. This may address in part the concerns of learner confidence which were raised in the evaluation, providing both direction for students and dialogue with them.

In line with the above points, the proposed model would build on the strengths of the Starting Out project and address the key concerns identified in the evaluation.

CONCLUSION

In closing I will draw from the Starting Out project and the review in section four some themes and conclusions which might be considered in flexible teaching and learning projects beyond that in Introduction to Law and perhaps beyond law teaching generally. With regard to the specifics of subject structure and design, the argument advanced has been predominantly one of reflection rather than prescription; the suggestion has not been that the Starting Out project presents either the best or the only way to approach flexible delivery of lectures. The point has been rather that reflection upon teaching objectives, strategies and practices both in flexible delivery and a course or subject as a whole might enrich the teaching and learning process such that a move into flexible delivery can act as a catalyst for constructive change. It is hoped that the project and review explore in a meaningful way some of the processes through which this might occur.

The claim was made at the outset that it is teaching and not technology which matters. While that observation may at first blush seem a little trite, I would argue that for two reasons this is not so. The first reason is that the attempt in this article has been to give some concrete form to what might otherwise remain an abstract point and in doing so to illustrate the rationale for developing a course in a particular way. Hopefully the paper provides food for thought in the development of subjects which may – or must – be flexibly delivered. The emergence of the Internet as a teaching tool is not always welcomed by academics (or is perhaps welcomed by some for the wrong reasons), but the experience in Starting Out suggests that it need not be to the detriment of teaching or learning. The need for reflective teaching exists regardless of the media employed; teaching over the Internet should not be considered a mere extension of another medium but is better conceived of as a different way of teaching and learning. The web does not make lecturing more or less difficult, but it makes lecturing different.

The second reason why it is important to stress that it is teaching that matters rather than technology lies in the very fact that the rapid shift to flexible delivery appears all too frequently to be driven by concerns of budgets, technology and marketing – rather than by teaching.⁵⁶ The implications of this incongruity form a defence of teaching in a budgetary climate where little more than lip-service is paid to the constitution and passing on of knowledge by any means other than research.⁵⁷

A defence of teaching in flexible delivery operates in different ways depending on one’s predisposition to and motivation for the use of new technologies. For those reluctant to let the Internet into the teaching process, there is a need to realise that it provides genuine opportunities for teaching and learning, many of which require little technical skill and can still foster positive learning outcomes. There is a need also to realise that where staff have (or have access to) substantial technical expertise, new and creative use of interactive or pioneering technology might well be combined with pedagogical objectives such that the outcome is an excellent form of Internet-based learning. Such projects will, however, require adequate resources and time.

At the other end of the spectrum, where flexible delivery is defined in terms of cost efficiency or pure technological advance, there is a need to recognise that pedagogically valuable use of Internet technology may not necessarily be interactive, and may not necessarily utilise the very limits of computing capacities. It may instead seek to develop better ways of using the now established media of the web or discussion pages. Where teachers draw on technology for flexible delivery, the provision of financial or other resource support should not always be dependent upon the extent to which any flexible delivery project will advance the frontiers of interactive media; the focus should be on the relationship between a flexible delivery project and the development of effective and meaningful teaching and learning strategies.

The Internet is a tool which holds great potential for teaching and learning in higher education, but only if teaching and learning remain the fundamental objectives. The questions which need to be asked do not turn on the capacity of the Internet to perform different functions at ever-increasing speed and volume, but on what we as teachers might best be able to do with the wide range of capabilities new technologies offer.

This article requires the same acknowledgements as its companion Part I piece: L McNamara, Lecturing (and not Lecturing) Using the Web: Developing a Teaching Strategy for Web-based Lectures (Flexible Delivery in a First Year Law Subject, Part I) 11 Legal Educ Rev 149. I would also like to thank Amanda Halpin for research assistance in the evaluation process.

^[1] The project was funded through the UWS Macarthur “Flexlearn” grants scheme with support from the UWS Macarthur Faculty of Law (now the UWS School of Law). The grant applicants were myself (as project coordinator) and Professor Robin Woellner, Dean of the Faculty of Law at UWS Macarthur.

^[2] L McNamara, Lecturing (and not Lecturing) over the Web: Developing a Teaching Strategy for Web-based Lectures (Flexible Delivery in a First Year Law Subject, Part I) Legal Educ Rev.

^[4] Two student focus groups were run at the conclusion of semester, though prior to the final examination. Four students participated in each group. The facilitator was a Faculty staff member who had taught the subject in previous years but had been on leave all semester and had not taught any of the students in Introduction Law or in any other subjects. Each focus group discussed the Introduction to Law web- lectures for approximately 30 minutes.

^[5] Computing problems (of which there were a number concerning browser compatibility at the start of semester) rated only nine responses, which was the highest of any of the various comments made across these three categories.

^[6] The web-based format attracted 38.6% of the 303 total positive comments and 32.3% of the total 161 negative comments (the high degree of concern here was consistent with the direction of the survey as a whole).

^[7] Lecture content attracted 36.3% of the total positive comments and 46.5% of the total negative comments.

^[8] The survey did not use the expression “teaching strategy”, nor was it expressly used in lectures. The results that follow in this section nevertheless relate quite clearly to the teaching strategy employed in the subject.

^[9] The student comments were open ended but have been categorised in the analysis. In the focus groups, the web-lectures were generally regarded as better than face-to-face lectures in other subjects, though the reasons varied from convenience alone to more substantive comments such as, “It ties in the readings with the actual lecture material. Other lectures are just general principles and then you have to go and do the readings. I wouldn’t say it was a disadvantage at all doing it [over the web].”

^[10] The last three years have seen a dramatic increase in student computing competency at the commencement of the course. The low numbers of students attending such classes would have been unthinkable with even the 1997 or 1998 intakes. The competency figures on entering the course are markedly higher than those described by Mark Freeman in 1997: M Freeman, Flexibility in access, interaction and assessment: the case for web-based teaching programs (1997) 13(1) Aust Jnl of Educ Tech 23, at 28. At the commencement of their subject 61% of Freeman’s students had little or # experience using the WWW, while at the start of semester in Introduction to Law only 20.3% of students had never used the web or were not confident at the start of semester. Thompson et al express a concern that entry-level students will not possess the requisite competencies and argue that the literature supports their fears: R Thompson, J Winterfield & M Flanders, Into the world of electronic classrooms: a passport to flexible learning (1998) 29(2) Br Jnl of Educ Tech 177, at 177-79. The experience in Introduction to Law does not fully support that fear; perhaps the rate of change with regard to fairly straightforward Internet use is changing rapidly.

^[11] Freeman (1997), id, again provides a point of comparison. In June 1997, 10% of his students had Internet access from home. Of the survey respondents in Introduction to Law, it appears that approximately 70% had access from home, with that access being suitably convenient for 60% of the respondents to make it their principal point of access. While the increase is rapid and substantial, it cannot be presumed that it will continue to rise at this rate, nor can it be assumed that students will all wish to use their home access (factors working against home access as a preference could include the quality of printing at home).

^[12] For instance, when printing through university systems, students will often pay up to 20 cents per page, which over a semester may add up to $30 extra (per subject) to student costs after lecture notes and assignment questions are printed from a web page. This, in addition to often expensive law texts, represents a substantial increase in costs for students. This may be particularly so where the web page is in addition to rather than in substitution for face-to-face teaching and thus there are no associated savings (for example reduced transport costs to university because students need to attend campus on fewer days than would otherwise be the case).

^[13] It may be wise to be a little sceptical of this result. It is possible that the 47 students who did not attend the seminar classes in the final week (or at least not in the first 15 minutes when the surveys were completed) also did not access the page as regularly as those who did attend the final tutorials.

^[14] This is particularly worrying in flexible delivery where, as Nikolova & Collis note, “the active learner assumption is axiomatic.” I Nikolova & B Collis, Flexible learning and design of instruction (1998) 29 Br Jnl of Educ Tech 59, at 60.

^[15] A second area of possible correlation may relate to the academic standing of the particular respondent. The survey did not ask respondents to identify their anticipated grade in the subject and so # conclusions can be drawn in this respect.

^[16] M Le Brun & R Johnstone, The Quiet (R)evolution: Improving Student Learning in Law (Sydney: Law Book Company, 1994) 89-90. See P Candy, Evolution, Revolution or Devolution: Increasing Learner Control in the Instructional Setting, in D Boud & V Griffin, Appreciating Adults Learning from the Learners’ Perspective (London: Kogan Page, 1993) 162-63.

^[17] R Clark, Student Opinion of Flexible Teaching and Learning in Higher Education, in W Wade et al eds, Flexible Learning in Higher Education (London: Kogan Page, 1994) 137-38, notes numerous positives, including the ability to work at one’s own pace, the opportunity for students to gauge their own starting level depending on prior experience, and the refreshing experience of a different and student-centred approach to learning.

^[18] A Jones & D Jones, Student orientations to independent learning (1996) 15(1) HERD 83.

^[19] J Scott, J Buchanan & N Haigh, Reflections on student-centred learning in a large class setting (1997) 28 Br Jnl of Educ Tech 19.

^[20] Jones & Jones, supra note 18. They note, however, the complexity in identifying correlations between learning and teaching strategies and learning outcomes.

^[24] Oliver & Omari also highlight the value students in their study placed on the face-to-face component of teaching: R Oliver & A Omari, Using online technologies to support problem based learning: Learners’ responses and perceptions (1999) 15 Aust Jnl of Educ Tech 58, at 77.

^[27] Clark, supra note 17, at 145. On preparing students, see also T Greening, WWW support of student learning: A case study (1998) 14 Aust Jnl of Educ Tech 49, at 55-56.

^[33] See P Ramsden, Learning to Teach in Higher Education (London: Routledge, 1992) 62-63; Le Brun & Johnstone, supra note 16, at 100-12.

^[35] One reviewer of this article suggested that in spite of the request for response, “students are seldom altruistic and will only do what will ultimately help them get ‘better marks.’” While there is nothing in the evaluation to suggest that this was the case in Introduction to Law, it would be consistent with the recognition that student approaches to learning are strongly driven by assessment, and thus by marks.

^[36] This is consistent with the Open University’s research findings that “the real key to the successful application of technology is good teaching.” P Thomas et al, A holistic approach to supporting distance learning using the Internet: transformation, not translation (1998) 29 Br Jnl of Educ Tech 149, at 161. The research by Nowaczyk, Santos & Patton takes an approach, which starts from the significance of technology, looking at “the effectiveness of multi-media as a positive influence in the learning process” (at 367). They explore student perceptions of video and graphics but do so from a technological and psychological viewpoint. The outcomes they describe (at 378-81) are nevertheless framed in students’ perceptions of the ways in which the materials and their content – not the media itself – influence the student learning process. The authors prefer, however, to discuss their conclusions in terms of the media: R Nowaczyk, L Santos & C Patton, Student perception of multimedia in the undergraduate classroom (1998) 25 Int’l Jnl of Instructional Media 367.

^[37] This is one of Owston’s three key themes in his discussion of the Internet as it might be used in primary, secondary and higher education: R Owston, The World Wide Web: A Technology to Enhance Teaching and Learning? (1997) 26(2) Educ Researcher 27, at 29-30.

^[38] P Candy, Self-Direction for Lifelong Learning (San Francisco: Jossey Bass, 1991) 250-51, drawing on R Saljo, Learning in the Learner’s Perspective II: Differences in Awareness Report of the Institute of Education (Goteborg, Sweden: Institute of Education, University of Goteborg, 1979) # 77, at 19.

^[40] Id at 250-51. Candy draws on the conceptions of learning identified by Saljo of which the most sophisticated are learning as the abstraction of meaning and learning as an interpretive process aimed at understanding reality.

^[41] Le Brun & Johnstone, supra note 16, at 90 (footnotes omitted). See further at 90-97.

^[43] The difficulties for many students go # doubt beyond what can be countered by such general strategies, or even within the academic environment alone. For a discussion of motivation and emotional aspects of learning and their impact on self-learning aims, see Ann Brown, Motivation to Learn and Understand: On Taking Charge of One’s Own Learning (1988) 5 Cognition and Instruction 311.

^[45] Le Brun & Johnstone, supra note 16, at 47 state that they wrote not only for law teachers but also for law students.

^[46] P Taylor, Reflections on students’ conceptions of learning and perceptions of learning environments (1996) 15(2) HERD 223, at 235. The last phrase is especially pertinent given the absence of feedback through the web-lectures. See also B McCombs & R Marzano, Putting the Self- Self-Regulated Learning: The Self as Agent in Integrating Will and Skill (1990) 25 Educ Psychologist 51.

^[47] There is perhaps an implicit assumption here that this process will continue after the foundation subject. Mark Israel, Teaching criminology through interview-based assignments [1997] LegEdRev 7; (1997) 8(2) Legal Educ Rev 141, at 144, notes that the literature suggests academics often encounter resistance from colleagues in pursuing such projects. This is to some extent inherent given the necessary freedom of academia. There is perhaps also the research-driven selection and promotion criteria; I suspect that few law academics would disagree with the proposition that in spite of the rhetoric about how much teaching matters, what really matters within the academy is research (especially due to the funding implications) and this is never clearer than at the point of promotion and recruitment. With the pressure to research so strong, teaching innovation or change is often bound to take second place.

^[48] Ramsden, supra note 33, at 62-63 (emphasis in original); for an illustration of the significance of perceptions in a reading task, see 41-42.

^[49] A discussion page would be threaded by the subject coordinator, but not contributed to by staff. It would provide a forum for students to learn from each other. Allison Brown provides an excellent analysis of collaborative learning through discussion pages: A Brown, Designing for learning: What are the essential features of an effective online course? (1997) 13(2) Aust Jnl of Educ Tech 115, at 118-22.

^[50] Such an outcome may parallel the situation described by Ramsden, supra note 33, at 63-64. See also the discussion in part I of this article, McNamara, supra note 2.

^[51] D Boud, Assessment and the Promotion of Academic Values (1990) 15 Studies in Higher Educ 101, at 103.

^[54] The benefits listed are concerned with teaching and learning. Other benefits would also arise; for instance, printing costs for students are reduced by removing the administrative aspects of the lecture from the web-guide and into a different section of the subject web page. Most administrative matters would not require printing. This would go at least some way to addressing equity issues and cost.

^[55] This should have perhaps been the question asked in the survey for Starting Out.

^[57] This should not be taken as acquiescence to the present quantum or distribution of research funding in higher education, nor agreement with the mechanisms for the determination of such funding.

Table 7 Lectures: Printing and On-line Use
Usually just printed the lectures and read them later	51.9%
Usually just read them briefly on-line and then printed them	39.7%
Usually read them thoroughly on-line and then printed them	2.3%