Journal of Law, Information and Science
|
|
Home
| Databases
| WorldLII
| Search
| Feedback
Journal of Law, Information and Science |
|
DR. ALISTAIR INGLIS[*]
For four decades computers have been holding out the promise of making learning much more interactive, exciting and effective. Yet up till recently, the reality has generally fallen far short of the promise. The high cost of computing power, slow response times, limited graphic capabilities, and in the case of on-line computing, slow transmission speeds, have greatly limited the capabilities that could be delivered in a working environment. As a result, computers have generally been relegated to playing supporting roles. In recent years, rapid advances have been made in the capabilities of technology. With the escalating demand for non-traditional forms of education, computer-based methods of delivery now have the opportunity to come into their own. The question we should now be asking is: ‘Are the systems we have available capable of doing the job?’
The concept of using computers for the management of learning, as distinct from the delivery of instruction, has been promoted since at least the late 1970s. However, most presently-available systems are either designed for local use or are based on architectures which are rapidly becoming out-of-date. If technology is going to start to deliver on its promises to education, then the champions of technology need to rethink the functions that computer-based systems are likely to be called on to perform.
This article examines the directions in which education is moving at the tertiary level. It considers what implications these developments have for the design of systems for managing student learning. It then draws from these to try to anticipate some of features we should expect of computer managed learning systems of the future.
When I first put forward the abstract for this article, my intention was simply to play the role of ‘Devil’s Advocate’. My intention was to put those of you who see yourselves as evangelists for CML to the test of defending some of the well-honed arguments in favour of the virtues of CML against a number of the more recent challenges that have been brought against it. I thought that that might lead to some worthwhile debate and perhaps help throw up some new ideas. I certainly didn’t see the role I had chosen for myself as a particularly risky one.
However, the title of my paper happened to catch the eye of the Program Committee chair. He thought that it might be an idea to start off the program by challenging some ‘sacred cows’ and that my paper looked like the one to do it. He asked whether I would be willing to have my paper brought up to the front of the program.
Well, this wasn’t quite the innocuous role I’d been intending to play. However, as I was a member of the Program Committee I felt that it wouldn’t be ‘good form’ to refuse.
However, that wasn’t the end of it. Having locked me into a commitment to lead off the paper sessions, the Program Committee then suggested that I make use of the collective wisdom of the conference presenters to add more fire power to the attack that I was proposing to make on some of the cherished beliefs of CML evangelists.
Well, by now, I was starting to wonder what I had let myself in for. After all, it’s one thing to play ‘Devil’s Advocate’ in the comparative safety of one of the parallel sessions. It’s quite another to find oneself cast in the role of the Devil himself as far as a conference hall of ‘true believers’ is concerned.
So let me say at the outset, while those of you who are evangelists for CML may find what I want to say difficult to accept, I speak to you as one of the believers. However, I am a believer who recognises that to succeed one must be willing to confront one’s failures.
I want to begin by helping you to recall how CML began.
Given the remarkable pace of development in the field of microcomputers in recent years, it requires some effort to imagine what it must have been like working with computers before the time that the microprocessor was invented. Yet it was back in that era that the idea of computer managed learning was first conceived. It is worthwhile recalling how computers came to be applied to the management of learning.
It should be recalled that computers were used in education even before they came into general use in business. Most of the early computers were installed as research machines in universities. They were used in a variety of exploratory studies including investigations of the possible applications of computers in teaching. Those who have been around in education long enough to have some knowledge of the early history of the educational application of computers will know that a number of major projects trialing the use of computers for delivery of instruction were initiated in the late sixties and early seventies. These included, for example, the TICCIT Project, funded by the National Science Foundation at the Brigham Young University and the Plato Project supported by the Control Data Corporation at the University of Illinios (Barker and Singh, 1982).
At that time, computers were also being used quite separately for machine-scoring of multiple choice tests. This application was really an extension of the concept of the teaching machine — an invention that started to appear just prior to the Second World War. The concept of the teaching machine had originated from the work on behaviourism. It was designed as a device for providing continuous feedback. As computers became more widely available it was recognised that they offered a more flexible means of providing such feedback.
By the mid-sixties, when video terminals started to appear, the educational application of computers was developing in two different directions — on the one hand, there was a major effort to exploit computers for the delivery of instruction, while on the other hand, computers were being used quite widely for machine-scored testing.
The major disadvantage that computers were seen to have in the delivery of instruction was the very high cost of entering into this type of teaching. This cost essentially comprised two components. Firstly, there was, the initial high capital cost of installing a system suitable for use in a teaching-learning situation. This cost was magnified by the need to achieve reasonable response times and to support a variety of display capabilities. For example, the Plato system used very expensive custom designed terminals offering what for then were high resolution graphics as well as touch-screen capabilities. The other major cost component was the cost of development of the courseware. One hour of completed instruction was generally estimated to take between 80 and 200 hours to develop. The high development costs were accounted for by the time required to develop screen layouts and create the various learning pathways through the subject matter.
It was not surprising then that, except in cases where there was really substantial external funding available, computers were regarded as being capable of playing only an adjunct role in the delivery of instruction. It needs to be remembered that back in those days the microprocessor had not yet been invented. Consequently, the developments in graphics, animation, and multimedia could not even have been conceived. Character-oriented video display terminals and card readers were the standard types of input device and the scope for displaying graphics, except on all but very specialised terminals, was extremely limited to non-existent. It is important not to underestimate the magnitude of the breakthrough that the invention of the microprocessor represented for computer-based education.
Some educators who recognised the potential that computers offered for individualising instruction, but who lacked the resources to implement CAI, saw a way of gaining access to the most important benefits that computers offer without the high cost. They recognised that most of the cost of CAI lay in presenting the instruction to the student. They reasoned that if existing resource materials were used to present the instruction, then the functions for which the computer was used could be restricted to those for which is was at that time best suited — the administration of machine-scored tests and the provision of information to guide the learner's study program.
The use of computers for test-scoring was much simpler, therefore, than for instructional presentation, because tests were able to be batch processed. It was a relatively small step to move from batch-processing of computer-scored tests to assembling banks of test items which could provide the basis of a learning management system. From there on, the developments in computer-managed learning followed a natural progression.
I don’t wish to go into any more detail than this on the historical origins of CML. However, Noel Stubbs provides some interesting historical information on the development of LMS, one of the first CML systems to be developed, and if you are interested in that aspect of CML I would recommend his paper to you.
I think that it can be argued that the most important benefit CML offered institutions was its cost effectiveness relative to more traditional modes of instruction (For a detailed analysis, see Montgomery, 1989). CML was more cost-effective, partly because it needed less in the way of equipment support, and partly because it could make use of already existing resource materials.
The primary objective of computer managed learning was to individualise instruction. It was meant to accomplish this by:
1. having courses structured into distinct modules defined in scope by a set of specific objectives;
2. drawing on existing resource materials of whatever kinds that might be available for the teaching of the modules;
3. exploring computer-administered objective tests to assess the learner's attainment of the competencies specified by the objectives; and
4. managing the learner's study program by giving directions on what to study based on analysis of students' performance on the tests.
In some respects the concept of computer managed learning resembled the personalised system of instruction (PSI) popularised by Keller almost half a decade earlier (Keller, 1968). Where CML differed from PSI was in its use of computers rather than mentors to administer achievement tests and provide remedial instruction and directions on what to study next.
In addition to the promise of being able to individualise instruction, CML promised a range of other benefits when compared to more traditional modes of teaching.
For the learner, CML promised:
• tests achievement precisely to the objectives of a course
• detailed targeted feedback indicating the extent to which the stated objectives of a course have been achieved;
• diagnostic information uncovering the sources of misconceptions;
• directions on where to go for remedial instruction; and
• flexible pacing capable of being matched to individual needs.
For the teacher, CML promised:
• regular reports on students’ progress, identifying the parts of the curriculum where they were encountering difficulties;
• diagnostic information related to the difficulties being experienced by individual students indicating which students were in the need of individual assistance; and
• analysis of the performance of test items enabling items which were failing to discriminate or which were in some way ambiguous to be culled out; and
• a freeing up of time spent in the classroom allowing more time to be spent on individual assistance to students.
Clearly the designation of CM, as it was originally conceived of, as a management system was quite appropriate.
Let us consider now how well CML delivered on its promises.
A CML system — that is, the software which provides the CML functionality — is essentially just another applications package. However, unlike CAI, CML requires no specialised hardware, no high speed processor, no laboratory of terminals to deliver full functionality.
From a programmer’s point of view, implementing a CML system is a relatively straightforward project even if in magnitude the project may be quite large. Not surprisingly, then, the early CML systems measured up well to their promise. The reliability of these systems and the quality of the reporting to both staff and students was high, even by today’s standards.
Unfortunately, one cannot be as positive about the way CML systems have ‘stood the test of time’. The functionality of computer software has improved enormously over the twenty years since the concept of CML was first developed. However, the sophistication of CML software has tended not to keep pace. The setup of CML control files has been one of the quite demanding aspects of managing a CML program. In the early days of CML this factor was not an issue. Most software demanded considerable expertise of the user. This was recognised as the price that needed to be paid to gain the benefits that computers could provide. However, as the sophistication of software increased, most software became more user friendly. However, improvements to user interfaces in CML systems have been slower to arrive.
By comparison with other segments of the software industry, the market for CML systems is not particularly large. Licences for CML systems are generally sold on an organisation-by-organisation basis and industry has been less inclined to adopt CML than the formal education system. The fact that the CML concept obviated the need for advanced presentation facilities meant that early CML systems were actually quite mature. Consequently, there has not been the perceived need for frequent updates that has been characteristic of the consumer software market. The combination of these two factors has limited the pace at which these systems have evolved. While this has not presented a major handicap for student users, it has denied teachers and support staff many of the convenience features that have been incorporated into more rapidly evolving products.
Over the years, the reaction of students studying by CML has generally been very favourable. Students appreciate:
• the anonymity, patience and politeness of the computer;
• the immediacy of the feedback;
• the specific guidance;
• being able to learn what they want to learn when they want to learn; and
• being able to choose their own rate of progress.
Several of the presenters at this conference will be reporting from their own experience the positive feedback they have received from student using CML. If one feels inclined to be sceptical of the strength of students’ appreciation for what CML has to offer, one need only turn to Jo Boycott’s paper describing her experiences working as an instructional assistant in the CML Lab in the School of Nursing at the Curtin University of Technology. One cannot help but be moved by some of the examples she gives — by the story of the timid female overseas student with limited English who, the first time she came to the university was shepherded into the Lab by her husband, but who went on to complete the 23 modules on the way to registration scoring no less than 88% on each module; or by the absolute dedication of another female student who, needing to have her Indian nursing qualification recertified, completed six CML tests in one day.
These are very positive signs. However, we need to remember that in the final analysis the primary function of a CML system is to deliver educational programs. It is important that students feel positive towards their experience of using CML. However, what is of most importance is the quality of the results that students achieve. Such a system ought therefore to be evaluated not just in terms of the opinions of its users, but more importantly in terms of its performance as measured against educational criteria.
The heart of a CML system is the bank of test items from which the sets of items forming each test are drawn. The quality of a course offered by CML is critically dependent on the reliability and quality of these items because it is upon a student’s performance on these items that the instruction and feedback are based.
By the time that the idea of CML was conceived, the technology of test item writing was already quite mature. Multiple-choice tests were widely used in public examinations and collections of test items covering a broad range of subjects were available from a variety of sources. The construction of other forms of objective test items such as multiple-choice, true/false, and matching, which are also capable of been administered by computer were likewise well understood. In theory, therefore, the concept of building a learning management system around a bank of objective test items was quite realistic.
However, as those of you who have experience at doing so know, implementing CML involves more than writing or collecting a sufficient number of items to serve as an item bank. The items need to meet criteria of validity and reliability. They need to be
• unambiguous;
• capable of discriminating between high and low achievers; and
• free from cues that might lead to response biases.
Most important of all, they need to be appropriate to the objectives of the course. It is in relation to this last requirement that the greatest threat to the validity of CML test items began to emerge.
It soon became evident that while most advanced level courses expect students to acquire intellectual skills and strategies — to use Gagné’s (1977) system of classification — subject specialists found it much easier to write items which were limited to testing factual knowledge.
The objective test item format is quite suitable for testing higher level skills with the possible exception of creativity. However, items which simply assess factual knowledge are unquestionably much easier to devise than items which test concepts or principles or problem solving. Hence, teachers tend to revert to writing knowledge items even when they are aware of the difference between knowledge items and items which test higher level skills. However, many teachers fail to make the distinction. This failure goes to the heart of what computer managed learning is all about.
If the purpose of computer managed learning is to structure the learning experiences of students, then the failure to match the items on a test to the intended outcomes of a course defeats the purpose of providing the tests in the first place. However, what may initially be regarded as simply as a lack of skill in test item construction stems from a problem which goes far deeper into the course development process.
The inability to write test items which test higher order skills generally stems from an inability to distinguish among these different types of skills. This inability to distinguish between different higher order skills also results in an inability to define the outcomes one intends students to attain. The inability to define a course’s intended outcomes results in the reversion to a mode of presentation typified by the presentation of information which encourages rote learning on the part of the student and legitimises knowledge items.
From this it can be seen that the weakness in the quality of test items points to a much more serious problem in the whole course design and development process.
It is now two decades since the idea of computer managed learning was first conceived. In that time, technology has advanced a long way. Many of the limits which technology placed on the use of computers for the delivery of instruction, initially, have since been lifted. Major advances have been made in a number of areas of technology which relate directly to computer-based learning:
• even the least expensive desktop computers are now standardly equipped with high resolution bit-mapped displays;
• colour screens can be had for a modest additional cost;
• CD-ROM drives and large capacity hard drives provide more than sufficient capacity to store a batch of lessons;
• microprocessors execute instructions fast enough to be able to support real-time video; and
• the operating systems of desktop computers are sufficiently advanced as to be able to orchestrate multimedia presentations.
However, different computer and communications technologies have not all advanced at the same rate during this period. While the technology of desktop computers has proceeded apace, the technology of telecommunications has very much lagged behind.
One factor which is critical to the performance of computers in educational applications is the speed with which data is able to be moved from the data storage device to the screen. This is usually the rate limiting factor determining the response time of the system when input is made (as, for example, after a student has responded to a test item). It is also what governs the capacity of the system to support graphics, animation and video.
In the case of networked devices, the rate limiting factor is the speed of transmission over the network. In the case of desktop devices, the rate limiting factors are the speed at which data can be transferred over the internal bus and also the speed of the CPU. For a range of reasons, the internal specifications of microcomputers are advancing at a much faster rate than the operating speeds of networks.
It is important to grasp the significance of this point, because it represents the critical stumbling block standing in the way of every attempt to upgrade the capabilities of systems for networked delivery of computer-based learning. The capacity of the internal bus of a microcomputer is already far higher than any data rate that the telecommunications carriers are likely to be able to support before the year 2000 and the gap widens further as each year goes by.
The effect of the existence of this differential in the rates of advance in technology has been to make desktop systems increasingly attractive as platforms for instructional delivery. CML, as it was originally conceived, was intended to bring the most important advantages of computers to teaching and learning at a cost that every educator can afford. In many ways the original conception of CML has been challenged by more recent advanced in technology. As a result, we are starting to see a split in the ranks of the champions of computer-mediated learning. On the one side, there are the people such as yourselves who are continuing to argue the case for the cost-efficiencies of CML. On the other side, there is this new group who are now shunning mainframe and distributed systems in favour of standalone desktop machines. As a result, advocates of CML are beginning to find themselves becoming embattled in endeavouring to champion the merits or even legitimacy of CML — having to take issue even with those who might otherwise have been seen as allies. The consequence of this split is that proponents of CML run the risk of being regarded as technological troglodytes — people who are blind to the revolution that has been occurring on the desktop.
I’m not saying that as representatives of the former group, you are unaware of developments in desktop computing. Rather, what I’m saying is that the benefits you are claiming for computer managed learning now stands the very high probability of being paled into insignificance by the much more glamorous developments in multimedia.
CML was originally conceived as a method of individualising on-campus teaching. At that time, technology which would have allowed courses to have been offered by CML off-campus was not available or at least not available at reasonable cost. Personal computers had not yet been invented, rental on dial-up lines was very high, and remote terminals were expensive.
The types of materials sent to students who were studying off-campus were not particularly suitable for adaptation to CML, either. They were generally conceived of as the equivalent of ‘lecture notes’. Indeed, in some cases they comprised transcriptions of even recordings of the lectures that had been delivered on-campus. They lacked the types of embedded activities that we now consider critical to having students engage with study materials. All that came later. External studies were considered ‘second best’.
The successful establishment of the British Open University led to a reconceptualisation of off-campus delivery. The decision to apply methods of delivery such as television, high quality print materials and computer-marked tests which were more appropriate to this mode of delivery began to reduce the high attrition rate. Other institutions came to observe the Open University's way of doing things and then began to follow suit. The success of the Open University started to change educators' perceptions of the value of the off-campus mode of teaching. Distance education was no longer seen as ‘second best’ but rather ‘equal but different’.
These developments in distance education coincided with developments in technology that allowed institutions to begin experimenting with the use of computers for teaching at a distance. In Australia, for example, an innovative program was established at Darling Downs Institute of Advanced Education. This began as simply a voluntary formative evaluation program based on sets of multiple-choice items and mark-sense cards. It then developed to a more interactive program using microcomputers in study centres. At this stage the student's results were returned to the institution on floppy discs. Further enhancements provided students with interactive instruction. The Darling Downs system was not a CML system as most of us know it in that the students' learning was not fully managed by computer. However, it has contained many of the elements of CML.
More recently, Deakin University and Box Hill has introduced the use of LMS in industry-based programs which have broken new ground in forging links between education and industry. Initially these were restricted to the field of technology management. However, more recently they have been extended to other fields.
Jim Strain describes the use the Deakin LMS facility in supporting an industry-based certificate program in instructional design. It is worth noting that in this case the LMS system is being used principally for assignment management and student tracking rather than for achievement testing.
Meanwhile at the TAFE level, following several committees of enquiry into the training needs of industry, has been the introduction of competency-based training and recognition for prior learning. Again, this has hastened the trend towards individualised materials-based instruction which in the TAFE sector — in recognition of the fact that it covered both on- and off-campus delivery — has come to be known as ‘flexible learning’.
Quite a number of presenters at this conference, describe their experiences of using CML systems to expand flexible learning at the TAFE level. Tom Aumann describes the experiences of the Outer Eastern College of TAFE here in Melbourne in implementing the LMS system. The Outer Eastern College of TAFE is a multi-campus institution and for this reason alone has particular justification for wanting to move towards more flexible modes of delivery. However, the College has also established a very successful off-campus program in the area of computing. Bob Anderson describes the establishment of Centre 42 — a centre for supporting CML statewide in South Australia which was established at the Regency Institute of TAFE. Kathy Cavanagh explains that the introduction of competency-based training was the dominant factor that led to the decision to pursue the introduction of CML throughout TAFE colleges in that state. She describes a number of different ways in which CML is beginning to function in that system, including in the workplace, in electronic learning centres, and in assessment centres. Peter Biggs describes the establishment by the Hobart Institute of TAFE of a CML-based program for workplace training of workers in the building and construction industry. Rod Crocker reviews the developments that have occurred in the remote parts of Western Australia, particularly in the Pilbara.
The common theme that runs throughout all these papers is that in the TAFE system the decision to adopt CML has in each case been driven by the need to move to a more flexible mode of delivery in order to enable courses to be made available in the workplace.
The growth in off-campus and flexible learning in all their forms, coming at the same time as the rapid improvement in computer and communications technology have quite rapidly begun to create a new ‘market’ for computer-based learning and therefore ought to offer a bright future for CML. However, the circumstances that favour the more widespread adoption of CML also at the same time favour the more widespread adoption of CAI. In particular they favour that form of CAI now being referred to as ‘multimedia’. The concept of multimedia has the added advantage that it is seen to be much more glamorous. The question we have to ask ourselves is ‘Has the window of opportunity opened up too late?’
To be able to respond sensibly to that question, we need to include in our consideration recognition of the fact that the concepts of CML and CAI are not mutually exclusive. As I’ve indicated earlier, the two defining characteristics of CML are that:
1. it supports the management of learning, and
2. it does so through the mediation of computer-administered tests.
A CML system can draw on any type of instructional resource — including, of course, resources that are computer based.
The educational rationale for CML nevertheless remains sound. Furthermore, although the concept of multimedia may look attractive to educators, its attraction lies principally in the area of the presentation of instruction. Multimedia approaches to educational delivery tend to be weak in the area of testing and learning management — the very areas where CML is strong. This suggests that what is needed at this stage is a rethinking of the concept of CML. The future for CML seems to lie, not in the restricted role of administering achievement tests and managing student progress data, but in a broader role which encompasses the full range of functions involved in the management of student progress through a course. When we begin thinking along these lines, we can begin to see that CML should sit within a broader context which includes the full range of transactions that occur between the educational provider and the student. In addition to the transactions involved in assessment, these transactions also include, for example, enquiry management, course enrolment and re-enrolment and the recording of transactions that occur by mail. What is needed therefore is an approach to computer-based learning which integrates all of these functions into a single application. An institution's window into the electronic classroom should give the appearance, from where the learner sits, of one seamless environment.
I would like to indicate what I believe will be the way in which this will eventually be accomplished.
Client-server systems are a class of network application which are designed to be used in situations where a substantial amount of the processing of data can be accomplished in the local workstation, while there is at the same time a need to draw from, or to deposit into, a central database information which either needs to be archived or to be available throughout an organisation. A client-server system is designed to function over a local area or wide area network. However, communication may also be maintained over a serial link, although performance suffers when communication is carried over a link of this type.
In an educational environment, the client-server model could be used to support a hybrid CAI/CML system by delivering instruction as a local process while managing learning as a client-server process. The computer-based instructional material might be provided from a local CD-ROM — the analogue of it being provided by a print package. Alternatively, it might be downloaded from the server as separate modules at intervals or even as a background process; i.e. while the learner is working on other material.
From a programmer’s point of view, a client server application for supporting CML and CAI could readily be implemented with today’s technology. System software for designing client-server applications is available from a number of suppliers and such systems can be designed to support a number of different types of clients; such as Macintoshes or PCs running under Windows. However, the performance of such systems will not be able to achieve their full potential until high speed links are available right into a person’s home at reasonable cost.
Two papers being delivered at this conference describe implementation of client server systems for supporting CML although neither has extended the concept to include support for multimedia delivery.
Rod Byrnes and Bruce Lo describe the development of a pilot project at the Southern Cross University called the Assignment Management System (AMS). The first incarnation of this system is geared to the receipt and return of assignments. However, plans for AMS include provision of support for computer-based testing as well as a number of other functions to do with student tracking. However, what is of interest from a systems point of view is that AMS is constructed as a client-server application in which the server runs under Unix (on a PC platform at the present time) and the client runs under Windows (It is envisaged that a Macintosh client will be developed later).
Vern Bawden and Robert Blakely describe a somewhat more ambitious project at the University of Queensland, going under the name ‘QUiZ’. This is also presently being trialed in pilot form. The way in which QUiZ has been conceptualised appears top have many similarities to AMS. However, in the case of QUiZ, the server runs under the Oracle database management system.
Both of these papers provide useful insights into the benefits of using a client-server approach to implementing CML although neither project incorporates support for multimedia presentation at the present time.
This brings us back to what I have suggested may be the most important issue facing advocates of the concept of CML — how to improve the quality of test item design.
If I am correct in saying that the nub of the problem here is not the technical difficulties encountered in writing good quality items but the whole approach to the design and development of self-instructional materials, then this is not specifically a problem for CML. It just happens to be the case that CML is more vulnerable to the problem than other forms of self-instruction. However, this does not mean that the champions of CML can afford to side-step the issue.
Improving the quality of CML-based courses demands greater investment at the design phase and that greater attention be paid to the theoretical framework in terms of which the design process is understood.
Integrating the functions of learning management and delivery of instruction, and thereby taking advantage of developments in hardware and software technology, will shortly lift the barriers to a major advance in CML functionality. Improving the use of CML systems to provide effective instruction requires action of a different kind. If we are willing to invest the necessary resources and apply ourselves with commitment and a vision to that problem, then CML offers the prospect of becoming the integrating element in institution-wide systems of computer-based learning. If not, then ten years from now the promise of CML will remain just that.
Barker, P.G. and Singh, R. (1982). Author Languages for Computer-based Learning, British Journal of Educational Technology, 3(13), 167-196.
Gagné, R.M. (1977). Analysis of objectives. In L.J. Briggs (Ed.) Instructional Design, Educational Technology Publications, Englewood Cliffs.
Keller, F. S. (1968). Goodbye teacher ..., J. Applied Behavioural Analysis, 1, 79-89.
Montgomery, A. Y. (1989). The economics of CML and CAI, Proceedings of the First International CML Users Conference, Limerick.
[*] Office of the Director, Educational Quality Assurance, Research and Development
Royal Melbourne Institute of Technology
GPO Box 2476V, Melbourne, 3001. This is the text of a paper delivered to a conference on computer managed learning held in Melbourne 1994. Whilst not specific to law, the editors considered the quality of the paper such that it was worthy of reproduction without amendment. (ed.)
AustLII:
Copyright Policy
|
Disclaimers
|
Privacy Policy
|
Feedback
URL: http://www.austlii.edu.au/au/journals/JlLawInfoSci/1995/3.html