AustLII Home | Databases | WorldLII | Search | Feedback

Precedent (Australian Lawyers Alliance)

You are here:  AustLII >> Databases >> Precedent (Australian Lawyers Alliance) >> 2017 >> [2017] PrecedentAULA 16

Database Search | Name Search | Recent Articles | Noteup | LawCite | Author Info | Download | Help

Sourdin, Tania --- "Justice in the age of technology: 'The rise of machines is upon us'" [2017] PrecedentAULA 16; (2017) 139 Precedent 4


JUSTICE IN THE AGE OF TECHNOLOGY

‘THE RISE OF MACHINES IS UPON US'[1]

By Tania Sourdin

Changing and emerging technologies are reshaping the justice system. First, and at the most basic level, technology can assist to inform, support and advise people involved in the justice system (‘supportive technology’). Second, technology can replace functions and activities that were previously carried out by humans (‘replacement technologies’). Finally, at a third level, technology can change that way that judges work and provide for very different forms of justice (‘disruptive technology’) that can include replacing some human decision-making.

As technology continues to change the way in which we work and function, there are predictions that many aspects of human activity will be replaced by technology and that work activities previously carried out by humans will increasingly be conducted or carried out by replacement technologies.[2] While many human activities have changed over time as a result of human advances, more recent shifts in the context of technological change are likely to reshape the process of adjudication by either replacing judges or supporting and supplementing the judicial role as the fourth industrial revolution is directed at almost all human activities. Technology also has the capacity to inform justice system reform through the use of big data sets, relationship data and more complex knowledge generation.

Currently, three differing levels of technological change are reshaping and will continue to reshape and alter the justice sector:

• Level One – Supportive Justice Technology

• Level Two – Replacement Justice Technology

• Level Three – Disruptive Justice Technology

At present, most technology-related justice reforms have involved the first and second level of technological innovation. These reforms have supported and supplemented the operation of many court-based processes. For example, at the first level of innovation, many people now locate justice services online and obtain information about justice processes, options and alternatives (including legal alternatives) through web-based information systems. People also increasingly locate and obtain legal support and services online and the growth of online legal firms who may provide ‘unbundled’ legal services has been significant over the past three years.[3]

Some web-based information (including digital video), video-conferencing (including internet-based group video calls[4]), teleconferencing and email can supplement, support and replace many face-to-face in court and dispute resolution approaches and could be defined as a second level technological approach. At this second level, justice is supported by technology and, in some circumstances, technology changes the environment in which alternative dispute resolution (ADR) and court hearings take place.[5] For example, online court processes are increasingly used in some types of disputes and in relation to criminal justice matters (particularly bail applications).

Other technologies may merge into the ‘third level’ and support negotiation as well as judicial processes by enabling people to access more sophisticated online ‘advice’ which is supported by artificial intelligence (AI), or to consider options and alternatives or engage in different ways. In contrast to the traditional rational decision-making approaches, some of these more sophisticated technological programs can also encourage the development and refinement of a number of options (rather than producing one outcome).[6] In addition, these disruptive technologies have the capacity to replace some judicial functions altogether.[7]

SUPPORTIVE TECHNOLOGY

Many supportive technologies are directed at supporting an understanding of and engagement in the justice system. For example, some courts now use Facebook, LinkedIn, Twitter and YouTube to engage with disputants about court activities and dispute resolution, and to support dispute avoidance and self-managed negotiation strategies.[8] In addition, most ADR and court services use websites to inform and support those involved in court activities and may reach out to others via social media. The extent to which websites and applications or ‘apps’ are used may vary according to the technological capacity and maturity of the main users of the services.

A variety of websites support justice services and can be considered to be supportive technologies. These websites enable people to access information about courts and file documents. Some court websites offer online forms and follow up email and related support requests. It is likely that websites will become more sophisticated in the future and increase in number. In the context of other supportive technologies, apps have become an increasingly popular way of accessing information and connecting to services over the past five years. A variety of apps have been created that provide people with free access to legal information and advice. This is a significant area of development, as apps may support those who are comfortable with smartphones and mobile devices but intimidated by or unable to access computers.[9]

A large number of legal information and services apps have emerged in the US. For example, ‘Ask a Lawyer: Legal Help’ enables people to message and chat live with lawyers for free.[10] Another American app, ‘BernieSez’, allows consumers to upload a photograph, a ticket, or other charge-related paperwork. Lawyers then compete to win work from the consumer, offering their fees upfront.[11]

Legal apps have also emerged in Australia. For example, Legal Aid South Australia has created an app called ‘LegalAidSA’. The app is free to download and provides legal information and resources on a wide range of areas, including family law, wills and motor vehicle accidents. The app facilitates calls to free legal help telephone lines and provides travel directions to South Australian Legal Aid offices.[12] Apps are an especially useful tool for communicating with young people who commonly use their smartphones to access information. Victoria Legal Aid, in conjunction with a number of interstate Legal Aid bodies, created an app called ‘Below the Belt’. Below the Belt provides young people with legal information about sexual offences and cyberbullying.

A variety of more general apps are also available for lawyers. These include apps that convert voice to text, count days and manage documents.

REPLACEMENT TECHNOLOGIES

The last decade has seen a significant growth in replacement technologies. These technologies can replace certain interactive parts of justice processes. As with supportive technologies, replacement technologies can have an impact on how complex disputes are managed and resolved. For example, many sophisticated technologies exist that can organise paperwork and files in disputes to simplify and reduce the time necessary to review and analyse files.[13] Within Australia, replacement technologies have continued to develop rapidly in some selected dispute areas. This has occurred as a result of a healthy justice environment, robust e-environment and the extension of higher bandwidth via new broadband arrangements. The new broadband arrangements will continue to expand as a result of the NBN rollout,[14] and newer technologies that support wi-fi remote operating options are likely to increase the potential for replacement technologies. Over the past decade, there has been considerable growth to support more complex interactive technologies. For example by 2014, 99 per cent of internet connections were broadband.[15]

In the ADR context, these replacement processes are often referred to as online dispute resolution (ODR) and technology is now often used to replace face-to-face conferences with video conferences and email or secure chat services, coupled with cloud-based or online documentation. The increasing engagement of government institutions with systems of ODR is indicative of its promise as a dispute resolution system.[16] However, much of the growth in ODR in Australia has been directed at first-tier complaint-handling mechanisms for e-commerce and consumer-based schemes or mixed supportive and replacement technologies.[17]

Numerous consumer disputes are addressed using only ODR. Modria is one of the largest commercial operators in this area and focuses mainly on US business.[18] Systems that deal mainly with consumer disputes and specialise in large-scale operations, such as Modria, have dealt with more than 400 million disputes. The eBay and PayPal online systems deal with approximately 60 million matters per year.[19]

These systems are expanding rapidly, particularly in Europe, following the establishment of the European Union (EU) ODR regulation. This extensive system has already been set up in some European countries and deals with consumer disputes.[20] Importantly, both the directive on consumer ADR (Directive 2013/11/EU of 21 May 2013) and a Regulation on Consumer Online Dispute Resolution,[21] published in July 2013, have led to additional commentary about ODR.[22] EU member states had 24 months to transpose the regulation into national legislation (that is, by mid-2015) and a further 6 months for ODR platforms to become operational in 2016.

It is likely that future developments in this area will be informed by both local and international developments. At the local level, developments supporting ongoing reforms have been endorsed by the 2014 Productivity Commission Report.[23] Many Australian courts have also embraced replacement technologies. Those within the litigation system have noted that technological changes are dramatically transforming the way in which dispute-related work is carried out, partly because much of the preparation work before a hearing will be carried out online.[24]

Within the court system, e-callovers,[25] e-filing,[26] video conferencing and applications[27] are now commonplace in many jurisdictions. Technology courts, virtual courts or cyber courts also exist in many jurisdictions,[28] and the presence of such initiatives may produce more participatory court processes as well as better communication and document management.

Internationally, recent work in the UK by the Civil Justice Council (CJC)[29] is likely to influence future developments in the justice sector. The CJC Online Dispute Resolution for Low Value Civil Claims report addresses concerns that the current court system of England and Wales is ‘too costly, too slow, and too complex, especially for litigants in person’.[30] The CJC’s central recommendation relates to replacement technologies with the establishment of a new internet-based court service for civil disputes with a value of less than £25,000.[31] The CJC estimates that the new court, the HM Online Court (HMOC), may be launched in 2017.[32] The CJC’s recommendations are similar to those of the EU’s Regulation on Consumer ODR.[33]

DISRUPTIVE TECHNOLOGIES

The third level of technological change consists of technologies that may significantly change ADR and judicial processes. Newer technologies may enable options for resolution to be developed via technology and use AI to develop alternatives and run processes. In this regard, some disruptive technologies are linked to AI which can be viewed as a form of dispute resolution or a system that has the capacity to render expert advice or decision-making. AI refers to computer systems that perform tasks and/or solve problems that usually require human intelligence.[34] These processes have emerged over the past 50 years[35] and have been directed at technical as well as legal analysis.[36] They have the capacity to be blended with existing adjudicatory or non-adjudicatory processes. However, it is probable that their impact will be most significant where determinative and advisory processes are used.

Legal information and AI systems can use sophisticated ‘branching’ and data-searching technology to create elaborate decision trees that can suggest outcomes to disputes. This is done by a system which emulates human intelligence (neural networks). [37] Essentially, what takes place is that the system asks the user a number of questions, or uses existing data about the user and questions about the dispute to enable an accurate description of the dispute to be built. The computer then forms a conclusion by applying the law to the dispute description. It does this by applying rules for specific sets of facts. Finally, the computer can perform tasks based on the description given.[38] This process may enable indicative decisions to be expressed.

Potential limitations of this approach:

• the deskilling of human staff;

• the threat to the independence of the decision-maker;

• the model's ability to support some users better than others;

• the potential for users to be misled as to the amount of knowledge contained in the system;

• the absence of a human element which may be required in special circumstances;

• the risk of replacing ouster clauses in legislation due to automated enforcement; and

• the undermining of the judiciary in acting as a check on the legislature.

These areas of technological innovation, at the ‘third level,’ have the capacity to be more disruptive than previous innovations that supported a ‘graft and grow’ approach and assumed that judging processes would not change in the context of their basic procedural stages. In this context, there is considerable opportunity for technological processes not only to support judges but potentially supplant them. Judges are not unique in this regard and there are numerous predictions that AI, together with other advances, will mean that many current employment arrangements will no longer exist in 20 years.[39] However, does this mean that judicial work will change, or will some judges be completely replaced by newer technologies? Arguably not in higher courts. This is because there are many factors that impact on decision-making. The Australian Law Reform Commission (ALRC) has noted that such factors include induction and intuition as well as the capacity to assess the social impact of decisions.[40] There may, however, be some significant impacts in relation to smaller civil claims (at least initially) where some discussions[41] appear to contemplate that AI processes may replace some judicial functions. In addition, if technologies can support decision-making they may play an increasing role in some forms of dispute resolution (by, for example, producing a draft decision that can then be considered by a human judge). These types of technology have already been trialled,[42] and have not been extended in the past because of connectivity, cultural, technological storage and access issues. Such issues will continue to pose considerable obstacles into the future. However, courts will face significant competition from the ODR area where these technologies are more likely to be adopted initially, and this competition may promote the take-up of some technologies in some courts and tribunals.

Perhaps even more significant than the impact they have on courts, disruptive processes and ‘big data’ gathering have the potential to change the way we think about and plan court systems. The quality of business decision-making, government administration, scientific research and much else can potentially be improved by analysing data in better ways.[43]

Better data analysis would support the more targetted delivery of justice and support services. In addition, it could enable better understandings to emerge about how and what justice interventions might be more effective. It is also possible that big data will support more sophisticated measuring in respect of the effectiveness of the justice system.

CONCLUSIONS

To date, many technological innovations have been directed at e-commerce and consumer-based schemes that operate as first-tier complaints-handling and dispute-resolution mechanisms. The growth in these schemes in recent years has been immense and there are now many examples of processes that are available to resolve disputes relating to online transactions.[44] Increasingly, ODR is also being blended with tribunal and court systems to provide support, intake and advisory processes that are intended to assist disputants to negotiate more effectively without having an ADR practitioner involved. This is likely to increase in the near future as many courts build online systems that support filing, referral and other activities.[45]

There is a growing convergence between online systems and court and tribunal systems. For example, in British Columbia, Canada, a new Civil Dispute Tribunal is already operating using an online platform so that disputants will make initial contact and commence proceedings through an online format.[46] It is intended that processes used by the Tribunal will be mainly online, at least initially.[47] Online-supported negotiation and supported ODR are features of the system, together with adjudication, with most cases decided ‘... on evidence and arguments submitted through the tribunal’s online tools. However, when necessary, the adjudicator will have discretion to conduct a telephone or video hearing.’[48]

In Ireland, the Northern Ireland Courts and Tribunal Service now offers an online process in respect of small claims. A specialised Civil Processing Centre operates according to time-based and other rules to make orders, although final adjudication remains a face-to-face option.[49] The CJC Online Dispute Resolution for Low Value Civil Claims report[50] recommends building a new dedicated internet-based court service for civil disputes with a value of less than £25,000[51] and would appear that, in the UK at least, there is a preparedness to consider disruptive technologies and the impact that they will have on the justice system. In recognising the significant impacts that technology has already had, there may need to be more questions asked about how and when these technologies will reshape the judicial and other justice functions given the way in which disruptive technology is already reshaping the business of litigation.[52] In addition, if AI systems are to replace some judicial or adjudicative functions, who will have control over the relevant algorithms and data field areas and, importantly, how will privacy concerns and also data management and data reach issues be dealt with?

Tania Sourdin is Professor of Law and Dean of the University of Newcastle, Australia. Parts of this article are drawn from T Sourdin, Alternative Dispute Resolution (Thomson Reuters, 5th ed, 2016) and T Sourdin, ‘Justice and Technological Innovation’ (2015) 25 Journal of Judicial Administration 96. EMAIL Tania.Sourdin@Newcastle.edu.au.


[1] Quote from <http://www.legaltechnews.com/id=1202769371543/Technology-Proves-Vital-Force-in-the-Future-of-Law-Firm-Business?cmp=share_twitter> (6 October 2016, accessed 25 January 2017).

[2] See <http://www.smh.com.au/money/how-to-beat-the-robots-the-future-of-jobs-in-australia-20160923-grn615> (27 September 2016, accessed 25 January 2017).

[3] See, eg, Lawyal, available at <https://lawyal.com.au/about-us> (accessed 25 January 2017). Unbundled legal service provision involves assistance with set tasks. That is a lawyer may be engaged to assist to prepare some documentation.

[4] Group video calls are available through subscription services such as SkypeTM.

[5] See, eg, procedural changes in relation to ODR in the arbitration area, available at <https://acica.org.au/use-online-dispute-resolution-technologies/> (accessed 25 January 2017).

[6] See, eg, Smartsettle, available at http://www.smartsettle.com/home/products/smartsettle-one/ (accessed 25 January 2017). It has been said that collaborative platforms, such as GroupMindExpress.com, are likely to be used more frequently in large multi-party disputes where information and participants are plentiful: see A Gaitenby, ‘Online Dispute Resolution’ in The Internet Encyclopedia (John Wiley & Sons Inc, 2004) vol. 2, p745, available at <http://www.onlinelibrary.wiley.com/doi/10.1002/047148296X.tie129 /full> (accessed 25 January 2017).

[7] There are already many examples of AI informing human decision making in the justice sector, eg, in Mexico, ‘Expertius’ advises judges and clerks ‘upon the determination of whether the plaintiff is or is not eligible for granting him/her a pension’.: see D Carniero, P Novais, F Andrade, J Zelenznikow and J Neves, ‘Online Dispute Resolution: An Artificial Intelligence Perspective’ (2014) 41 Artificial Intelligence Review 211, p227.

[8] <http://gov2.net.au/blog/2 009/12/31/guest-post-the-victorian-department-of-justice-and-web-2-0/#more-1750> (31 December 2009, accessed 25 January 2017).

[9] See J Dysart, 20 Apps to Help Provide Easer Access to Legal Help (2015), available at <http://www.abajournal.com/magazine/artic le/20_apps_providing_easier_access_to_legal_help/> (1 April 2015, accessed 25 January 2017).

[10] See <https://play.google.com/store/apps/details?id=com.absmallbusinessmarketing.askalawyer&hl=en> (accessed 25 January 2017); J Dysart, 20 Apps to Help Provide Easer Access to Legal Help (2015), available at <http://www.abajournal.com/magazine/article/20_apps_providing_easier_access_to_legal_help/> (1 April 2015, accessed 25 January 2017).

[11] See <http://www.berniesez.com/> (accessed 25 January 2017); J Dysart, 20 Apps to Help Provide Easer Access to Legal Help (2015), available at <http://www.abajournal.com/magazine/article/20_apps_providing_easier_access_to_legal_help/> (1 April 2015, accessed 25 January 2017).

[12] Legal Services Commission of South Australia, What’s New: LegalAidSA, available at <http://www.lsc.sa.gov.au/cb_pages/news/LegalAidSAdownloadoverview.php> (accessed 25 January 2017).

[13] Stratify, Stratify Legal Discovery 6.0 Service Expands Discovery Capabilities and Increases Attorney, available at <http://technology.findlaw.com/electronic-discovery/stratify-legal-discovery-6-0-service-expands-discovery.html> (accessed 25 January 2017).

[14] See T Sourdin and C Liyanage, ‘ODR in Australia (Chapter 7)’ in MS Abdel Wahab, E Katsh and D Rainey, Online Dispute Resolution: Theory and Practice – A Treatise on Technology and Dispute Resolution (Eleven Publishing, Egypt, 2011). See also Department of Communications, National Broadband Network, available at <http://www.communications.gov.au/broadband/national_broadband_network> (accessed 25 January 2017).

[15] ABS, Internet Activity, Australia, December 2014 (Report, ABS, 2014), available at <http://www.abs.gov.au/aus stats/abs@.nsf/Latestproducts/8153.0Main%20Features1December%202014?opendocument & tabname=Summary & prodno=8153.0 & issue=December%202014 & num= & view=> (accessed 25 January 2017).

[16] The adoption of ODR by government institutions, or the ‘institutional phase’, is recognised as the fourth phase in the adoption of ODR, taking it beyond the earlier ‘hobbyist’, ‘experimental’ and ‘entrepreneurial’ phases. See M Conley-Tyler, 115 and Counting: The State of ODR (paper presented at the United Nations Third Annual Forum on Online Dispute Resolution, Melbourne, 5-6 July 2004), available at <http://www.ombuds.org/unforum2004/ConleyTyler.htm> (accessed 25 January 2017).

[17] See M Conley-Tyler, 115 and Counting: The State of ODR (paper presented at the United Nations Third Annual Forum on Online Dispute Resolution, Melbourne, 5-6 July 2004) note 24, available at <http://www.ombuds.org/unforum2004/ConleyTyler.htm> (accessed 25 January 2017); M Conley-Tyler and M McPherson, ‘Online Dispute Resolution and Family Disputes’ (2006) 12(2) Journal of Family Studies 7, note 13, available at <http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1032743> (accessed 25 January 2017).

[18] See <http://modria.com> (accessed 25 January 2017).

[19] Ibid.

[20] See a discussion of these developments at <http://www.infolaw.co.uk/newsletter/2013/01/online-dispute-resolution/> (January 2013, accessed 25 January 2017).

[21] Available at <http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:165:0001:0012:EN:PDF> (accessed 25 January 2017).

[22] In the UK a report – Department for Business, Innovation and Skills, Alternative Dispute Resolution for Consumers (Report, Department of Business Innovation and Skills, 2014) available at <https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/377522/bis-14-1122-alternative-dispute-resolution-for-consumers.pdf> (accessed 25 January 2017) sets out material relating to the UK response. In March 2015, government regulations were published which included requirements for traders to provide certain information to consumers regarding ADR by 9 July 2015.

[23] Productivity Commission, Access to Justice Arrangements (Report No. 72, Productivity Commission, Australia, 2014).

[24] Australian Law Reform Commission (ALRC), Review of the Adversarial System of Litigation. Technology – What it Means for Federal Dispute Resolution (Issues Paper No. 23, ALRC, 1998). See also Australian Lawyers Alliance, Precedent, Issue 132, January/February 2016, which reports on a range of issues relevant to the use of technologies in the legal context.

[25] For example, New South Wales Land and Environment Court.

[26] For example, Federal Court of Australia.

[27] Bail applications are commonly carried out by video in the Supreme Court of New South Wales.

[28] See Hong Kong Judiciary, Technology Court, available at <http://www.judiciary.gov.hk/en/crt_services/tech_crt.htm> (accessed 25 January 2017).

[29] CJC, Online Dispute Resolution for Low Value Civil Claims – Online Dispute Resolution Advisory Group (Report, CJC, 2015) p3.

[30] Ibid.

[31] Ibid.

[32] Ibid, p29.

[33] Ibid, p27.

[34] R Susskind, The Future of Law: Facing the Challenges of Information Technology (Clarendon Press, Oxford, 1996) p120.

[35] For a history of the development of ALI, see P Gray, Artificial Legal Intelligence (Brookfield, Dartmouth, United Kingdom, 1997) Ch 2.

[36] P Savasdisara, ‘Computer-assisted Legal Analysis Systems: Part 1: The Origins of Computer-aided Support Systems’ (1994) 5(2) Computers and Law 28.

[37] See, eg, NB Chaphalkar, KC Iyer and SK Patil, ‘Prediction of outcome of construction dispute claims using multilayer perceptron neural network model’ (2015) 33(8) International Journal of Project Management 1827.

[38] See above note 37.

[39] See T Dolphin (ed), Technology, Globalisation and the Future of work in Europe (2015, Institute for Public Policy Research, UK) available at <http://www.oxfordmartin.ox.ac.uk/publications/view/1900> (accessed 25 January 2017).

[40] ALRC, Review of the Adversarial System of Litigation. Technology – What it Means for Federal Dispute Resolution (Issues Paper No. 23, ALRC, 1998) p100.

[41] Significantly, the UK CJC Online Dispute Resolution for Low Value Civil Claims report (see above note 29) explores the potential role of disruptive technology in lower value civil cases.

[42] See, for example, the systems described in D Carniero, P Novais, F Andrade, J Zelenznikow and J Neves ‘Online Dispute Resolution: An Artificial Intelligence Perspective’ (2014) Artificial Intelligence Review 41 211.

[43] D Bollier, The Promise and Peril of Big Data (Report, The Aspen Institute, 2010) 2, available at <http://india.emc.com/collateral/analyst-reports/10334-ar-promise-peril-of-big-data.pdf> (accessed 25 January 2017).

[44] See Australian Government, The Treasury, Consumer Affairs Division, Dispute Resolution in Electronic Commerce (Canberra, October 2001).

[45] See, eg, in NSW the new online court website at <https://onlineregistry.lawlink.nsw.gov.au/content/> (accessed 25 January 2017) and <http://www.smh.com.au/nsw/nsw-government-trials-online-court-for-civil-cases-in-sydney-20150807-giuig2.html> (accessed 25 January 2017). The Federal Court of Australia has had an e courtroom and been expanding online lodgement services for some years – see Federal Court of Australia, eCourtroom, available at <http://www.fedcourt.gov.au/online-services/ecourtroom> (accessed 25 January 2017).

[46] <http://www.ag.gov.bc.ca/legislation/civil-resolution-tribunal-act/> (accessed 25 January 2017).

[47] British Columbia Ministry of Justice, Online Civil Dispute Tools to Save Time, Money (2012), available at <http://www2.news.gov.bc.ca/news_releases_2009-2013/2012JAG0068-000600.htm> (accessed 25 January 2017).

[48] See <http://www.ag.gov.bc.ca/legislation/civil-resolution-tribunal-act/pdfs/CRT-Business-Model.pdf> (accessed 25 January 2017).

[49] See Northern Ireland Courts and Tribunals Service, Small Claims Online: A Users Guide, available at <http://www.courtsni.gov.uk/SiteCollectionDocuments/Northern%20Ireland%20Courts%20Gallery/Online%20Services%20User%20Guides/Small%20Claims%20Online%20User%20Guide.pdf> (accessed 25 January 2017).

[50] See above note 29.

[51] Ibid, p29.

[52] For example, computer-assisted document coding and review, often referred to as ‘predictive coding’, M Tamburro, The Future of Predictive Coding – Rise of the Evidentiary Expert, available at <http://technology.findlaw.com/electronic-discovery/the-future-of-predictive-coding-rise-of-the-evidentiary-expert-.html> (accessed 25 January 2017).


AustLII: Copyright Policy | Disclaimers | Privacy Policy | Feedback
URL: http://www.austlii.edu.au/au/journals/PrecedentAULA/2017/16.html