University of New South Wales Law Journal
|
|
Home
| Databases
| WorldLII
| Search
| Feedback
University of New South Wales Law Journal |
|
EXPERT REPORTS AND THE FORENSIC SCIENCES
GARY EDMOND,[*] KRISTY MARTIRE[**] AND MEHERA SAN ROQUE[***]
All Australian jurisdictions regulate the admission of expert opinion evidence. The rules focus on ‘specialised knowledge’, the existence of a ‘field’, and ‘training, study or experience’.[1] They purport to regulate oral testimony but also the expert reports prepared in advance of potential proceedings, which often shape the way charge decisions are made, pleas are negotiated, and cases are settled, prosecuted, defended and occasionally appealed. In recent decades, in conjunction with attempts to enhance the efficiency of legal proceedings, courts have embarked on efforts to regulate the content and disclosure of expert reports through court rules and practice notes. Most of these emerged from judicial concerns about partisanship and the resources consumed by contested expert opinion evidence in civil proceedings. Only later were they extended to criminal proceedings. This article reviews the rules regulating expert reports, particularly around determining guilt in criminal proceedings. Through a detailed review of responses to an expert certificate in a recent trial in NSW, along with a revised report template developed in its aftermath, this article explains the importance of complying with codes of conduct and practice notes and engaging with scientific research and advice.
This article begins with a brief review of rules and standards regulating expert witnesses and their reports or certificates,[2] before moving to critically review a recent challenge to the admissibility of fingerprint evidence in the Children’s Court of New South Wales in JP v Director of Public Prosecutions (NSW).[3] That case included an unusually robust attempt to exclude the evidence of a police latent fingerprint examiner called by the Crown.[4] In reviewing the certificate and oral evidence adduced at trial, as well as the revised reporting template developed in the shadow of the appeal, we draw attention to relevant literatures, perspectives and criticisms that point to the unsatisfactory nature of many of the certificates and reports routinely produced for criminal investigations and routinely relied upon in criminal prosecutions.[5] While it is important to acknowledge that the report template developed in response to the challenge raised by JP v DPP represents an improvement on past practice, this article identifies and explains important limitations. Many of the issues raised in this article are applicable – either directly or indirectly – to other types of forensic science and medicine evidence.[6]
In terms of rules, the High Court has explained on a number of occasions that opinions based on specialised knowledge (so-called ‘expert opinion evidence’) should be presented in a form that enables the decision-maker to determine whether the evidence is admissible:[7]
the provisions of s 79 will often have the practical effect of emphasising the need for attention to requirements of form. By directing attention to whether an opinion is wholly or substantially based on specialised knowledge based on training, study or experience, the section requires that the opinion is presented in a form which makes it possible to answer that question.[8]
This applies not only to testimony, but perhaps more importantly to reports and certificates prepared in the pre-trial stage, given the potential effect such reports may have on defence and prosecutorial decision-making. Compliance with admissibility rules, such as section 79 of the UEL, in these terms is fundamental when adducing the opinions of those presented as experts, because decision-makers – whether judge, jury or counsel – must be placed in a position to rationally evaluate the evidence:[9]
it is a primary duty imposed on experts in giving opinion evidence to furnish the trier of fact with the criteria to enable the evaluation of the expert conclusion: Makita (Australia) Pty Ltd v Sprowles ... The ‘bare ipse dixit’ of a scientist upon an issue in controversy should carry little weight. See Davie v Magistrates of Edinburgh ...[10]
This idea, along with some sense of judicial anxiety about witnesses making unsupported assertions, is reflected in a number of prominent decisions. In Makita, Heydon JA explained that ‘[t]he jury cannot weigh and determine the probabilities for themselves if the expert does not fully expose the reasoning relied on’.[11] It follows that expert reports must be written in a manner that, at the very least, enables a technically proficient individual to make an assessment of the conclusion and the reasoning behind it. Ideally, reports should be written in a manner that enables lay users, namely lawyers, judges and potentially jurors, to understand not only the conclusion or gist but also what was done, the reasoning process and, critically, provide insight into uncertainties and limitations.
Significantly, judges in the Federal Court of Australia have also read these requirements into the terms of section 79. Finding expert reports inadmissible in Ocean Marine Mutual Insurance Association (Europe) OV v Jetopay Pty Ltd, Black CJ, Cooper and Emmett JJ explained:
it is not permissible to conclude, simply because a person expresses an opinion on a particular subject, referring to particular technology, that that person has any specialised knowledge in relation to that subject. There must be specific evidence as to specialised knowledge of the person in relation to that subject and as to the training, study or experience upon which that specialised knowledge is based. ... The further requirement that an opinion be based on specialised knowledge would normally be satisfied by the person who expresses the opinion demonstrating the reasoning process by which the opinion was reached.[12]
Subsequently in Pan Pharmaceuticals Ltd (in liq) v Selim, detailed reports prepared by individuals with extensive experience in the pharmaceutical industry were deemed inadmissible or to lack sufficient probative value because the ‘specialised knowledge’ was not identified and the relationship between the opinion, the specialised knowledge and any ‘training, study or experience’ was not explained.[13] Interestingly, deficiencies in the reports were not necessarily repaired, or said to be reparable, by additional testimony from the author.[14]
Most jurisdictions now have codes of conduct or practice notes regulating expert reports. Not rules of admissibility per se, codes and practice notes are important guides for expert witnesses in criminal and civil proceedings. They explain the duties owed by expert witnesses, what should be included in reports, and are intended to encourage experts to impartially prepare their evidence in order to facilitate legal decision-making. Non-compliance with codes of conduct and practice notes, especially minor omissions or oversights, is unlikely to lead
to the exclusion of evidence.[15] Nevertheless, compliance is generally vital for admissibility gatekeeping and gauging the probative value and weight of expert opinions, and yet the substantive requirements of the codes of conduct and practice notes are often overlooked in practice. Compliance with the terms and spirit of these procedural rules is essential for those engaged in plea and charge decisions, admissibility challenges via section 79 of the UEL, attempts to gauge probative value (even ‘at its highest’) according to sections 135 or 137 of the UEL, decisions about the need to obtain expert advice or to engage a rebuttal expert, decisions about how to cross-examine a witness, or frame directions and warnings under section 165 of the UEL.[16] Below we review the NSW Expert Witness Code of Conduct (‘NSW Code’),[17] the Victorian practice note entitled Expert Evidence in Criminal Trials (‘Victorian Practice Note’)[18] and the Australian Standard entitled Forenics Analysis: Part 4: Reporting (‘Forensic Reporting Standard’).[19]
Applicable to both civil and criminal proceedings, the NSW Code, revealingly, forms part of the Uniform Civil Procedure Rules 2005 (NSW).[20] Slightly revised in 2016, as part of standardization led by the chief justices of Australia, it was based upon a substantially similar practice note used in the Federal Court of Australia.[21] The Federal Court guidelines, developed decades ago in the aftermath of The Ikarian Reefer litigation and an influential review of civil justice in England conducted by Lord Woolf, have also recently been revised.[22]
According to the NSW Code, expert witnesses are engaged for the purpose of ‘providing an expert’s report for use as evidence’ or ‘giv[ing] evidence in proceedings or proposed proceedings’.[23] The expert witnesses’ ‘paramount duty’, ‘overriding any duty to the proceedings’ or retainer is ‘to assist the court impartially on matters relevant to the area of expertise of the witness’.[24] To make this crystal clear, the NSW Code states that an expert witness is ‘not an advocate for a party’.[25] Courts purport to place a premium on the impartial assistance of those with relevant expertise.[26] These obligations should inform interpretation of the NSW Code and the forensic practitioner’s performance;[27] preparing and presenting evidence in certificates, reports and oral testimony.[28]
The NSW Code is not exhaustive, but specifies things that ‘must’ be included in an expert report, many of which support the presentation of the evidence in a form that enables the judge to determine admissibility. Titled ‘Content of report’, clause 3 of the NSW Code states:
Every report prepared by an expert witness for use in court must clearly state the opinion or opinions of the expert and must state, specify or provide:
(a) the name and address of the expert, and
(b) an acknowledgement that the expert has read this code and agrees to be bound by it, and
(c) the qualifications of the expert to prepare the report, and
(d) the assumptions and material facts on which each opinion expressed in the report is based (a letter of instructions may be annexed), and
(e) the reasons for and any literature or other materials utilised in support of each such opinion, and
(f) (if applicable) that a particular question, or matter falls outside the expert’s field of expertise, and
(g) any examinations, tests or other investigations on which the expert has relied, identifying the person who carried them out and that person’s qualifications, and
(h) the extent to which any opinion which the expert has expressed involves the acceptance of another person’s opinion, the identification of that other person and the opinion expressed by that other person, and
(i) a declaration that the expert has made all the inquiries which the expert believes are desirable and appropriate (save for any matters identified explicitly in the report), and that no matters of significance which the expert regards as relevant have, to the knowledge of the expert, been withheld from the court, and
(j) any qualification of an opinion expressed in the report without which the report is or may be incomplete or inaccurate, and
(k) whether any opinion expressed in the report is not a concluded opinion because of insufficient research or insufficient data or for any other reason, and
(l) where the report is lengthy or complex, a brief summary of the report at the beginning of the report.[29]
The NSW Code requires the identification of ‘assumptions and material facts’ grounding opinions as well as the reasons for each opinion.[30] Persisting with common law terminology (eg, ‘field’), it also obliges the expert to specify: the scope of the opinion, if that is relevant; literature and other materials ‘utilised’; and ‘any examinations, tests or other investigations ... identifying the person who carried them out’.[31] Reports should include a declaration that all ‘desirable and appropriate’ inquiries have been made and that ‘no matters of significance ... have ... been withheld from the court’. The report should not omit information or qualifications ‘without which the report is or may be incomplete or inaccurate’. Similarly, if the opinion is constrained by ‘insufficient research or insufficient data or for any other reason’, this must be disclosed. However, in many
respects the requirements remain expressed in subjective terms, depending on
the individual expert’s stated ‘beliefs’, rather than embodying the expectation
of more comprehensive disclosure.[32] The NSW Code also contemplates the possibility of a forensic practitioner changing his or her opinion and the production of a supplementary report.
Finally, the NSW Code refers to expert conferences. Where experts participate in a conference they ‘must exercise ... independent judgment ... and must not act on any instruction or request to withhold or avoid agreement’.[33]
First published in 2014, and reissued in 2017, the Victorian Practice Note represents a more recent and important contribution in this domain.[34] As the title implies, it was specifically drafted for expert evidence in criminal proceedings in Victoria.[35] It serves as a useful comparator. Like the NSW Code and the earlier Federal Court practice note, the Victorian Practice Note imposes ‘an overriding duty to assist the Court impartially, by giving objective, unbiased opinion on matters within the expert’s specialised knowledge’.[36] Notwithstanding recent revisions to the NSW Code in an attempt to enhance harmonisation of procedural rules across jurisdictions, the Victorian Practice Note diverges in several significant ways, all of which represent an improvement on the NSW Code.
First, the Victorian Practice Note explains its purpose, namely, ‘to enhance the quality and reliability of expert evidence relied on by the prosecution and the accused in criminal trials’.[37] There is an explicit focus on quality and reliability in order to enhance the value of evidence and the efficiency of criminal proceedings.
Secondly, the Victorian Practice Note is more insistent and directed than the NSW Code. The Note employs the language of the Evidence Act 2008 (Vic) while reinforcing the centrality of impartiality. Paragraph 4.1, following federal and NSW courts, refers to expectations around impartiality, ‘objective’ and ‘unbiased’ performances, but also incorporates ‘specialised knowledge’ from section 79 of the UEL. This insistence is also found in the more detailed expectations in relation to the kinds of information that will inform understandings of quality, validity, reliability and probative value in the provisions specifying the content of reports. In addition to the requirements set out in the NSW Code, paragraph 6.1 of the Victorian Practice Note requires expert reports to disclose:
(i) any qualification of an opinion expressed in the report, without which the report would or might be incomplete or misleading;
(j) any limitation or uncertainty affecting the reliability of
(i) the methods or techniques used; or
(ii) the data relied on –
to arrive at the opinion(s) in the report; and
(k) any limitation or uncertainty affecting the reliability of the opinion(s) in the report as a result of –
(i) insufficient research; or
(ii) insufficient data.
The additional requirements in (j) and (k) are more directed and informative than the diffuse expectations expressed in the NSW Code. The Victorian Practice Note requires the impartial and unbiased witness to disclose any qualification that if omitted might lead to an ‘incomplete or misleading’ report. Interest in ‘reliability’, emphasised in the purposes section, reappears among the specific obligations. Whereas the NSW Code refers to the need to disclose if an opinion is not ‘concluded’, the Victorian version emphasises the obligation to disclose ‘any limitation or uncertainty’ that affects the reliability of the opinion, and thus might influence the interpretation of the report. The Victorian Practice Note makes it clear that there is an obligation to address limitations or uncertainty at the level of method and technique as well as limitations in data and/or research.[38]
Thirdly, and significantly, the Victorian Practice Note requires the expert to disclose the existence of controversy. Paragraph 6.2 states that if the expert is aware of any ‘significant and recognised disagreement or controversy’ that is ‘directly relevant to the expert’s ability, technique or opinion, the expert must disclose [its] existence’. Where there is non-trivial controversy or criticism there is an onus on the expert to disclose, even where they find the concerns unpersuasive or believe they do not materially affect the opinion. This is an objective standard. Where there is some non-trivial controversy or dispute that touches upon the evidence or the assumptions, underlying methods, interpretation or expression of results, the Note requires the report to draw it to the attention of the non-expert audience.[39]
Finally, the Victorian Practice Note refers to and applies to different kinds of reports. The Note applies to all reports that a party proposes to rely on in court, including primary reports and ‘responding’ reports, and also enables the accused to request that the prosecution obtain additional reports addressing a specific matter in dispute.[40]
As we shall see, the Victorian Practice Note is better suited to the provision and evaluation of expert evidence in criminal proceedings than NSW and federal equivalents. Nevertheless, it does not direct explicit attention to issues that are fundamental to a great deal of forensic science and medicine evidence, namely validity, error rates, human factors (eg, cognitive bias) and demonstrable proficiency.[41]
In this context it is also useful to draw attention to another important – though largely neglected – resource: the Forensic Reporting Standard, published in 2013.[42] The Standard lists most of the features required by the NSW Code and Victorian Practice Note, but also places emphasis on clarity, technical review, limitations and some dangers. Interestingly, the Standard is not referenced in most expert reports prepared by forensic practitioners in Australia. This omission appears to have much to do with an apparent reluctance, or inability, to comply with its strictures. This might be considered curious given that those responsible for the industry-oriented Australian Standards were employed by, or representing, Australian police services and other forensic science bodies.[43] NSW Police were represented on the relevant Australian Standards committee and the Forensic Services Group were consulted during drafting.
The Forensic Reporting Standard rewards reading, though for our purposes a few requirements warrant specific attention. The Standard places emphasis on the need for clarity and transparency in both what was done and what was concluded, but also ‘the limitations associated with the process’. These are provided for in the ‘General Provisions’:
In all instances, the author of a forensic report shall be concerned solely with reporting the results and opinions based on forensic examinations. Reports should be clear to the reader, so that it is readily understood what was done, what was concluded and the limitations associated with the process.[44]
Limitations are explicitly revisited in clause 9.4 of the Standard:
Known limitations of the methods or procedures and results and opinions used should be stated clearly and unambiguously in the report or an appendix attached to the report. This may include references to authoritative and critical literature, whether the method has been validated, and known error rates where available and relevant.[45]
The Standard refers to two types of review for reports, specifically ‘administrative review’ and ‘technical review’. Administrative review is a form of editorial review that may not involve any checking of technical data, results or opinions.[46] Technical review is of a different order:
The technical review shall include all observations, results and opinions in the case notes, to ensure their validity. Where a report has been subjected to technical review, evidence of the review shall be maintained.
All technical reviews shall be completed by a peer or authorized person. ... Where such technical review is not possible, the report shall contain a disclaimer to the effect that such review was not carried out.
Results or opinions on which agreement has been reached between the examiner and the reviewer may be included as results or opinions in the contents of a report.
If agreement between the examiner and the reviewer cannot be reached, an additional independent review may be conducted. Facilities shall have documented policies on the reporting of findings where a dispute exists between different examiners as to the evaluation of data or observations or the interpretation of results.
Disagreement between a reviewer and examiner shall be recorded.
The final decision as to the reported findings shall rest with the examiner when both of the following conditions are met:
(a) All relevant documented experimental protocols have been followed.
(b) Release of the disputed finding has been approved by the facility director.[47]
Self-evidently, the Forensic Reporting Standard descends into more detail than its legal counterparts. Technical review includes an assessment of ‘all observations, results and opinions in the case notes, to ensure their validity’. Not only does the Standard, written specifically for the forensic sciences, refer to formal processes of review, but it countenances the possibility of disagreement between forensic practitioners and those technically reviewing reports. Curiously, the Standard seems designed to leave decisions about whether to report disagreement to forensic examiners.[48]
Clause 9.1 lists things that should be included in a forensic science report. Most of these resonate with the requirements of the NSW Code and Victorian Practice Note. Clauses 9.2 and 9.3 respectively, refer to ‘Collection and continuity of forensic material’ and ‘Analysis and comparison of material’. Clause 9.5 addresses the ‘Reporting of opinions’:
Where professional judgment is involved in evidence collection, examination and/or analysis, opinions shall be differentiated from other sections of the report that may deal with factual observations, for example, scene description or continuity of possession.
Opinions derived from the interpretation of results should form part of the report’s conclusion. The conclusion should clearly outline the basis on which each opinion is formed and the process by which it is derived.
The author of an opinion shall take into account all relevant observations and results from the examinations and/or analyses. Care should be taken not to overstate or understate the value of any observations or testing carried out.
Opinions should not be presented in such a way as to overemphasize or underemphasize their certainty. The possibility of alternative explanations should be acknowledged. ...
If a client expressly requests an examiner to evaluate the results of their examinations in a particular context, then the context and the circumstances of the request shall be clearly stated in the report.
The report should clearly state which tests form the basis for an opinion.
Any reasonable, alternative explanations or opinions should be included together with reasons for their rejection or lower ranking.
Here we can discern the need to: distinguish opinions from other parts of the report; avoid overstating or understating the value of opinions; explain the basis for each opinion; carefully consider the information accessed; and, document what the forensic practitioner was given and told. There is, in addition, a need to consider reasonable, alternative explanations as well as reasons for their rejection or ranking.
The Standard descends into even more specificity for opinions linking a trace (eg, DNA, a fingerprint, a bullet casing or fibre) to a particular person or source: ‘common source’ attributions. Clause 9.5 continues:
Caution should be used in making common source opinions. When expressing opinions regarding a common source, examiners should –
(a) be aware of the logical objections to absolute individualization;
(b) be familiar with the relevant literature concerning the use of statistics in their field of expertise;
(c) avoid statements that cannot be supported by appropriate scientific testing; and
(d) recognize that opinion evidence is subjective in the sense that it entails professional judgement.
A result that, when interpreted, tends to support or tends to refute a hypothesis, should not be reported in such a way as to appear neutral.
These stipulations reveal concerns about opinions presented by those in the pattern recognition or comparison ‘sciences’ but the resulting expectations are to some extent incoherent. For example, the Forensic Reporting Standard appears to: recognise the logical impossibility of individualisation (in (a), in part reflected in the probabilistic approach applied to DNA evidence); require that statements, including those that positively identify (or individualise), must be ‘supported by appropriate scientific testing’ (in (c), notwithstanding the ‘logical objections’); and require those making illogical and unsubstantiated individualisations to be familiar with relevant statistical literatures that question the legitimacy of positive identification.[49] Rather than provide clear guidance, the Standard suggests the need for caution where forensic practitioners express opinions linked to a ‘common source’ as in the case of latent fingerprint evidence. Overall, we should expect disclosure and justification where forensic practitioners are engaged in ‘common source’ attributions.
In summary, the NSW Code, Victorian Practice Note and Forensic Reporting Standard impose obligations on forensic practitioners preparing reports for criminal proceedings that go beyond broad-brush commitments to nonpartisanship. They demand more than a statement that the practitioner agrees to be bound by the relevant rules. They offer a scheme intended to enhance the way lawyers, judges and other decision-makers understand and evaluate expert opinion evidence.
We now turn to consider an expert certificate relied upon in a recent prosecution based exclusively on latent fingerprint evidence.[50] More than a century after the High Court confirmed the admissibility of latent fingerprint evidence in Parker v The King there was a challenge to fingerprint evidence in JP v DPP.[51] JP, a minor, was tried and convicted of an aggravated break and enter in the Children’s Court in Dubbo. The case against JP was circumstantial, based entirely on him being ‘identified’ by a single latent fingerprint recovered from the scene, attributed to JP’s left thumb.[52] This was an individualisation or ‘common source’ attribution. The positive ‘identification’ was made by a NSW Police latent fingerprint examiner based in Dubbo.[53] On the basis of his assessment of the prints, the examiner produced an expert certificate, reproduced at the end of this article as Appendix I. The defence challenged the admissibility of the fingerprint evidence, including the certificate, at trial. Notwithstanding an unusually well-prepared challenge, the certificate and the examiner’s opinion were deemed admissible and relied upon to satisfy the magistrate beyond reasonable doubt that JP was responsible for the break and enter.[54] The sufficiency and admissibility of the fingerprint evidence, along with JP’s conviction, were subsequently raised in an unsuccessful appeal.[55]
JP v DPP was by no means the only challenge to the admissibility of fingerprint evidence in the decades following Parker v The King, although it is one of surprisingly few challenges, and the first to have drawn upon recent reviews that question some of the methods, practices and claims associated with fingerprint comparison.[56] The materials generated in JP v DPP, along with the production of a revised report template in its aftermath, afford a particularly good opportunity to critically examine the adequacy of reports and certificates produced routinely by state-employed forensic practitioners.
The expert certificate prepared for the investigation and used in the prosecution of JP is short and opaque. While it might be argued that it complies with the apparently minimal requirements of section 177 of the UEL, its form is inconsistent with the expectations expressed in the UCPR and the NSW Code. It is hard to imagine how a magistrate or judge could have determined whether the opinion was substantially based on ‘specialised knowledge’, and whether the knowledge was based on ‘training, study or experience’ for the purposes of section 79(1) of the UEL, let alone the value of the examiner’s conclusion. Rather than provide insight into the process and the reasoning, the certificate presents a small target. In the examiner’s opinion two different fingerprints – a latent fingerprint recovered from the crime scene and one in a police database – ‘matched’ and were therefore produced by the same person, namely JP. Apart from the bare references to retrieval and comparison, the certificate provides very little insight into what was done, what procedures were used and what standards applied. There are no references to limitations or uncertainties, no recognition of even the possibility of error, and no discussion of controversies. It is important to emphasise, given the legal response to the defence challenge, that the expert certificate prepared for the case is not unrepresentative.
Considering the admissibility of the expert opinion and the suitability of the certificate on appeal in the Supreme Court, Beech-Jones J found it to be inadequate:
nowhere in the certificate was there any statement of what [the] examination revealed. Instead there was simply a statement of the ultimate opinion formed ...[57]
I consider that the certificate provided by the prosecution’s fingerprint expert did not provide any reasoning sufficient to support the admissibility of the expert’s opinion, but the oral evidence of the expert rectified that discrepancy.[58]
Surprisingly, the inadequacies were said to be repaired by the answers supplied through the cross-examination undertaken by defence counsel.[59] We return to this issue in Part VI.
The outcome in JP v DPP throws the kinds of predicament facing defence counsel into sharp relief. On one hand, if admissibility is not challenged the evidence would be admitted and very likely relied upon, thereby foreclosing avenues of appeal. On the other hand, challenging the evidence, according to the judicial officers in JP v DPP, provided a corrective that rendered the certificate and oral evidence admissible. The limitations and frailties with the fingerprint identification procedure, the conditions under which it was undertaken, the nature of the conclusion, the examiner’s proficiency, and most significantly controversy within the domain, were not in the end disclosed by the Crown or addressed substantively by either of the judicial officers involved. At the trial level, the Court was not equipped to address fundamental issues raised by the defence. On appeal, relevant scientific literature was not before the Court. Nevertheless, both courts expressed a preference for the status quo: finding the positive identification evidence admissible and capable of suppoting guilt beyond reasonable doubt. Both judicial officers accepted that the witness remained ‘unshaken’ when confronted with authoritative scientific criticism.[60]
In response to the impending appeal in JP v DPP, NSW Police latent fingerprint examiners began to revise their certificates and reports.[61] The new template (‘Revised Certificate’) is reproduced at the end of this article, as Appendix II.[62]
The first thing to say about the Revised Certificate template is positive. It represents an improvement on what preceded it. Most Australian judges would presume this sort of certificate compliant with relevant procedural rules and deem derivative oral testimony admissible. We should, however, not get carried away with the likelihood or significance of legal endorsement. Considering the longevity and breadth of use, there have been remarkably few sustained challenges to the admissibility or probative value of latent fingerprint evidence.[63] After all, prosecutors have adduced and most courts have admitted the kind of non-compliant certificate tendered in JP v DPP, and similar reports from other forensic domains, for decades. In a large number of cases, where the identity of the offender was in issue, deficient reports passed without objection or judicial comment. The following discussion brings admission and legal credulity into focus. Rather than critical engagement, via cross-examination and sceptical legal norms, prosecutors, judges and defence counsel have, by and large, been acquiescent or complicit in the production, admission and reliance upon inadequate certificates and exaggerated opinions.[64]
At this point we turn to consider this Revised Certificate and, by implication, its predecessors in a little more detail. We are particularly interested in whether the Revised Certificate complies with the terms and spirit of the NSW Code and Forensic Reporting Standards, along with the degree to which it enables a reader to comprehend and evaluate the opinions offered by latent fingerprint examiners.
While the Revised Certificate represents an improvement on past practice there are still very significant problems. These include the description of the underlying procedure (ie, Analysis, Comparison, Evaluation, and Verification (‘ACE-V’)), the level of detail provided, the explanation of the reasoning, and the strength of the opinion expressed.[65] Most troublingly, the value of the evidence and the abilities of latent fingerprint examiners are exaggerated and serious limitations, uncertainties and controversies are not disclosed. Notwithstanding the provision of more descriptive information, those reading the Revised Certificate or hearing testimony based upon it are not placed in a position to rationally assess the putative expertise, the probative value or weight of the opinion.
Our first concern is with the way in which the Revised Certificate explains and frames the ‘methodology’ employed by latent fingerprint examiners. At a general level, the Revised Certificate asserts that:
The ACE-V methodology, as applied by qualified, practising fingerprint experts, has been subject to extensive research and validation studies and has been shown to be highly accurate, reliable and repeatable.[66]
In recent reviews of latent fingerprint evidence, conducted by the US National Academy of Sciences (‘NAS’), the US National Institute of Standards and Technology (‘NIST’) and the President’s Council of Advisors on Science and Technology (‘PCAST’), the allegedly straightforward and efficacious nature of ACE-V was authoritatively questioned. The NAS Report states:
ACE-V provides a broadly stated framework for conducting friction ridge analyses. However, this framework is not specific enough to qualify as a validated method for this type of analysis. ACE-V does not guard against bias; is too broad to ensure repeatability and transparency; and does not guarantee that two analysts following it will obtain the same results. For these reasons, merely following the steps of ACE-V does not imply that one is proceeding in a scientific manner or producing reliable results. A recent paper by Haber and Haber presents a thorough analysis of the ACE-V method and its scientific validity. Their conclusion is unambiguous: ‘We have reviewed available scientific evidence of the validity of the ACE-V method and found none’.[67]
Summarising the implications of the dearth of systematic study on the pattern matching or identification ‘sciences’ – that is, those involving common source attributions – the NAS Report embodies the concerns of attentive scientists, in terms unprecedented for their directness and critical tone:
With the exception of nuclear DNA analysis, however, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.[68]
Problems exposed through these reviews, many unanswered, were familiar to the latent fingerprint examiners responsible for drafting the Revised Certificate. These reviews and others cast latent fingerprint analysis, and the ‘method’ purportedly supporting it, in a light significantly less favourable than suggested in the Revised Certificate. Consequently, presenting ACE-V as a rigorous method capable of producing accurate identifications and effectively eliminating errors misrepresents what is known. It is not an impartial representation. The Revised Certificate misrepresents, through exaggeration and omission, both the value of ACE-V and its ability to eliminate errors, as well as the capacity of the available research to support the validity of the ACE-V ‘method’.
Only in the last few years have the first serious attempts to evaluate the performance of latent fingerprint examiners started, following notorious misattributions by FBI examiners who were applying ACE-V. In the Mayfield case the procedure resulted in three mistaken verifications.[69] Scientific studies in the US and Australia have found that qualified latent fingerprint examiners possess impressive abilities at matching and discriminating between prints. Studies by Ulery, Tangen and colleagues confirmed that latent fingerprint examiners are accurate, and are more accurate than laypersons.[70] However, the recent studies all reported small – though non-trivial – numbers of errors.[71]
All of the available studies were recently reviewed and summarised by PCAST,[72] which produced the following table:
Table 1: Error Rates in Studies of Latent Print Analysis.[73]
These studies and reviews represent the state of the science. This ‘state of the science’ is not consistent with the summary included in the Revised Certificate. These studies have not validated ACE-V, particularly its application.[74] Rather, the initial studies were concerned with examiners’ willingness and ability to correctly discriminate between fingerprints. That is, scientists have only tested the most fundamental components of fingerprint comparison. Significantly, the claim that ACE-V is valid and effective at eliminating errors is not supported by scientific research.[75]
Examiner recourse to scientific literature and the (strategic) deployment of scientific studies in reports and testimony raises two additional issues, worth considering in a section on methods and ACE-V. The first concerns the ‘training, study or experience’ of latent fingerprint examiners. Many latent fingerprint examiners, most conspicuously those employed by state police services, do not possess formal scientific qualifications. Technically, they are not forensic scientists.[76] While we accept that certified latent fingerprint examiners have demonstrable abilities at comparing prints, their expertise may not extend to research methods, statistics and probabilities, or cognitive science, in order to mitigate contextual biases. This means that where methods and practices are challenged, latent fingerprint examiners and other ‘technicians’ may not be competent to responsibly answer questions and criticisms.[77] Recall the NSW Code’s insistence that experts should not transgress the ‘field of expertise’.[78] The Forensic Reporting Standard cautions them not to ‘knowingly go beyond their area of expertise’.[79]
Breach of these fundamental strictures passes without critical comment in JP v DPP. The examiner ‘conceded that he had not read a lot of the literature referred to by [counsel] in the cross-examination’.[80] Nevertheless, ‘he maintained his view that if the protocol [ie,‑ ACE-V] was followed properly it should not involve bias or incorrect assessment’.[81] The position adopted by the examiner, and supported by the magistrate and judge on appeal, is at odds with the best scientific research and advice. The ‘protocol’ does not necessarily protect against bias, error or inconsistency. Here we can observe a legally-recognised expert, apparently oblivious to or unwilling to engage with the detail of critical reviews by independent and authoritative scientific and technical organisations, and apparently unfamiliar with the detail of supportive research by Ulery et al and Tangen, Thompson and McCarthy, adhering to personal beliefs and impressions.
The second point is that all of the latent fingerprint groups in Australia are aware of the NAS Report, the NIST Report, the Fingerprint Inquiry Report as well as a range of other studies and commentaries. The decision to omit criticisms and even references to these reports from the Revised Certificate is deliberate.[82]
A related deficiency is the failure to explain the reasoning process that has been used by the fingerprint examiner. This was at the centre of the challenge in JP v DPP and yet even the Revised Certificate provides only a schematic overview of the components of ACE-V. The results of analysis, comparison and evaluation, and the specific decisions – around the features observed, their number, any ‘distortion[s]’, and apparent (ie, superficial) differences as well as identifications and non-identifications are either omitted or asserted rather than explained.[83] This appears inconsistent with the requirements of section 79 of the UEL as well as Makita and Ocean Marine v Jetopay, the NSW Code, the Victorian Practice Note and the Forensic Reporting Standard.
We are not provided with information about: how an examiner determines whether a print is sufficient for analysis and comparison; how the search on the National Automated Fingerprint Identification System (‘NAFIS’) was conducted and whether there was more than one; the type and number of features observed, and how these led to the particular decision – whether ‘Identified’, ‘Inconclusive’, ‘Not identified’ or not suitable for searching.[84] The Revised Certificate does not incorporate or refer to marked-up images (or an expectation that these will be made available to the defence). The Revised Certificate only provides information as to the conclusions of the named expert. We are told about a form of review by way of the general description of the procedure contained within annexure 3, but the Revised Certificate does not indicate who performed the review, what it involved and the circumstances in which it took place. Conclusions remain declaratory; they resemble ‘bare ipse dixit’.[85]
The Revised Certificate describes the ACE-V process in simplistic terms, but there is no explanation of the actual practices or interpretation in the Revised Certificate. There is no indication as to whether ACE-V is undertaken sequentially.[86] Was, for example, ‘Analysis’ conducted before ‘Comparison’, in order to reduce vulnerability to contextual bias? Were the results of each stage documented sequentially? Was ‘Verification’ blind and did the ‘Verification Expert’ agree with the ‘Evaluation’?[87] None of this is discernible. Finally, the conclusions are reported in a tabular form and contain only a single reference to the conclusions being ‘opinion’ (important in the NSW Code, Victorian Practice Note and Forensic Reporting Standard) rather than ‘statement[s] of fact’.[88]
The simplistic description of the ACE-V ‘method’ belies the variability inherent in its operation. The available research suggests that analysis, comparison, evaluation and verification is much more variable than implied by the outline of ACE-V presented in the Revised Certificate.[89] There are few, if any, standards or guidelines regulating practice. What is a feature and how many features or combinations of features are required to call a latent print sufficient for analysis, and to identify or exclude? What makes a latent print too distorted for analysis?[90] When can anomalies be considered to be caused by distortion or some other interference? How much variation should be tolerated? The lack of standardisation is inconsistent with the advice of independent scientific reviewers and likely to introduce uncertainty and increase (subjective) variation and error in fingerprint comparison and evaluation regardless of whether these are reported.[91]
The Revised Certificate tends to extrapolate from relatively limited research on the capacity of (some) latent fingerprint examiners to differentiate between prints to general conclusions as to the validity and accuracy of ACE-V. A corollary of this is the manner in which the Revised Certificate deals with error. In approaching limitations, uncertainty and errors, it is useful to reproduce the relevant section, revealingly titled ‘Potential for Error’:
The comparison of fingerprint impressions is a task conducted by humans, and subsequently there exists a potential for error. However, studies have demonstrated that qualified, practicing fingerprint experts are ‘exceedingly accurate’ when performing fingerprint identifications. To mitigate risk of error, NSW Police Force – Forensic Services Group incorporates strict peer review practices requiring independent verification of fingerprint identifications by a minimum of one appointed verification expert. My conclusion(s) is not a statement of fact, but one of expert opinion.[92]
In its review of the forensic sciences, the NAS Report adopted a more concrete approach to error and uncertainty:
Few forensic science methods have developed adequate measures of the accuracy of inferences made by forensic scientists. All results for every forensic science method should indicate the uncertainty in the measurements that are made, and studies must be conducted that enable the estimation of those values.[93]
Subsequently, following a review of the surprisingly sparse scientific research, PCAST concluded that:
it would be appropriate to inform jurors that (1) only two properly designed studies of the accuracy of latent fingerprint analysis have been conducted and (2) these studies found false positive rates [ie, misidentifications] that could be as high as 1 in 306 in one study and 1 in 18 in the other study. This would appropriately inform jurors that errors occur at detectable frequencies, allowing them to weigh the probative value of the evidence.[94]
Provision of this information is fundamental because misidentification (ie, the ‘false positive rate’) ‘is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis’.[95]
Error is not taken seriously in the Revised Certificate. It is not treated as a real and ubiquitous feature of fingerprint comparison.[96] There is no indication of its incidence or magnitude. Rather, it is represented as something that is implicitly abstract and remote – merely ‘potential’. In the Revised Certificate, the remote possibility of error is attenuated by a misleading reference to the accuracy of fingerprint examiners, recourse to ‘independent verification’, and characterising identification evidence as opinion rather than fact.[97] We address claims about the value of verification in more detail in Part V(F). In the present section we consider the Revised Certificate’s failure to engage with error in the empirical terms implored by the NAS along with the incidental references to the evidence being opinion.[98]
The main limitation is the trivialisation of the real risk of error. Not only is error characterised as abstract and remote, the abilities of examiners in conjunction with their ‘method’ are presented as eliminating even the remote possibility. But, as we have seen in the independent reviews and formal studies, there is little evidence that ACE-V actually reduces or eliminates error. Responding to the submissions of fingerprint examiners, the NAS Report concluded that: ‘Although there is limited information about the accuracy and reliability of friction ridge [ie, fingerprint] analyses, claims that these analyses have zero error rates are not scientifically plausible’.[99]
The Revised Certificate misrepresents the conclusion from Tangen, Thompson and McCarthy cited in support of the accuracy of fingerprints. The relevant paragraph from their conclusions is reproduced below:
We have shown that qualified, court-practicing fingerprint experts are exceedingly accurate compared with novices, but are not infallible. Our experts tended to err on the side of caution by making errors that would free the guilty rather than convict the innocent. Even so, they occasionally made the kind of error that can lead to false convictions. Expertise with fingerprints appears to provide a real performance benefit, but fingerprint experts – like doctors and pilots – make mistakes that can put lives and livelihoods at risk.[100]
The bare claim that fingerprint examiners are ‘exceedingly accurate’ is a partial and potentially misleading use of the findings from this independent scientific research.[101] Significantly, the elision of error and fallibility occurs in the section of the Revised Certificate purporting to deal with the subject of error. Actual cases, as well as findings by Tangen, Thompson and McCarthy and Ulery et al (reported by PCAST), confirm that latent fingerprint examiners make small numbers of errors, including ‘false alarms’ (or misidentifications).[102]
By failing to genuinely engage with uncertainties and error, the NSW Police latent fingerprint examiners have not provided decision-makers with means
of evaluating their reports and testimony.[103] While trained and certified latent fingerprint examiners obviously possess genuine expertise – demonstrated in their objectively high accuracy, and accuracy relative to novices – the only independent studies available reported mistakes. Consequently, without insight into actual abilities and general levels of performance, how is the decision-maker to determine whether to accept a particular opinion? How is the decision-maker in JP v DPP to determine whether this opinion evidence can support proof beyond reasonable doubt?[104] The Revised Certificate does not place the decision-maker in a position to rationally evaluate the opinion.[105] In the absence of genuine engagement with error and its incidence, the Crown always gets the benefit of a process that is misleadingly characterised as effectively error free.
The other issue emerging in annexure 3 to the Revised Certificate is the declaration that any identification (or exclusion and so on) is merely an opinion. According to the NSW Code and the Forensic Reporting Standard, this should be made explicit in the text where the opinion (ie, the identification) is expressed. While the change in nomenclature from ‘fact’ to ‘opinion’ is appropriate, and consistent with the recommendations of the Fingerprint Inquiry Report, merely characterising a conclusion as opinion does not address validity, uncertainty, limitations or error. Conceding that something is an opinion does not mean that its probative value will be appropriately discounted by non-technical audiences. Moreover, it does not provide a means of gauging probative value or weight.[106]
One of the problems that plagued the original certificate in JP v DPP and the oral testimony that followed, is that apparent matches between fingerprints were equated with identification in categorical terms. In JP v DPP, the match was characterised as positive identification (or individualisation) of JP. The Revised Certificate maintains this categorical approach in its terminology.[107]
According to the NIST Report:
a fingerprint identification was traditionally considered an ‘individualization,’ meaning that the latent print was considered identified to one finger of a specific individual as opposed to every other potential source in the universe. However, the recent attention focused on this issue reveals that this definition needlessly claims too much, is not adequately established by fundamental research, and is impossible to validate solely on the basis of experience.[108]
The original certificate, oral testimony and Revised Certificate all ‘needlessly claim too much’. Identification or individualisation is inconsistent with the best scientific advice, as well as practices in analogous comparison procedures that have been formally validated – eg, DNA profiling. Scientifically-based DNA profiling evidence is, for example, reported in probabilistic terms or as a likelihood ratio derived from the application of population statistics to genetics. The results of DNA testing are almost never reported in terms of positive identification. With latent fingerprints, in contrast, we have a ‘protocol’ that is not standardised, let alone appropriately validated, and yet courts allow latent fingerprint examiners (many without scientific training and qualifications) to express their subjective interpretations as positive evidence of identity and to reject any real possibility of error.
Independent scientists and statisticians who have reviewed the procedures used by latent fingerprint examiners have advised against positively identifying persons in most circumstances.[109] Recommendation 3.7 of the NIST Report states:
Because empirical evidence and statistical reasoning do not support a source attribution to the exclusion of all other individuals in the world, latent print examiners should not report or testify, directly or by implication, to a source attribution to the exclusion of all others in the world.[110]
That is, in most cases, latent fingerprint examiners should not attribute latent fingerprints to a specific individual. In the Fingerprint Inquiry Report, Sir Anthony Campbell recommended that ‘[e]xaminers should discontinue reporting conclusions on identification or exclusion with a claim to 100% certainty or on any other basis suggesting that fingerprint evidence is infallible’.[111]
These authoritative recommendations are neither adopted nor disclosed in the Revised Certificate.[112]
The challenge in JP v DPP included some discussion of the potential for human factors (such as confirmation and contextual biases) to influence the decisions of latent fingerprint examiners. The defence pointed to the limited nature of the comparison the examiner was asked to perform, as well as the fact that he was already aware that the fingerprint had been linked to JP when undertaking his assessment.[113] The Revised Certificate provides more detail than the original certificate, but it does not address vulnerabilities arising from human factors.
Vulnerabilities from human factors, such as suggestion, anchoring and reliance on recollection of the frequency of features, are not ‘hypothetical’.[114] Scientists and biomedical researchers are routinely blinded to avoid notorious dangers, particularly suggestion. Formal training and extensive experience do not enable scientists to resist a range of insidious cognitive influences. This is one of the reasons that most clinical trials are double-blinded. Neither the treating doctors nor the patients are aware of who is receiving the test drug and the comparator or placebo. The US National Commission of Forensic Sciences, established after the NAS Report, explained:
Contextual bias is not a problem that is unique to forensic science. It is a universal phenomenon that affects decision making by people from all walks of life and in all professional settings. People are particularly vulnerable to contextual bias when performing tasks that require subjective judgment and when they must rely on data that are somewhat ambiguous.
Studies show that the contaminating impact of contextual bias can occur beneath the level of conscious awareness. This finding means that contextual bias is by no means limited to cases of misconduct or bad intent. Rather, exposure to task-irrelevant information can bias the work of FSSPs [forensic science service providers] who perform their job with utmost honesty and professional commitment. Moreover, the nonconscious nature of contextual bias also means that people cannot detect whether they are being influenced by it. It follows that task-irrelevant information can bias the work of FSSPs even when they earnestly and honestly believe they are operating with utmost objectivity.[115]
It is difficult to reconcile the systematic use of blinding procedures in mainstream scientific and biomedical research and the widespread insensitivity to the same risks in the day-to-day work of forensic practitioners.
Scientists reviewing the forensic sciences have repeatedly recommended the use of blinding – as early and for as long as possible.[116] This means exposing forensic practitioners only to information that is required to successfully perform their analysis – such as the provision of prints and maybe information about the surfaces on which the prints were located, along with the methods used to collect and enhance them, for example.[117] This is often described as task- or domain-relevant information. Latent fingerprint examiners do not need to know about other evidence such as a confession, the opinions of police officers about the identity of the offender, the type and seriousness of offence, the suspect’s criminal record or address, and so on. Indeed, exposure to these kinds of gratuitous information has the potential to contaminate decisions – especially where the interpretive task is difficult.[118]
The NAS Report places emphasis on the need to study and reduce the
threats posed by contextual bias.[119] The Fingerprint Inquiry Report made three recommendations, designed to reduce exposure to domain-irrelevant information and record exposure where it occurs.[120] The NIST Report on latent fingerprints was primarily concerned with threats posed by bias and other human factors such as vision. The full title of the report is Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach.[121] More recently, the National Commission on Forensic Science made the following recommendations:
1. FSSPs [forensic science service providers] should rely solely on task-relevant information when performing forensic analyses.
2. The standards and guidelines for forensic practice being developed by the Organization of Scientific Area Committees (OSAC) should specify what types of information are task- relevant and task-irrelevant for common forensic tasks.
3. Forensic laboratories should take appropriate steps to avoid exposing analysts to task- irrelevant information through the use of context management procedures detailed in written policies and protocols.[122]
Dangers posed by human factors are neither marginal nor remote and easily resisted by forensic practitioners. Human factors and the risks they introduce are central to recent reviews of the forensic sciences and recommendations for reform. The Revised Certificate does not refer to the dangers, it does not document the information provided or available to the examiner, or whether the ‘Verification’ process was blind or suggestive.
This omission is not only intentional, it ignores the fact that in response to the NAS Report and NIST Report recommendations, the FBI has refined the way it applies ACE-V. The FBI has adopted ‘linear ACE-V’, a ‘procedure [that] involves temporary masking of reference prints while analysts make and record their initial assessments of the evidentiary prints’.[123] So-called linear ACE-V involves the analyst proceeding through ‘Analysis’ and ‘Comparison’ in sequence and ‘blind’.[124] The examiner should make an assessment of sufficiency and mark the meaningful features of the unknown or latent print before undertaking a comparison with the known print. This marking should be documented and recorded at the time.[125] Latent fingerprint examiners in the pre-eminent American investigative bureau have revised the way they perform ACE-V in response to emerging dangers. The fingerprint examiner in JP v DPP, in contrast, appears to have engaged in analysis and comparison simultaneously. The NSW Police have not revised their procedures to avoid these notorious dangers and there are no references to human factors, alternative procedures, or dangers in the Revised Certificate.
Rather than require forensic practitioners to guard against risks, the magistrate and judge in JP v DPP placed an expectation on the defendant to somehow demonstrate where and how the examiner’s ‘determination was tainted by ... bias or other incorrect assessment’.[126] Such an approach is unrealistic and inconsistent with the allocation of the burden of proof in criminal proceedings.[127] It is the responsibility of the Crown to eliminate doubts whether introduced through inadequate procedures or other sources. We should not expect impecunious defendants to persuade credulous judicial officers, or jurors, of the reality of these dangers or explain their impact on the probative value (or weight) of forensic science evidence.
Verification is the independent analysis, comparison and evaluation of the friction ridge detail carried out by another qualified fingerprint examiner. In the NSW Police Force Forensic Services Group, the verification step is undertaken by a Verification Expert, who is a senior, practicing fingerprint expert appointed to that role based on their skills, knowledge, training and experience in fingerprint analysis.[128]
Reliance is placed on ‘Verification’ in the Revised Certificate. Verification is presented as a form of peer review that removes the ‘potential for error’. The Revised Certificate states that it is performed by a senior latent fingerprint examiner who purportedly undertakes a second and ‘independent analysis, comparison and evaluation’. We question whether verification reduces errors in the stages of analysis, comparison and evaluation. There is little evidence that verification, particularly where the reviewer is not blind, operates in the manner suggested.[129]
The first thing to note is that we are not provided with much detail about ‘Verification’. What does it involve? How are verification experts selected for particular cases? Is everything verified? What information is provided to the examiner engaged in verification? Does, for example, the verifying examiner know the original result? Is the verifying examiner provided with the original examiner’s work notes and conclusions or is the review conducted de novo? Does the fact of, or the need for, verification suggest that another examiner has identified a particular print? If the examiner performing verification knows the original result, even if only by implication, then in what sense can the verification be said to be independent? If the examiner is not provided with the original result, what blinding mechanisms are in place to avoid formal or informal suggestion?[130]
We know that verifiers do not begin the process anew. They rely on the prints and comparators selected by the original examiner. They are only asked to review prints that are said to match – ie, identified to a person of interest. So, the ‘verification expert’ comes to the process – regardless of what they are told and the materials provided to them – aware that another (possibly identified) examiner has already attributed a latent fingerprint to a specific person. The ‘verification expert’ reviews the fingerprints on this suggestive basis. There is no information available to the public about the frequency with which those engaged in verification disagree with original conclusions or question decisions around sufficiency or conclusiveness. The absence of information could be because latent fingerprint examiners do not make mistakes, but this is unlikely and inconsistent with the emerging scientific studies of latent fingerprint examiners in Australia and the US.[131] A more likely explanation is that the way verification is performed in NSW undermines its potential. Verification has not been evaluated. If mistakes or even disagreements are exposed through verification they are not routinely disclosed.[132]
The Revised Certificate’s reliance on accreditation and proficiency testing, to guarantee the method and approach, overstates the significance of compliance with the generic standards. Revealingly, there are no references to the Forensic Reporting Standard for the forensic sciences or the need for special caution with ‘common source opinions’.
The only standard referenced in the Revised Certificate is the International Standard entitled General Requirements for the Competence of Testing and Calibration Laboratories (‘International Standard’), which does not support the validity and reliability of practices undertaken by latent fingerprint examiners.[133] Rather, and in slightly simplified terms, the International Standard is of a general nature and intended to confirm that practices in scientific laboratories are basically consistent with written procedures. The Standard and its assessment by the National Association of Testing Authorities (‘NATA’) involves a paper audit of practices, performed by industry insiders.[134] Given that there are relatively few detailed protocols for ACE-V, many aspects of fingerprint analysis, comparison and evaluation are not standardised, and conformity with the Standard reveals little about the probative value of the evidentiary products.[135] Of significance, there are few specific requirements for reports to be compliant with the Standard. Moreover, we should not overlook the fact that NATA, the organisation responsible for accrediting the NSW Police Forensic Services Group in accordance with the Standard, approved the issuing of certificates and reports – like the one relied upon in JP v DPP – prior to the start of validation research in 2009 and continuing to 2017.
Accreditation is generally highly desirable, but only where the underlying standards and practices, informing accreditation, are valid and reliable. Accreditation against procedures that are not validated or not operationalised in ways that help to eliminate risks tends to be a ‘whitewash’. Reference to accreditation in the Revised Certificate is used to suggest that procedures and reports are epistemologically robust when all that is being confirmed is that institutions are basically compliant with their own procedures. These may, as in the case of non-blind verification, be untested and not standardised.[136]
Similarly, the proficiency tests used by most Australian police services, for the forensic sciences, are supplied by commercial providers. Something of a misnomer, they do not represent a credible test of proficiency. They are not designed to identify limitations and errors or to improve methods.[137] While even weak proficiency tests might occasionally identify a problem, this is not how they are generally deployed. Rather, what they accomplish for the NSW Police Forensic Services Group is to satisfy a prerequisite for accreditation.[138]
The Revised Certificate does not refer to authoritative independent reviews such as the NAS Report, NIST Report and Fingerprint Inquiry Report. From our perspective, the omission of such prominent reports is not an omission that a group endeavouring to impartially serve the courts would make.
Over the last two decades there have been a number of high profile reviews by the FBI, NAS, NIST, PCAST and Sir Anthony Campbell, following notorious mistakes in the US and UK.[139] There have also been a large number of critical papers and commentaries by attentive scholars from a variety of disciplinary backgrounds.[140] These are materials that would assist a lawyer or scientist to assess the probative value of latent fingerprint evidence.[141] They explain procedures as well as outline deficiencies and limitations. They discuss relevant research, missing research and place concerns about methodological limitations and managing human factors in context.
Before moving to consider some of the broader implications of our review, we want to respond to the contention that the oral testimony rectified the inadequate report in JP v DPP. Without descending into detail, our reading of the trial transcript casts doubts on both the adequacy of the testimony of the Crown’s fingerprint expert and the ability of that testimony to repair the shortcomings in the certificate or substantially answer the issues raised during cross-examination. To the extent that the Revised Certificate replicates some of the limitations evident in the original certificate, the same issue arises. The suggestion that answers supplied during oral testimony somehow addressed the kinds of issues raised in this article trivialises the deeply destabilising nature of these fundamental scientific oversights. Notwithstanding inconsistent judicial conclusions, the subjects raised in this article are precisely the kinds of factors that ‘materially [affect]’ the weight of the examiner’s ‘opinion that the fingerprints were identical’.[142]
It is not our intention to suggest that this fingerprint evidence is, or should be, inadmissible. Nor do we contend that the identification in JP v DPP was necessarily mistaken. Rather, as we explained in the previous section, our concerns are primarily with transparency, methodological rigour, legitimate forms of expression and the provision of means to appropriately evaluate or weigh the evidence. There are serious and unanswered limitations with the ‘method’, standards and consistency, categorical expression (ie, individualisation), vulnerability to bias and so on. While it is difficult to imagine a trial judge excluding fingerprint evidence in jurisdictions insensitive to validity and reliability, obviously there are serious problems with the over-claiming associated with individualisation and the non-engagement with scientific research, limitations and error.
Neither the magistrate nor the judge seem to have recognised the magnitude of the epistemic issues raised by the defence. Difficulties experienced by judicial officers would seem to be difficulties that might confront jurors where they are presented with latent fingerprint evidence. Problems identified by authoritative scientific reviewers, explored with varying degrees of insight and clarity during the trial, were not credibly addressed by the magistrate or the judge. The magistrate’s limited response to the admissibility challenge is reproduced below:
In this matter I have oral and written evidence from [the examiner]. His evidence was unshaken on his view as to the matching of the thumbprint of [JP]. In my view I disagree with the submissions in this matter, he has given sufficient evidence in these proceedings as to how he reached that determination. As an expert his expertise was not shaken, his opinion was not shaken. He is tasked, as he said, purely to compare W3 to [JP’s] prints. There is clearly in terms of the procedures involved, checks and balances in place. He acknowledged he is aware of case studies where potential impacts and bias of proceedings have occurred. His view as the expert in the field or presented as the expert in this matter is that where the appropriate procedures have taken place, is unlikely to have those errors occur. He also conceded that he had not read a lot of the literature referred to by Ms Graham in the cross-examination. Again he maintained his view that if protocol was followed properly it should not involve bias or incorrect assessment.
The difficulty of course with a lot of material that was cross-examined on is there is no method, no chance to actually test the validity of those arguments.
...
I have no evidence before this Court of the method used in this instance by [the examiner] ... [not transcribable] ... helping assist in his determination was tainted by the bias or other incorrect assessment by not following the protocols. I have no expert evidence on the defence showing in this particular matter that the thumbprint is not or could not be the accused’s. I say that of course there remains at all times the prosecutions responsibility to prove the matter beyond reasonable doubt. It was suggested that [the examiner] was contradictory or failed to make proper concessions, I actually find to the contrary. He answered appropriately in all circumstances especially where the questions were extremely open-ended and hypothetical. He did not attempt in any way to make his evidence or his position any greater than what it should in terms of the protocols that were involved.
...
The evidence by [the examiner] in giving his opinion in determination has not been proved forensically challenged in this matter. There is no Court decisions making such material unacceptable. What has been raised and I accept is that perhaps it is unreliable. ... At best I have nothing else binding before me that would exclude the evidence of [the examiner]. I can only scrutinise it on the material before me specific to this case. I accept [the examiner’s] evidence in that regard.[143]
The magistrate treated defence concerns about methodological limitations, exaggerated expression and bias as hypothetical issues. Rather than require the Crown to demonstrate that its routine procedures protect examiners from scientifically notorious dangers, there is an expectation that the defence will identify actual mistakes and errors – implicitly through an expert witness. Defence concerns, built on authoritative scientific literature and recommendations, are dismissed because ‘there is no method, no chance to actually test the[ir] validity’. This response is unprincipled. Rather than require the Crown to credibly demonstrate that procedures routinely used to identify persons are valid and performed in ways that minimise known risks of error, the magistrate discounted the impact of questions and criticisms because he held that there was no chance to test the validity of the questions and criticisms and no alternative expert witness asserted that the fingerprint was not JP’s. The advice and recommendations of independent scientists and modifications to practices in response to scientific advice around bias (such as linear ACE-V), by leading forensic science institutions (such as the FBI) might be thought to disrupt the dismissive attitude toward the defence’s concerns.
Consider also the superficial treatment of continuing problems with latent fingerprint evidence in the appeal judgment:
[w]hile a number of criticisms were made of [the examiner’s] evidence it was open to his Honour to conclude that there was no material to indicate that, to the extent the criticisms were sustained, they materially affected the weight to be attached to [the examiner’s] opinion that the fingerprints were identical. Otherwise his Honour had the distinct advantage of being able to observe [the examiner] give evidence and respond to criticism.[144]
In context, this summary is not merely unpersuasive, but surely mistaken. The criticisms and their sources must materially impact upon the weight of the opinion. The exaggerated form of the opinion (ie, positive identification), misrepresentation of the ‘method’ and its value, inattention to limitations, uncertainty and error, omission of and non-engagement with contextual bias and human factors, and the examiner’s reluctance or inability to make appropriate concessions must reduce probative value and weight.[145] Furthermore, the ability to observe the examiner’s demeanour and responses to questions did not provide the magistrate with any significant advantage over those reading through the transcript or this article.[146] The impressions and beliefs of a latent fingerprint examiner, however experienced, do not overcome scientific studies or their absence, insufficiently detailed protocols, and exaggerated abilities and conclusions.[147] The performative dimensions of the exchanges can hardly be a factor in addressing the probative value of the procedure or the propriety of categorical ‘common source attributions’. To the more technically proficient, the examiner’s performance appears naïve. Several responses in cross-examination were misguided, partial or misleading.[148] They were not informed by relevant ‘specialised knowledge’.
Nevertheless, this was law’s crude attempt to enable engagement with epistemic issues confronting fingerprint evidence emerging a century after Australian courts first began to rely upon it.[149] Decisions by the magistrate and judge privilege the status quo but without credible engagement with the validity and reliability of the evidence. Courts and latent fingerprint examiners seem very reluctant to engage in appropriate forms of re-calibration.
While the Revised Certificate does represent an improvement over the certificate relied on in JP v DPP and earlier reports, it does not constitute a sufficiently transparent or serious engagement with the broad range of issues and challenges confronting contemporary latent fingerprint examiners. It does not provide a clear indication of the known value of the conclusion or the means to assign one. In concluding, we intend to raise some of the policy implications. These include the system costs flowing from judicial accommodation and legal reliance on the adversarial process both to identify deficiencies in the evidence and, paradoxically, repair those deficiencies. Excusing routinely deficient certificates and reports is unlikely to encourage widespread institutional reform or scientific research. Many forensic practitioners look to the courts, rather than relevant scientific communities, for recognition and legitimacy. If a contested, inadequate and purportedly expert certificate or report can be repaired through cross-examination, how are those accused of criminal offences expected to make sensible pleas or tactical decisions ahead of trial?
When it comes to negotiating pleas and prosecuting criminal offences it is essential that opinions said to be based on ‘specialised knowledge’ are presented in ways that enable the application of admissibility rules and facilitate rational evaluation (UEL s 79). Forensic science and forensic medicine evidence adduced by the Crown should satisfy jurisdictional admissibility rules and, relatedly, codes or practice notes regulating the form of expert reports and certificates. This requires, at the very least, clear indication of the personnel involved, the procedures and reasoning employed, along with identification of non-trivial limitations, uncertainties and any controversy. Most forms of scientific, medical and technical evidence adduced by the Crown, particularly those in regular use, should refer to the studies or research supporting the underlying procedure (eg, independent validation), specify limitations and uncertainties (ideally including indicative error rates), and describe how risks raised by human factors were managed and mitigated. Conclusions should be expressed in terms that are linked to empirical information and known abilities rather than impressions. Such practices and the information they provide are based on knowledge.
Codes of conduct, practice notes and admissibility rules insist on: ‘specialised knowledge’; opinions to be ‘wholly or substantially based on that knowledge’; and, for the specialised knowledge to be based on the person’s ‘training, study or experience’. In general, these will not be satisfied by an individual – however experienced – declaring that they possess ‘specialised knowledge’ or expertise.[150] More is required.[151] Cases such as HG v The Queen, Dasreef, Honeysett, Ocean Marine v Jetopay, Makita and Campbell v The Queen have explained the need to identify the ‘specialised knowledge’ and explain how the opinion is based on it. The process of reasoning should be made clear: ‘[t]he jury cannot weigh and determine the probabilities for themselves if the expert does not fully expose the reasoning relied on’.[152] In this regard, state-employed forensic practitioners should be model expert witnesses.[153]
In closing, it is useful to respond to a few potential objections, concerning costs, the ability of adversarial proceedings to address or overcome problems, the risk of overwhelming the lawyers, and legal engagement with scientific knowledge and advice.
For readers tempted to suggest that the Crown’s forensic science and medicine reports are generally adequate, and that the trial represents the appropriate forum for exploring deficiencies, the historical failure of lawyers and judges to recognise and respond to serious limitations, such as those associated with the comparison forensics, should loom large. For a hundred years, most prosecutors (as ministers of justice), most defence lawyers (advocating on behalf of clients claiming to be innocent), and most judges have accepted claims by fingerprint examiners that were not empirically-grounded or supported by scientific research. No Australian court has ever required fingerprint examiners, firearm and tool mark examiners, fire investigators, shoe and footprint examiners, forensic odontologists (for bite marks), those engaged in image and voice comparisons and so on, to produce the results of validation studies or to demonstrate actual proficiency doing the specific task – eg, discriminating between fingerprints, or bullet casings, or voices, or comparing persons in images for the purpose of identification, as in Honeysett.[154] Trial and appellate judges tend to be unsympathetic, or perhaps oblivious to, the issues animating mainstream scientific and medical concerns.[155]
On this point, it is not and cannot be the responsibility of the defendant and his or her lawyer to somehow identify errors, uncertainties and frailties retrospectively, possibly months or years after the original collection and analysis of materials.[156] Those producing and relying on forensic science evidence are the only ones in a position to formally evaluate procedures and disclose limitations.
The inexorable issue of cost is the second potential objection. It might be argued that providing more information would be too costly for forensic practitioners and lawyers. It might be that more detailed reports will be more expensive, though there is little evidence to support that proposition. Many forensic science providers are currently expanding their reports in response to authoritative advice and new versions of codes and practice notes, such as the Victorian Practice Note. Even the NSW Police appear capable of revising their fingerprint reports, as the Revised Certificate makes clear. While there may be additional costs, at least initially, in drafting new report or certificate templates, these are not prohibitive. Moreover, for standard procedures the basic report or certificate template is generic. Provided the template captures the necessary information, forensic practitioners just have to fill in a few sections rather than redraft a report in every case.
Thirdly, there is a danger that longer or more informative reports might overwhelm the lawyers. This is not a particularly serious threat.[157] Expert reports should provide relevant information to assist prosecutors, defence lawyers and their clients, and judges make sensible decisions about the evidence and their options. Providing too much information or unnecessary information is inconsistent with what an impartial expert should do. Nevertheless, if the choice is between too little or too much information, the preference would almost always be to provide more than less. Experienced lawyers may not have to read every part of a standardised report in every case. Moreover, impartial experts, and prosecutors in their capacity as ‘ministers of justice’, should proactively draw direct attention to salient issues and any limitations or frailties.
Significantly, cost is not an excuse for non-compliance with legal rules and procedures.
Finally, this article illustrates just how poorly our criminal justice system is set up to take advantage of independent and authoritative insights from mainstream scientists. Our system requires expert opinions to be substantially based on specialised knowledge and prides itself on the value of adversarial engagement and the equality of arms. Yet judges (and juries) are not always provided with relevant materials and, when they are, do not necessarily appreciate their significance.[158] Criminal courts routinely and credulously admit and rely upon procedures and opinions that have not been formally evaluated. These opinions are not supported by knowledge and are not susceptible to rational evaluation.[159] There are relatively few means of conveying mainstream scientific research and perspectives to judges and triers of fact. This is why strict compliance with procedural and admissibility rules is so important for those producing forensic science and forensic medicine evidence.
As things stand, a surprisingly large proportion of the expert reports or certificates prepared by the Crown’s forensic scientists remain noncompliant with codes, practice notes and rules of admissibility. Apart from superficial declarations, few reports or certificates identify relevant ‘specialised knowledge’ or explain how the opinion is based on knowledge.[160] Consider the formulation from the Revised Certificate: ‘I have specialised knowledge based on my training, experience and study of fingerprints’.[161] Moreover, results are not presented in ways that enable readers, including technically proficient readers, to determine what was done or to rationally evaluate the conclusions. The latent fingerprint reports – old and revised – do not, for example, provide the reader with information about the number of times that a fingerprint examiner engaged in a similar kind of comparison might make a mistake. Without that information, the reader is not in a position to make sense of the ‘match’ or the leap from a putative match to positive identification. What does the opinion mean? Are we simply to assume that it is correct?[162]
This is all highly undesirable. It threatens the fairness of proceedings and brings into question the ability of adversarial procedures and legal actors to credibly engage with scientific, medical and technical forms of evidence and advice. We might wonder why, more that a century after their evidence was first admitted, and more than a decade after codes of conduct were extended to criminal proceedings, latent fingerprint examiners are only now starting to substantially revise their reporting practices.[163] Simultaneously, we might wonder about reporting practices in forensic sciences that are not erected upon rigorous scientific foundations. What about ballistics, tool marks, blood spatter, microscopic hair analysis, bite marks, voice and image comparison, the use of gait and shoe wear, shoe and tyre prints, document analysis, accident reconstruction, fire investigation and so on and so forth?
© State of New South Wales through the NSW Police Force
© State of New South Wales through the NSW Police Force
[*] Professor, School of Law, UNSW and Research Professor, Northumbria Law School, Northumbria University. This research was supported by the ARC (LP16010000).
[**] ARC DECRA Fellow and Senior Lecturer, School of Psychology, UNSW.
[***] Senior Lecturer, School of Law, UNSW.
[1] In most Australian jurisdictions the admission of expert opinion evidence is governed by s 79 of the Uniform Evidence Law (‘UEL’): see Evidence Act 1995 (Cth); Evidence Act 2011 (ACT); Evidence Act 1995 (NSW); Evidence Act 2001 (Tas); Evidence Act 2008 (Vic); Evidence Act 2004 (NI); Evidence (National Uniform Legislation) Act 2011 (NT). The common law continues in Queensland, South Australia and Western Australia. See Kristy Martire and Gary Edmond, ‘Rethinking Expert Opinion Evidence’ [2017] MelbULawRw 14; (2017) 40 Melbourne University Law Review 967; Gary Edmond and Kristy Martire, ‘Knowing Experts? Section 79, Forensic Science Evidence and the Limits of “Training, Study or Experience”’ in Andrew Roberts and Jeremy Gans (eds), Critical Perspectives on the Uniform Evidence Law (Federation Press, 2017) 80.
[2] Section 177 of the UEL permits evidence of an expert opinion to be presented via a certificate, without requiring the expert who prepared it to give evidence. The section is intended to facilitate proof via a streamlined procedure, but the opinion contained in the certificate must satisfy the ordinary admissibility requirements of s 79 of the UEL: see, eg, R v Tang [2006] NSWCCA 167; (2006) 65 NSWLR 681, 696–7 [55]–[56] (Spigelman CJ); UEL ss 177(1)(b)‑‑–(c). The party against whom the certificate is to be adduced can require the individual who signed the certificate to be called to give evidence, and once the party has stipulated this requirement the certificate is not admissible in the absence of the witness. Anecdotally, some prosecutors in NSW may have been trying to obtain consent to the adduction of certificates (rather than potentially more detailed reports) by threatening to seek costs orders. On the making of costs orders under s 177(7) of the UEL, see: Badans v The Queen [2012] NSWCCA 97; DPP (NSW) v Streeting [2013] NSWSC 789.
[3] [2015] NSWSC 1669 (‘JP v DPP’). The initial matter was unreported and was heard in the Children’s Court sitting at Dubbo before Magistrate Mijovich in January 2015. The authors had access to various materials from the trial and appeal, including a transcript of the fingerprint evidence. We would like to thank Felicity Graham for bringing this case to our attention.
[4] This article is primarily concerned with expert reports and certificates produced and relied upon by the Crown. The Crown produces the vast majority of expert reports and expert testimony, is expected to conduct proceedings as a model litigant, and carries the burden of proof. The performance of prosecutors should inform the practice of the defence, although there is scholarly debate about whether the relevant rules should be as strictly applied to expert evidence adduced by the defence; see, eg, Gary Edmond and Kent Roach, ‘A Contextual Approach to the Admissibility of the State’s Forensic Science and Medical Evidence’ (2011) 61 University of Toronto Law Journal 343, 380, 406–9.
[5] Consider the critical discussion of expert evidence in Wood v The Queen [2012] NSWCCA 21; (2012) 84 NSWLR 581 (‘Wood’) and the ensuing proceedings for malicious prosecution.
[6] See Gary Edmond, ‘What Lawyers Should Know About the Forensic “Sciences”’ [2015] AdelLawRw 3; (2015) 36 Adelaide Law Review 33.
[7] See HG v The Queen [1999] HCA 2; (1999) 197 CLR 414; Dasreef Pty Ltd v Hawchar [2011] HCA 21; (2011) 243 CLR 588 (‘Dasreef’); Honeysett v The Queen (2014) 253 CLR 122 (‘Honeysett’). The relevant exception to the exclusionary opinion rule (UEL s 76) is s 79(1) of the UEL: ‘If a person has specialised knowledge based on the person’s training, study or experience, the opinion rule does not apply to evidence of an opinion of that person that is wholly or substantially based on that knowledge’.
[8] HG v The Queen [1999] HCA 2; (1999) 197 CLR 414, 427 [39] (Gleeson CJ).
[9] This does not mean that the decision-maker will understand and rationally evaluate the evidence, only that they must be placed in a position that enables them to do so. They should not be forced to use potentially misleading heuristics and proxies, such as demeanour, confidence and experience, or popular beliefs: see Gary Edmond, ‘Forensic Science Evidence and the Conditions for Rational (Jury) Evaluation’ (2015) 39 Melbourne University Law Review 75, 92; Tony Ward, ‘Experts, Juries, and Witch-Hunts: From Fitzjames Stephen to Angela Cannings’ (2004) 31 Journal of Law and Society 369.
[10] Hillstead v The Queen [2005] WASCA 116, [49] (Pullin JA), citing Makita (Australia) Pty Ltd v Sprowles [2001] NSWCA 305; (2001) 52 NSWLR 705, 729 [59] (Heydon JA) (‘Makita’); Davie v Magistrates of Edinburgh [1953] SC 34, 39–40 (Lord President Cooper). The information required to comprehend opinions based on specialised knowledge is explored in Edmond, ‘Forensic Science Evidence and the Conditions for Rational (Jury) Evaluation’, above n 9.
[11] Makita [2001] NSWCA 305; (2001) 52 NSWLR 705, 733 [67]; see also 739–40 [79]. Heydon JA explained at 744 [85] that:
the opinion of an expert requires demonstration or examination of the scientific or other intellectual basis of the conclusions reached: that is, the expert’s evidence must explain how the field of ‘specialised knowledge’ in which the witness is expert by reason of ‘training, study or experience’, and on which the opinion is ‘wholly or substantially based’, applies to the facts assumed or observed so as to produce the opinion propounded.
[12] [2000] FCA 1463; (2000) 120 FCR 146, 151 [22]–[23] (‘Ocean Marine v Jetopay’) (emphasis removed). See also Dasreef [2011] HCA 21; (2011) 243 CLR 588, 604 [37] (French CJ, Gummow, Hayne, Crennan, Kiefel and Bell JJ); Beckett v New South Wales [2014] NSWSC 1112; Nicholls v Michael Wilson & Partners Ltd [2012] NSWCA 383, [234] (The Court); Sydneywide Distributors Pty Ltd v Red Bull Australia Pty Ltd [2002] FCAFC 157; (2002) 234 FCR 549, 576–7 [89] (Weinberg and Dowsett JJ).
[13] [2008] FCA 416, [30], [98]–[99], [125], [162]–[163], [171] (Emmett J) (‘Pan Pharmaceuticals’). See also Campbell v The Queen [2014] NSWCCA 175; (2014) 312 ALR 129.
[14] Pan Pharmaceuticals [2008] FCA 416 [125]–[126], [136], [140] (Emmett J). Although, this was considered to be a possibility in Ocean Marine v Jetopay [2000] FCA 1463; (2000) 120 FCR 146, 156 [48] (The Court).
[15] A few decisions have considered exclusion, but in most cases retrospective acknowledgement of the relevant code or practice note was sufficient to remedy the breach; see, eg, the cases discussed in Ian Freckelton and Hugh Selby, Thomson Reuters, Expert Evidence Online (at 7 March 2016) [5.5.10]; Wood [2012] NSWCCA 21; (2012) 84 NSWLR 581, 619 [728] (McClellan CJ at CL); Re Idylic Solutions Pty Ltd [2012] NSWSC 731. Even serious breaches of the code of conduct in Wood were considered unlikely, of themselves, to render the opinion evidence inadmissible. McClellan CJ at CL raised the possibility that a breach of the NSW Code might be a factor when considering exclusion under ss 135 or 137 of the UEL: Wood [2012] NSWCCA 21; (2012) 84 NSWLR 581, 620 [729].
[16] See Gary Edmond, ‘Icarus and the Evidence Act: Section 137, Probative Value and Taking Forensic Science Evidence “At Its Highest”’ (2017) forthcoming Melbourne University Law Review and the discussion of IMM v The Queen (2016) 257 CLR 300 and Tuite v The Queen [2015] VSCA 148 therein.
[17] Uniform Civil Procedure Rules 2005 (NSW) sch 7.
[18] Supreme Court of Victoria, Practice Note No 3 – Expert Evidence in Criminal Trials, 30 January 2017.
[19] Standards Australia, ‘Forensics Analysis: Part 4: Reporting’ (Standard No AS 5883.4, 2 May 2013).
[20] See UCPR rr 31.23, 31.27. Rule 75.3J of the Supreme Court Rules 1970 (NSW) applies the NSW Code to criminal proceedings including those listed in sch 3 of the Supreme Court Act 1970 (NSW).
[21] The current Federal Court Practice Note, and its associated Harmonised Code of Conduct, was issued in 2016: Federal Court of Australia, General Practice Note – Expert Evidence, 25 October 2016. The original version was published in 1998: see Australian Law Reform Commission, Managing Justice: A Review of the Federal Civil Justice System, Report No 89 (2000) 446 [6.96].
[22] See Sir Harry Woolf, Access to Justice: Final Report to the Lord Chancellor on the Civil Justice System in England and Wales (Her Majesty’s Stationery Office, 1996); National Justice Compania Naviera SA v Prudential Assurance Co Ltd [1993] 2 Lloyd’s Rep 68, 81–2 (Cresswell J) (‘The Ikarian Refeer’). See also, Federal Court of Australia, General Practice Note – Survey Evidence, 25 October 2016.
[23] UCPR r 31.18.
[24] NSW Code cl 2.
[25] NSW Code cl 2.
[26] Such expectations are consistent with professional obligations – such as membership of the Australian and New Zealand Forensic Science Society and public sector (or police) obligations pertaining to integrity and impartiality.
[27] We prefer the inclusive term ‘practitioner’ to ‘scientist’. Many of the individuals working as forensic scientists are not scientifically trained and many of their procedures and practices are not based on scientific research or knowledge: see generally Jennifer L Mnookin et al, ‘The Need for a Research Culture in the Forensic Sciences’ (2011) 58 UCLA Law Review 725.
[28] UCPR r 31.18 refers to ‘providing an expert’s report [as to his or her opinion] for use as evidence’ or ‘giving opinion evidence in proceedings or proposed proceedings’, that is to say oral evidence cannot be received by the court unless an expert witness has agreed to be bound by the NSW Code: UCPR r 31.23(4). The NSW Code is understood to apply to the form and content of UEL s 177 certificates: see Appendix I, the Certificate prepared for the trial of JP.
[29] NSW Code (emphasis added). Substantially similar language appears in UCPR r 31.27. Item (i) is a recent addition to the NSW Code, and items (j) and (k) were amended to express the obligation on the expert preparing the report in more objective terms, thereby bringing the NSW Code into line with the harmonised code of conduct introduced in the Federal Court in 2016, and closer to the Victorian Practice Note.
[30] NSW Code cl 3(d)–(e).
[31] NSW Code cl 3(g) (emphasis added). See also Melendez-Diaz v Massachusetts, 557 US 305 (2009).
[32] This is perhaps even the case with the reoriented obligations in items (j) and (k), the content of which may be moderated by the subjective nature of declaration required under (i). Though, we should not forget that these are the beliefs of an expert acting impartially.
[33] UCPR r 31.24: NSW Code cl 6(a). As for court-appointed experts the UCPR recognises their possibility: r 31.46.
[34] Supreme Court of Victoria, Practice Note No 2 – Expert Evidence in Criminal Trials, 25 June 2014; Supreme Court of Victoria, Practice Note No 3 – Expert Evidence in Criminal Trials, 30 January 2017.
[35] The Victorian Practice Note has similarities with pt 19 of the Criminal Procedure Rules 2015 (UK) SI 2015/1490. See especially r 19.2 (Expert’s duty to the court) and r 19.4 (Content of expert’s reports). See also Criminal Practice Directions 2015 [2015] EWCA Crim 1567 which introduced a more detailed list of factors that a court should take into account when assessing the reliability of expert opinion evidence: at [19A.3], [19A.5] and [19A.6].
[36] Victorian Practice Note, 4 [4.1].
[37] Victorian Practice Note, 1 [1.4]. IMM v The Queen (2016) 257 CLR 300 threatens the use of ‘reliability’ in the Victorian Practice Note. However, IMM v The Queen was concerned with probative value at trial. In England and Wales, the Lord Chief Justice embedded the need for reliability in the Criminal Procedural Rules 2015 (UK) SI 2015/1490, when the government was unwilling to introduce a statutory reliability standard as recommended by the Law Commission: see Law Commission, Expert Evidence in Criminal Proceedings in England and Wales, Law Com No 325 (2011); Sir Brian Leveson, ‘Review of Efficiency in Criminal Proceedings’ (Judiciary of England and Wales, January 2015) 60–3 [223]–[232]; Criminal Practice Directions 2015 [2015] EWCA Crim 1567, [19A.3].
[38] There is an explicit obligation to disclose the possibility (or risk) of error associated with a method or technique. See further the discussion below at Part V(C). These more specific obligations and the emphasis on reliability counterbalance the less onerous obligation, mirrored in the NSW Code, to have ‘made all the inquiries and considered all the issues which the expert believes are desirable and appropriate’: Victorian Practice Note [6.1(h)].
[39] An example might be the controversy around the ability to determine whether shaking caused particular paediatric injury: see Deborah Tuerkheimer, Flawed Convictions: ‘Shaken Baby Syndrome’ and the Inertia of Injustice (Oxford University Press, 2014); David A Moran et al, ‘Shaken Baby Syndrome, Abusive Head Trauma, and Actual Innocence: Getting It Right’ (2012) 12 Houston Journal of Health Law & Policy 209.
[40] Victorian Practice Note 2 [3.1], [5.1]–[5.2]. Thus it applies to expert certificates prepared under the auspices of s 177 of the Evidence Act 2008 (Vic) in the expectation that they will be relied on in court either directly or indirectly if the person who prepared the certificate is required to testify. One drawback with the current version of the Victorian Practice Note is that it appears to contemplate the use of a preliminary form of report (‘Forensic Reports’), prepared by a forensic practitioner, but not intended to be relied on in court and not necessarily subject to the same stringent expectations as the more detailed report (the ‘Primary Report’). So-called Forensic Reports appear to be widely used in pleas and proceedings, including cases where the forensic science evidence is in issue.
[41] See Bryan Found and Gary Edmond, ‘Reporting on the Comparison and Interpretation of Pattern Evidence: Recommendations for Forensic Specialists’ (2012) 44 Australian Journal of Forensic Sciences 193.
[42] Standards Australia, ‘Forensics Analysis: Part 4: Reporting’ (Standard No AS 5883.4, 2 May 2013).
[43] Gary Edmond is a member of the committee responsible for the Australian Standards for the forensic sciences: Committee CH-041. The Forensic Reporting Standard was primarily drafted by industry insiders and do not necessarily embody the authors’ perspectives.
[44] Forensic Reporting Standard cl 5.
[45] Forensic Reporting Standard cl 9.4.
[46] Forensic Reporting Standard cl 6.3. On the different types of review and their value and limitations, see Kaye Ballantyne et al, ‘Peer Review and the Forensic Sciences’ (2017) forthcoming Forensic Science International.
[47] Forensic Reporting Standard cl 6.2.
[48] Cf Sir Anthony Campbell’s report published following the Fingerprint Inquiry on the need for disclosure: ‘The Fingerprint Inquiry Report’ (Report, December 2011) 669–671 [37.75]–[37.84] (‘Fingerprint Inquiry Report’). See also R v T [2003] EWCA Crim 1977; [2011] 1 Cr App R 9; Lord Chief Justice John Thomas, ‘Expert Evidence: The Future of Forensic Science in Criminal Trials’ (Speech delivered at the Kalisher Lecture, Criminal Bar Association, 14 October 2014) <https://www.judiciary.gov.uk/wp-content/uploads/2014/
10/kalisher-lecture-expert-evidence-oct-14.pdf>.
[49] Individualisation, like positive identification, involves the identification of a specific person. See Jonathan J Koehler and Michael J Saks, ‘Individualization Claims in Forensic Science: Still Unwarranted’ (2010) 75 Brooklyn Law Review 1187; Simon A Cole, ‘Individualization is Dead, Long Live Individualization! Reforms of Reporting Practices for Fingerprint Analysis in the United States’ (2014) 13 Law, Probability and Risk 117.
[50] NSW Police fingerprint examiners typically produce expert certificates.
[51] Parker v The King [1912] HCA 29; (1912) 14 CLR 681; JP v DPP [2015] NSWSC 1669. See also Jeremy Gans, ‘A Tale of Two High Court Forensic Cases’ [2011] SydLawRw 22; (2011) 33 Sydney Law Review 515.
[52] JP v DPP [2015] NSWSC 1669, [8]–[10] (Beech-Jones J).
[53] It is not our intention to single out the fingerprint examiner who appeared in JP v DPP. There is no reason to believe that he is anything other than a typical NSW latent fingerprint examiner. He is certainly not a rogue or bad examiner.
[54] JP v DPP [2015] NSWSC 1669, [23]–[24] (Beech-Jones J).
[55] In JP v DPP [2015] NSWSC 1669, Beech-Jones J found that: the expert certificate (and evidence) was not capable of amounting to an admissible opinion (Ground 1) and that the magistrate had erred in admitting the evidence (Ground 6): at [50]–[63]. Among other matters, the transcript reveals that the cross-examination sought to introduce a range of secondary materials including police training materials, the Fingerprint Inquiry Report and the NAS Report (discussed below in Part V), to challenge the fingerprint examiner’s approach and levels of certainty in the expression of his opinion. This material was rejected by the magistrate. Ground 7 sought to challenge the magistrate’s rejection of the relevant secondary materials, but this was not pressed on appeal: at [92].
[56] See, eg, Bennett v Police [2005] SASC 167; Bennett v Police [2005] SASC 415; Hillstead v The Queen [2005] WASCA 116; R v SMR [2002] NSWCCA 258. Australian appellate decisions demonstrate very limited engagement with criticisms and methodological limitations: cf R v Smith [2009] EWCA Crim 905; [2011] 2 Cr App R 16, [61]–[62] (Thomas LJ).
[57] JP v DPP [2015] NSWSC 1669, [54].
[58] Ibid [6] (emphasis added).
[59] Ibid [56]–[57] (Beech-Jones J).
[60] Ibid [23] (Beech-Jones J). The fingerprint examiner’s denials, confidence and ignorance are used to overcome authoritative advice that does not make it into the various written decisions.
[61] Fingerprint Operations, NSW Police have, to their credit, begun to consult with scientific researchers and attentive scholarly commentators in recent years. Gary Edmond was invited to provide verbal comments on a draft shown, but not provided, to him in a cafe near police headquarters at Parramatta. The Revised Certificate does not, however, adequately address issues raised by him and discussed in this article.
[62] The challenge at trial and the issues raised on appeal made it abundantly clear that their practices were not compliant with jurisdictional rules of evidence and procedure. Gary Edmond has discussed these issues at national conferences hosted by the Fingerprint Operations, NSW Police in 2013 and 2015.
[63] See JP v DPP [2015] NSWSC 1669 [33], [43] (Beech-Jones J).
[64] Accommodating legal responses tend to discourage scientific research and formal evaluation, as forensic practitioners look to the courts for legitimacy: see President’s Council of Advisors on Science and Technology, ‘Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods’ (Report, 20 September 2016) (‘PCAST Report’) 21.
[65] For a general discussion of issues associated with the communication of evidence strength, see Kristy A Martire, Richard I Kemp and Ben R Newell, ‘The Psychology of Interpreting Expert Evaluative Opinions’ (2013) 45 Australian Journal of Forensic Sciences 305.
[66] Revised Certificate, page 7 (citations omitted) (see Appendix II of this article).
[67] National Research Council, ‘Strengthening Forensic Science in the United States: A Path Forward’ (Report, National Academy of Sciences, August 2009) (‘NAS Report’) 142–3 (citations omitted). See also Expert Working Group on Human Factors in Latent Print Analysis, ‘Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach’ (Report, National Institute of Standards and Technology, Department of Commerce (US), February 2012) 8–9 (‘NIST Report’); PCAST Report, above n 64, 67–81; Lyn Haber and Ralph Norman Haber, ‘Scientific Validation of Fingerprint Evidence under Daubert’ (2008) 7 Law, Probability and Risk 87.
[68] NAS Report, above n 67, 7, 100. Lest readers think these are historical and/or isolated problems, in 2016 PCAST produced an even more critical assessment, noting that ‘work to date has not addressed the [NAS Report’s] call to examine the fundamental scientific validity and reliability of many forensic methods used every day in courts’: see PCAST Report, above n 64, 39.
[69] Office of the Inspector General, ‘A Review of the FBI’s Handling of the Brandon Mayfield Case’ (Report, Department of Justice (US), March 2006).
[70] See, eg, Jason M Tangen, Matthew B Thompson and Duncan J McCarthy, ‘Identifying Fingerprint Expertise’ (2011) 22 Psychological Science 995.
[71] In controlled conditions, where correct answers (or ‘ground truth’) are available to those testing performance, experienced fingerprint examiners made small numbers of mistakes. They were more likely to fail to match fingerprints known to be made by the same person (ie, to ‘misclassify’ matching prints as non-matching) than to match similar fingerprints made by different persons (ie, give a ‘false alarm’), though they made both kinds of errors by a range of examiners. Fingerprint examiners made dramatically less errors than novices, especially when fingerprints were similar but from different sources: ibid 997.
[72] See, eg, Bradford T Ulery et al, ‘Accuracy and Reliability of Forensic Latent Fingerprint Decisions’ (2011) 108 Proceedings of the National Academy of Sciences of the United States of America 7733; Tangen, Thompson and McCarthy, above n 70; Igor Pacheco, Brian Cerchiai and Stephanie Stoiloff, ‘Miami-Dade Research Study for the Reliability of the ACE-V Process: Accuracy & Precision in Latent Fingerprint Examinations’ (Final Technical Report, Miami-Dade Police Department Forensic Services Bureau, December 2014).
[73] PCAST Report, above n 64, 98. Full references and descriptions of all the studies are provided in the report. Note that these studies only started after the NAS inquiry was under way and that the ‘early studies’ were published in 2009. We appreciate that there have been critical responses to PCAST; see, eg, Friction Ridge Subcommittee, ‘Response to Call for Additional References Regarding: President’s Council of Advisors on Science and Technology Report to the President’ (Response, Organization of Scientific Area Committees, 14 December 2016). However, these responses were dismissed by PCAST in the addendum to its Report: ‘An Addendum to the PCAST Report on Forensic Science in Criminal Courts’ (Addendum, 6 January 2017).
[74] PCAST Report, above n 64, 101–2.
[75] Ibid 102: ‘Additional black-box studies are needed to clarify the reliability of the method’.
[76] This does not prevent the presentation of fingerprint comparison in the Revised Certificate as the ‘Science of Fingerprints’: Revised Certificate, page 4 (see Appendix II of this article). For Simon Cole, ‘[r]esearchers [sh]ould have the last word on whether a method or technique is valid. Technicians would no longer be put in the awkward position of having to defend the validity of techniques they apply’: Simon A Cole, ‘Acculturating Forensic Science: What Is “Scientific Culture”, and How Can Forensic Science Adopt It?’ (2010) 38 Fordham Urban Law Journal 435, 468.
[77] They might answer, but answers could be be naïve and misleading.
[78] NSW Code cl 3(f).
[79] Forensic Reporting Standard cl 9.5.
[80] JP v DPP [2015] NSWSC 1669, [23] (Beech-Jones J).
[81] Ibid.
[82] The PCAST Report was released after the draft of the Revised Certificate, but the PCAST Report merely summarises existing research and endorses earlier expressions of concern. See above n 73 and accompanying text.
[83] Revised Certificate, pages 6–7 (see Appendix II of this article).
[84] Ibid, see also R v Pakula 2017 ABPC 33, [56]–[64].
[85] The phrase translates as ‘he himself said it’ and it refers to assertion or bare declaration.
[86] PCAST Report, above n 64, 5–6, 78. See also ‘As a matter of scientific validity, examiners must be required to “complete and document their analysis of a latent fingerprint before looking at any known fingerprint”’: at 100.
[87] See Part V(F).
[88] Revised Certificate, page 2 [9] (see Appendix II of this article). Note that there is one other reference to ‘opinion’ in Annexure 3 to the Revised Certificate.
[89] See Bradford T Ulery et al, ‘Changes in Latent Fingerprint Examiners’ Markup between Analysis and Comparison’ (2015) 247 Forensic Science International 54; Bradford T Ulery et al, ‘Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners’ (2012) 7(3) PLoS One e32800; Itiel E Dror et al, ‘Cognitive Issues in Fingerprint Analysis: Inter-and Intra-expert Consistency and the Effect of a ‘Target’ Comparison’ (2011) 208 Forensic Science International 10.
[90] C Neumann, I W Evett and J Skerrett, ‘Quantifying the Weight of Evidence from a Forensic Fingerprint Comparison: A New Paradigm’ (2012) 175 Journal of the Royal Statistical Society 371.
[91] See Fingerprint Inquiry Report, above n 48, 741 (recommendation 5), 742 (recommendation 17). See also NIST Report, above n 67, 42–3 (recommendation 3.1), 54–5 (recommendation 3.4), 94 (recommendation 5.1), 100 (recommendation 5.2), 127 (recommendation 6.3).
[92] Revised Certificate, page 7 (see Appendix II of this article).
[93] NAS Report, above n 67, 184 (emphasis added). See also at 122.
[94] PCAST Report, above n 64, 96. See also at 26, 74.
[95] Ibid 101.
[96] Simon A Cole, ‘More than Zero: Accounting for Error in Latent Fingerprint Identification’ (2005) 95 Journal of Criminal Law & Criminology 985; John R Vokey, Jason M Tangen and Simon A Cole, ‘On the Preliminary Psychophysics of Fingerprint Identification’ (2009) 62 Quarterly Journal of Experimental Psychology 1023.
[97] Revised Certificate, page 7 (see Appendix II of this article).
[98] See NIST Report, above n 67, ch 2.
[99] NAS Report, above n 67, 142. This was written before studies were conducted by groups led by Ulery and Tangen.
[100] Tangen, Thompson and McCarthy, above n 70, 997 (emphasis added).
[101] Revised Certificate, page 7 (see Appendix II of this article), citing Tangen, Thompson and McCarthy, above n 70.
[102] Cole, ‘More than Zero’, above n 96.
[103] See, eg, the measures identified in the PCAST Report, above n 64, 121, 143. The methods and standards are insufficiently detailed and not capable of supporting positive identification in most cases.
[104] We should not rely on the history of convictions because we do not know how many were mistaken or based on other evidence.
[105] Cf David L Faigman, John Monahan and Christopher Slobogin, ‘Group to Individual (G2i) Inference in Scientific Expert Testimony’ (2014) 81 University of Chicago Law Review 417.
[106] Fingerprint Inquiry Report, above n 48, 741 (recommendation 1); PCAST Report, above n 64, 35, 124. PCAST concluded that the error rate for latent fingerprint evidence is ‘substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis’: at 101.
[107] The conclusion is expressed to be an opinion, but that is diminished by reporting the categorical identification in a table in the Revised Certificate, page 2 (see Appendix II of this article).
[108] NIST Report, above n 67, 72. See also at 48, 63, 197; NAS Report, above n 67, 87, 104, 143; PCAST Report, above n 64, 59–60. See also Cole, ‘Individualization is Dead’, above n 49.
[109] It may be that where multiple prints are matched the evidence is much more compelling.
[110] NIST Report, above n 67, 72. Dropping the words ‘to the exclusion of all others’ does not resolve these issues. In this context, ‘individualisation’, ‘positive identification’ and ‘positive identification to the exclusion of all others’ are not logically distinguishable.
[111] See Fingerprint Inquiry Report, above n 48, 741 (recommendation 3).
[112] See also NAS Report, above n 67, 144; See generally PCAST Report, above n 64, 42–3.
[113] JP v DPP [2015] NSWSC 1669, [18]–[19], [21] (Beech-Jones J).
[114] Ibid [23] (Beech-Jones J).
[115] National Commission on Forensic Science, ‘Ensuring That Forensic Analysis Is Based upon Task-Relevant Information’ (Views Document, National Institute of Standards and Technology, Department of Commerce (US), 8 December 2015) 4 (citations omitted), citing Saul M Kassin, Itiel E Dror and Jeff Kukucka, ‘The Forensic Confirmation Bias: Problems, Perspectives, and Proposed Solutions’ (2013) 2 Journal of Applied Research in Memory and Cognition 42; NIST Report, above n 67; William C Thompson, ‘What Role Should Investigative Facts Play in the Evaluation of Scientific Evidence?’ (2011) 43 Australian Journal of Forensic Science 123. See also, D Michael Risinger et al, ‘The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expecta‑tion and Suggestion’ (2002) 90 California Law Review 1; Reinoud D Stoel et al, ‘Minimizing Contextual Bias in Forensic Casework’ in Kevin J Strom and Matthew J Hickman (eds) Forensic Science and the Administration of Justice (Sage Publications, 2015) 67.
[116] Bryan Found, ‘Deciphering the Human Condition: The Rise of Cognitive Forensics’ (2015) 47 Australian Journal of Forensic Sciences 386, 390.
[117] National Commission on Forensic Science, above n 115, 2. The Commission defined task-irrelevant information: ‘The test of whether such ancillary contextual information is relevant to a forensic assessment is whether it helps the examiner draw an accurate forensic conclusion from the physical evidence designated for testing using accepted methods’: at 2.
[118] See Itiel E Dror, David Charlton and Ailsa E Péron, ‘Contextual Information Renders Experts Vulnerable to Making Erroneous Identifications’ (2006) 156 Forensic Science International 74; Itiel E Dror and Greg Hampikian, ‘Subjectivity and Bias in Forensic DNA Mixture Interpretation’ (2011) 51 Science & Justice 204.
[119] NAS Report, above n 67, 191 (recommendation 5).
[120] Fingerprint Inquiry Report, above n 48, 741 (recommendations 6–8). But see also 742 (recommendation 18), 743 (recommendations 21–22, 24), 744 (recommendations 29–32), 746 (recommendation 42).
[121] See NIST Report, above n 67, 44 (recommendation 3.3), 143 (recommendation 7.1).
[122] National Commission on Forensic Science, above n 115, 1. See also PCAST Report, above n 64, 10, 12.
[123] National Commission on Forensic Science, above n 115, 5. See also, Office of the Inspector General, ‘A Review of the FBI’s Progress in Responding to the Recommendations in the Office of the Inspector General Report on the Fingerprint Misidentification in the Brandon Mayfield Case’ (Report, Department of Justice (US), June 2011) 5, 27.
[124] This approach was also proposed in the Fingerprint Inquiry Report, above n 48, 743 (recommendation 26). See the description in Dan E Krane et al, ‘Sequential Unmasking: A Means of Minimizing Observer Effects in Forensic DNA Interpretation’ (2008) 53 Journal of Forensic Sciences 1006, 1006.
[125] According to Sir Anthony Campbell, features subsequently observed should be accorded less weight: Fingerprint Inquiry Report, above n 48, 743 (recommendation 26).
[126] JP v DPP [2015] NSWSC 1669, [23] (Beech-Jones J).
[127] Woolmington v DPP [1935] AC 462, applied in Griffiths v The Queen [1994] HCA 55; (1994) 125 ALR 545, 548–9 (Brennan, Dawson and Gaudron JJ); see also UEL s 141(1).
[128] Revised Certificate, page 7 (see Appendix II of this article).
[129] PCAST Report, above n 64, 74–75, 89–90, 96–97. The PCAST Report concluded, in relation to ‘blind’ review at 96:
It is likely that a properly designed program of systematic, blind verification would decrease the false-positive rate, because examiners in the studies tend to make different mistakes. However, there has not been empirical testing to obtain a quantitative estimate of the false positive rate that might be achieved through such a program. And, it would not be appropriate simply to infer the impact of independent verification based on the theoretical assumption that examiners’ errors are uncorrelated. (emphasis in original) (citations omitted).
[130] See Christopher T Robertson and Aaron S Kesselheim (eds), Blinding as a Solution to Bias: Strengthening Biomedical Science, Forensic Science, and Law (Academic Press, 2016).
[131] If fingerprint examiners did not make mistakes, there would be no need for verification.
[132] We accept that in many cases the outcome of fingerprint comparisons might be relatively uncontroversial. The significance of a match is, however, more complicated.
[133] The Standard ‘is applicable to all laboratories regardless of the number of personnel or the extent of the scope of testing and/or calibration activities: Committee on Conformity Assessment, International Organization for Standardization, ‘General Requirements for the Competence of Testing and Calibration Laboratories’ (International Standard No ISO/IEC 17025, International Organization for Standardization and International Electrotechnical Commission, 15 May 2005) 1 [1.2].
[134] Consider the National Association of Testing Authorities (‘NATA’), ‘ISO/IEC 17025 Standard Application Document for Accreditation of Testing and Calibration Facilities’ (Standard Application Document, March 2015) 9 [5.10].
[135] NATA (and the National Institute for Forensic Sciences) did not identify the serious deficiencies in procedures and reporting reported by independent committees such as the NAS, NIST and PCAST.
[136] PCAST Report, above n 64, 66; see also at 147:
Foundational validity is a sine qua non, which can only be shown through empirical studies. Importantly, good professional practices – such as the existence of professional societies, certification programs, accreditation programs, peer-reviewed articles, standardized protocols, proficiency testing, and codes of ethics – cannot substitute for empirical evidence of scientific validity and reliability.
See generally Daniel Carpenter and David A Moss (eds), Preventing Regulatory Capture: Special Interest Influence and How to Limit It (Cambridge University Press, 2014).
[137] PCAST Report, above n 64, 38. PCAST was disparaging of proficiency testing used by forensic scientists: ‘To ensure integrity, proficiency testing should be overseen by a disinterested third party that has no institutional or financial incentive to skew performance. We note that testing services have stated that forensic community prefers that tests not be too challenging’: at 57.
[138] See National Association of Testing Authorities, above n 134, 9 [5.9.1]: ‘[p]roficiency testing (PT): Each applicant or accredited facility is required to participate in appropriate PT activities’.
[139] Cole, ‘More than Zero’, above n 96.
[140] See, eg, Simon A Cole, Suspect Identities: A History of Fingerprinting and Criminal Identification (Harvard University Press, 2001); Haber and Haber, above n 67; Jennifer L Mnookin, ‘The Validity of Latent Fingerprint Identification: Confessions of a Fingerprinting Moderate’ (2008) 7 Law, Probability and Risk 127; Gary Edmond, Matt Thompson and Jason Tangen, ‘A Guide to Interpreting Forensic Testimony: Scientific Approaches to Fingerprint Evidence’ (2014) 13 Law, Probability & Risk 1.
[141] See the discussion of the Canadian fingerprint case R v Bornyk [2014] BCCA 450 in Gary Edmond, David Hamer and Emma Cunliffe, ‘A Little Ignorance Is a Dangerous Thing: Engaging with Exogenous Knowledge Not Adduced by the Parties’ (2016) 25 Griffith Law Review 383.
[142] JP v DPP [2015] NSWSC 1669, [90] (Beech-Jones J). According to the PCAST Report, forensic science evidence is not susceptible to rational evaluation without information about validity and error rates: above n 64, 46.
[143] The unreported remarks of Magistrate Mijovich were quoted in JP v DPP [2015] NSWSC 1669, [23] (Beech-Jones J) (emphasis altered).
[144] Ibid [90] (Beech-Jones J).
[145] The reluctance to make concessions makes expert opinion evidence less convincing, see the example in IMM v The Queen that might be applied to opinions based on specialised knowledge: (2016) 257 CLR 300, 314–15 [50] (French CJ, Kiefel, Bell and Keane JJ).
[146] The major difference between the trial and appeal is the ability to ask questions of the witness. Confidence and demeanour do not address validity and scientific reliability. See the discussion of criteria for evaluating expert opinion evidence: Gary Edmond, ‘Legal versus Non-legal Approaches to Forensic Science Evidence’ (2016) 20 International Journal of Evidence & Proof 3; Kristy Martire and Gary Edmond, ‘Rethinking Expert Opinion Evidence’ [2017] MelbULawRw 14; (2017) 40 Melbourne University Law Review 967.
[147] PCAST Report, above n 64, 55: ‘a forensic examiner’s “experience” from extensive casework is not informative’ and ‘expression[s] of confidence based on personal professional experience or expressions of consensus among practitioners about the accuracy of their field is no substitute for error rates estimated from relevant studies. For a method to be reliable, empirical evidence of validity ... is required’ (emphasis in original).
[148] For example, the examiner suggested that the ACE-V method removed errors. This is certainly inconsistent with the only independent scientific advice on verification. See discussion in Part V(F) of this article.
[149] And, the reluctance of courts to consider, or their lack of exposure to, emerging evidence that questions validity and reliability, or conventional forms of expressing opinions, rather than defer to prior admissibility decisions, some dating back more than a century such as Parker v The King [1912] HCA 29; (1912) 14 CLR 681.
[150] PCAST Report, above n 64, 97. Enforcing admissibility standards and excluding ‘unreliable methods have historically helped propel major improvements in forensic science – as happened in the early days of DNA evidence’: at 122–3.
[151] PCAST Report, above n 64, 66, 147.
[152] Makita [2001] NSWCA 305; (2001) 52 NSWLR 705, 733 [67] (Heydon JA).
[153] See Gary Edmond et al, ‘Model Forensic Science’ (2016) 48 Australian Journal of Forensic Sciences 496.
[154] The most vigorous challenges to forensic science evidence have been around DNA profiling evidence. This might be considered ironic, given that DNA evidence has its origins in mainstream scientific research and is ordinarily presented in probabilistic terms.
[155] Inadvertently, this article casts an indirect and unwelcome light on ad hoc experts. ‘Ad hoc experts’ are those who have apparently acquired some familiarity with a voice or the appearance of a person of interest, through exposure to a large volume of surveillance materials (usually audio) or repeated though often quite limited exposure to a person or images of a person. Based on this exposure, an ad hoc expert may be permitted to give evidence positively identifying the defendant. Ad hoc expertise does not produce opinions based on specialised knowledge. Most ad hoc experts are not in a position to produce reports that comply with codes and practice notes. Ad hoc expertise has been said to fall within the terms of s 79, to have survived alongside the UEL (notwithstanding s 76), or under s 78 (which is emerging as the preferred approach in Victoria): see Morgan v The Queen [2016] NSWCCA 25; Kheir v The Queen [2014] VSCA 200; (2014) 43 VR 308; Tran v The Queen [2016] VSCA 79; Nguyen v The Queen [2017] NSWCCA 4. For rare counter-examples to the permissive trend: see R v Hall [2001] NSWSC 827, R v Sterling (2014) 19 DCLR (NSW) 74; R v Nguon (2014) 22 DCLR (NSW) 302; Smith v The Queen [2001] HCA 50; (2001) 206 CLR 650. See also Gary Edmond and Mehera San Roque, ‘Quasi-Justice: Ad Hoc Experts and Identification Evidence’ (2009) 33 Criminal Law Journal 8.
[156] See F H R Vincent, ‘Inquiry into the Circumstances that Led to the Conviction of Mr Farah Abdulkadir Jama’ (Report, Victorian Government, 29 March 2010).
[157] Some of the additional materials could be included in an appendix or made available online.
[158] A rare exception is Tuite v The Queen [2015] VSCA 148 where an Australian appellate court endeavoured to engage with mainstream scientific literature and advice.
[159] PCAST Report, above n 64, 29: ‘several forensic feature-comparison methods that have been in wide use have nonetheless not been subjected to meaningful tests of scientific validity or measures of reliability’. Unlike England and Wales, Australia has no independent forensic science regulator and, unlike the US, has no federal commission leading the reform of the forensic sciences such as the National Commission on Forensic Science. The Australian National Institute of Forensic Sciences, notwithstanding the impressive name, is funded primarily by police, has only a few employees and has vacillated in the face of a decade of critical revelations.
[160] Judgments tend to rely heavily on training and long experience.
[161] Revised Certificate, page 1 (see Appendix II of this article).
[162] Edmond, ‘Icarus and the Evidence Act’, above n 16.
[163] Interestingly, these revisions are not in response to admissibility rules and codes of conduct, but defence challenges and remarkably soft judicial censure.
AustLII:
Copyright Policy
|
Disclaimers
|
Privacy Policy
|
Feedback
URL: http://www.austlii.edu.au/au/journals/UNSWLawJl/2017/22.html