Bast, CM; Samuels, L B --- "Legal studies scholarship: setting the standard for excellence" [2010] LegEdDig 12; (2010) 18(1) Legal Education Digest 41

• Legal studies scholarship: setting the standard for excellence

---

Legal studies scholarship: setting the standard for excellence

C M Bast & L B Samuels

Journal of Legal Studies Education, Vol 26 No. 2, 2009, pp263-323

This article describes who legal studies professors are, explores what scholarship means in higher education, and describes the several publication types available for legal studies scholarship, including peer-review journals and law reviews.

A considerable number of attorneys teach law-related courses at post-secondary institutions outside law schools. Many of these ‘legal studies’ professors are housed within the business school of the university; however, a significant number are housed in colleges or schools within the university other than business schools.

Many legal studies professors, especially those who teach in business schools, are affiliated with the Academy of Legal Studies in Business (ALSB). ALSB, founded in 1924, is an international academic organisation, whose members teach undergraduate and graduate law-related courses outside of law schools in the United States, Canada, Australia, and the United Kingdom.

This diversity of the home institutions of ALSB members is reflected also in their classification by the Carnegie Foundation for the Advancement of Teaching. The Carnegie classifications are designed to highlight similarities and differences among educational institutions, providing a research-teaching scale for institutions of higher education.

The percentages of ALSB members located within the various Carnegie classifications are likely to reflect the importance of scholarship at their home institutions. In 2005, more than three-quarters of ALSB members worked at comprehensive doctoral, doctoral, and master institutions. Scholarship is generally recognised to be the discovery of knowledge and its dissemination through publication, which is generally viewed as a positive for society. Publication immortalises an author’s ideas and allows the author to claim credit for ideas and applications. The discovery of knowledge enriches or creates a dialogue in the scholarly community when it is communicated to other scholars. Many academic institutions, especially those at the comprehensive doctoral level, require scholarship as part of the professor’s job and base salary, promotion, and tenure decisions, at least in part, on this component of performance.

Many academic institutions require their professors to attain ‘excellence in scholarship,’ though the determination of whether a professor’s body of work meets this standard can be as difficult as the determination of whether a particular artwork can be considered fine art. Methods for evaluation often include inquiring whether the scholarship ‘is recognised and respected by peers’, ‘makes a substantial, high-quality contribution to the field’, ‘is original and innovative’, ‘is widely disseminated’, ‘is part of an ongoing program of study rather than an isolated project’, ‘is communicated effectively’ and ‘brings credit to the university’. A review of these various standards, makes apparent that evaluation of scholarship requires subjective judgement by the evaluator. Also, the process is faulty and a violation of academic freedom if the evaluator is biased because of the professor’s unpopular views or unorthodox presentation.

Academic freedom, a concept that includes fairness in the evaluation of a professor’s scholarship, has been championed for over 90 years by the American Association of University Professors (AAUP).

Because scholarship is central to academic freedom, the AAUP policies contain broad principles regarding scholarship in academe. These principles encourage a scholarly dialogue between professors without restriction on subject matter or presentation format. The fairness and due process principles of the AAUP policies require that the institution adequately disclose to the professor the institution’s expectation as to scholarship in writing, that this expectation not unduly burden the professor, that the evaluation includes review by the professor’s peers, and that the evaluation be discipline specific.

For AACSB accreditation purposes, intellectual contributions include the broad categories of ‘contributions to learning and pedagogical research, contributions to practice, and discipline-based scholarship’. The first category includes ‘preparation of new materials for use in courses, creation of teaching aids, and research on pedagogy’; the second category includes ‘articles in practice-oriented journals, creation and delivery of executive education courses, development of discipline-based practice tools, and published reports on consulting’; and the third category ‘adds to the theory or knowledge base of the faculty member’s field’ and includes ‘published research results and theoretical innovation’. The inclusion of pedagogical research as one of the three broad categories of faculty intellectual contributions reflects the growth of the scholarship of teaching and learning in higher education.

For both tenure and promotion, many institutions emphasise either scholarship or teaching, requiring excellence in one (generally research) and high competence in the others.

The many disciplines comprising the educational institution differ greatly, both in subject matter and in what they value as scholarship. AAUP policy on tenure and promotion emphasises the important and ‘primary’ role faculty play in evaluating their fellow professors and recognises that evaluation of scholarship should be discipline specific.

A general requirement of scholarship is that the professor’s knowledge be communicated through publication.

Through much of academe, the quintessential publication vehicle for scholarship is the peer-reviewed academic journal article. Although the concept of peer review of manuscripts submitted for publication dates from the 18th century, its utilisation to select those worthy of publication by a particular journal is about 50 years old. Nevertheless, peer review is the commonly accepted standard in social science and science journals, and in those fields, articles published without vaulting the peer-review hurdle are suspect and may not count as scholarship at the author’s educational institution for purposes of promotion, tenure, renewal, and salary increases. Though the peer-review process is organised to provide controls in the determination of the quality of submissions, the process can be beneficial and fair but also detrimental and unfair.

Peer reviews for journal article selection are completed by scholars within the journal’s discipline, from the perspective of the journal’s mission. The primary goal of the process is identification of those submissions that meet the journal’s standards and therefore should be accepted for publication. Peer review takes time and involves multiple levels of review; because of this, most journals involved in the peer-review process will only review articles that are unique submissions. A good quality manuscript may be rejected by a journal because, among many other reasons, the manuscript’s substance does not fall within the mission of the journal, the manuscript would not be relevant to the journal’s readers, another article has been published or accepted on the topic, the topic does not fall within a symposium issue, or the journal staff is overloaded with submissions. Oftentimes, the reviewers of peer-reviewed journals conclude that the submitted article is neither a clear accept or reject, but rather should be revised and resubmitted in accordance with instructions communicated in the review. Occasionally submissions are lost in the process and never receive a ‘fair’ reading. There is definitely a human element involved that relies on the integrity of the reviewers and the management skills of the editors.

Generally, the editors and reviewers who serve on academic journals are volunteers, providing service to the profession, which will help shape the journal and, therefore, advance the discipline, while, at the same time, raising their visibility in the discipline and contributing to their external service record. Editors may be selected because of their quality work performed as reviewers and staff editors, together with their perceived ability to complete the often arduous tasks required of an editor.

Peer review can be open or closed, the distinction being that in open peer review the identities of the author and reviewers are known, whereas closed peer review involves anonymity. Closed peer review is generally classified as either single blind peer review or double blind peer review.

There are advantages and disadvantages to both formats. The perceived advantage to closed peer review is that the reviewers are less biased by the identity of the author and the author’s institution and can be fairer and more honest in the review, knowing that the review comments cannot be associated by the author with any particular reviewer.

Perceived advantages to open peer review are that the quality of the review can be better if the reviewer knows the background of the author, the fact that the reviewer’s identity is known will operate to make the reviewer more wary of being biased, the reviewer cannot hide behind the fact that the reviewer’s identity is concealed, and the reviewer will be able to more easily determine whether the manuscript is duplicative of any of the author’s previously published articles or whether the author has self-plagiarised.

A reviewer whose identity is disclosed may be more willing to offer guidance to improve the manuscript.

Peer reviewers may not be qualified to judge whether a manuscript should be published. For example, they may not be knowledgeable about the literature within the discipline and, therefore, may not be qualified to judge whether a manuscript duplicates material adequately covered in previously published articles. A peer reviewer may be reluctant to recommend a manuscript that takes an unorthodox approach. Further, peer reviewers may overlook mistakes, not offer very much guidance, or offer the wrong kind of guidance, sometimes because of lack of training, overwork, or lack of experience. They may not check all of the authorities cited, relying on the accuracy of the author. Unless intimately familiar with the literature, they may not detect even blatant plagiarism. Different reviewers may have wildly disparate reactions to the same manuscript.

It is very difficult to have a level playing field. An author with an impressive publication record may produce higher quality manuscripts. However, an author with an impressive publication record may have had an acceptance from a prestigious journal early on in the author’s career, which benefitted the author even when submitting lesser quality manuscripts.

The reality is that even with a double-blind review, the review is partially open because the editor, who makes the final publication decision, usually knows the identity of the author and the author’s institution and the identity of the reviewers completing the reviews.

The editor-in-chief may select manuscripts to be published for reasons other than the manuscript quality. The fairness of the process may be called into question if the editor-in-chief exercised favouritism in the selection process, in tweaking the overall recommendation by selecting particular reviewers, in disregarding the reviewers’ conclusions, or by selecting one manuscript over another other for reasons other than quality. The editor-in-chief may choose to publish one manuscript over another because of the identity of the author, the author’s standing within the organisation producing the journal, or the prestige of the author’s institution. Also, an editor may be influenced by the quantity of manuscripts submitted, being more selective in recommending publication when there is a greater quantity of manuscripts to choose from and being less selective when the number of submissions is fewer.

The law review is the traditional form in which legal scholarship is communicated to other legal scholars, judges, attorneys, and legislators. Harvard Law Review was the first law review published, and it began the tradition of law school students selecting, editing, and publishing scholarly legal articles. Within the legal community, the law review became the quintessential publication vehicle for scholarship.

Learned discourse in law has shifted dramatically in the last few decades from treatises to the law school-sponsored law reviews. It is in this medium that legal scholars now seek to be heard, and it is in this medium where the most influential work is to be found.

The scholarship published in law reviews is far ranging, seeking to ‘understand, describe, evaluate, and ultimately improve complex legal principles and processes’, and is multidimensional. The law review article generally provides an in-depth study of a legal issue, encapsulating the results of the author’s research on the legal issue and usually offering an opinion on the legal standards to be followed or recommending a course of action concerning the issue studied.

The research necessitated by such an in-depth study is usually extensive and time consuming, as is the time to write and revise the manuscript. As in other disciplines, the author may present the manuscript at a conference and/or post a working paper online, requesting comments prior to submitting the manuscript for publication.

In recent years, a professor wishing to submit a manuscript for publication may consult the Washington and Lee University Law School website on law journal submissions and ranking information about journals. As its name implies, the website page contains links to most (English language and a few foreign language) law journals and provides information on journal ranking, in general and for specific topic areas.

Although law reviews allow simultaneous submission to multiple law reviews, a law review can be very discerning in the manuscripts it selects for publication, given the vast quantity of manuscripts submitted. It is estimated that the top law reviews annually receive between 600 to 1,400 manuscripts, with the Harvard Law Review receiving up to 2,000. The acceptance rate for the most prestigious journals may be as low as 0.2 percent, given that the number of articles published annually by an individual law review is between four and twenty. It has become somewhat common in the last few years for an author who has received a publication offer from a law review to attempt to obtain a publication offer from a more prestigious law review by asking the more prestigious law review for an expedited review. In fact, there is some belief that many law reviews from the top fifty law schools typically refuse to consider a manuscript until the author has an offer in hand and requests an expedited review.

In addition, studies published in 2007 and 2008 indicate that law review editors may reach a publication decision based on factors other than the merit of a particular article. The volume of submissions limits the time devoted to reviewing a manuscript, with most editors formulating their initial (and often final) impression of the manuscript after five to thirty minutes.

Because law review editors are heavily influenced by an author’s credentials, a manuscript submitted by a legal studies professor, even if teaching at a prestigious business school, may receive less attention than an author who is a professor at a law school.

Today, well-known authors may be content to publish articles in lower-tier law reviews and specialty law reviews because, with electronic research, they are confident that they will be cited, regardless of the publication vehicle. Thus it appears that prestige does not correlate to influence, especially in the electronic age.

Accordingly, there is ample evidence that that the quality of articles published may not correlate with the perceived prestige of the law review where it is published.

A newer form of communication is the blog, which is a web page on which the blog author and blog readers log comments. A number of law school and legal studies professors author law-related blogs (sometimes called ‘blawgs’), each of which may concentrate on a specific area of the law, with the blog authors recognised as authorities on the subject. Whether and how blogs are related to legal scholarship is an issue that has already spawned several law review symposia.

Some consider law-related blogs to be legal scholarship because they are becoming a source of material and are occasionally cited in court opinions and law review articles by professors, judges, and attorneys. Blog readers provide a peer-review function vis-à-vis the posted information, critiquing what the author has posted.

Information can be published in a blog in a much more timely fashion than a print journal because the blog author publishes instantly, without the ‘preapproval’ of a journal. However, there is still a review process in that when blog authors provide commentary on breaking legal events, blog readers comment on web log postings. The commentary can create a real time dialogue between experts in a discipline and result in a contribution to the development of law.

Blogs can be a useful source for legal research because they are searchable databases on particular topics of interest to researchers. They can be repositories for information not appearing in print or on any other electronic database, such as recent lower-court decisions and practitioner developed case documents. In addition, a blog can become clearinghouse for information on its topic, with the blog author providing access to resources or making blog references to other sources and blog readers providing pertinent information to the blog author. The blog author and readers can alert each other to emerging legislation, new case law and critique legislative, regulatory and judicial changes on a very timely basis, as they are made or even as they are considered.

Blogging can be a precursor to more traditional forms of scholarship. The blog author can post material, float ideas, gather feedback from readers, revise ideas, post additional material, and continue the cycle, while, at the same time or consequently, publishing in traditional outlets. While a blog undoubtedly raises the profile of the organiser(s) and even of participants, its interface with the process of review of scholarship for purposes of promotion, tenure, renewal, and salary determinations is only beginning to be considered.

As might be expected, the percentage of ALSB members who published articles during the relevant time period was the highest for members located at comprehensive doctoral institutions, with the publication percentage lower for ALSB members located at less research intensive institutions. Analysis of InfoTrac data showed that 184 ALSB members (46 percent) located at comprehensive doctoral institutions produced publications.

Analysis of InfoTrac data showed that 152 ALSB members (37 per cent) located at master, undergraduate, and associate’s institutions produced publications.

A striking finding gleaned from reviewing the tables is that an overwhelming majority of legal studies professors published in specialty law journals rather than in general law reviews. If one of the goals of higher education is to develop and disseminate knowledge, a specialty law journal seems well suited to this goal; the specialty law journal allows the author to begin an intellectual dialogue with other experts in the field.

Legal scholars follow a different research model than other areas of the social sciences; the legal scholarship model, with a descriptive focus, is closer to that of history or philosophy than to sociology. Traditionally, legal scholarship is based on analytic and qualitative analysis rather than empirical and quantitative analysis and is not grant-intensive. Legal studies scholarship can be the same as law professor scholarship, but it can also be different in subject matter, form, publication outlet, and intended audience.

As a result, the evaluation of excellence in legal scholarship should be done with an understanding of the specific type of research produced.

Perhaps because of difficulty determining whether and to what extent scholarship contributes to knowledge, many institutions have developed quantifiable methods for gauging these contributions. Though practices vary among institutions, some simply count the number of publications while others focus on citation counts. Another approach that can be followed in a multidisciplinary department is to list a number of factors indicating quality of scholarship across the disciplines within the department and rank the candidate’s scholarship according to the factors applicable to a particular article.

An increasingly popular method is tiering journals, either formally on a list or informally. The advantage to this system is that it is seemingly easy to administer; however, it can be problematic. There are always journals that do not appear in any of the tiers. Does that mean that the journal is worthless, or is there a mechanism for evaluation of the journal and the article?

When developing and utilising a tier system, journals can be measured in many ways that can yield different ratings, including acceptance rate, peer-review or non–peer-review status, institutional affiliation of its editors and authors, ranking in surveys of peers within the discipline, and frequency of citation of its articles, either generally or in selected publication outlets. Sadly, some institutions place so much reliance on these proxies that attention is diverted from the substance of the scholarship. In effect, the particular publication vehicle often serves as a proxy for evaluating the quality of scholarship and becomes the only evaluative tool.

Use of ranking systems based on the publication vehicle may seem to discretely quantify a scholar’s success, but it can be misleading, especially when clear comparisons among disciplines are elusive.

One way to set the standard for excellence for legal studies scholarship is to examine excellence in scholarship in general. Glassick and coauthors identify six fundamental criteria as keys to quality scholarship: ‘clear goals’, ‘adequate preparation’, ‘appropriate methods’, ‘significant results’, ‘effective presentation’, and ‘reflective critique’.

Thus, building on Glassik et al, the authors of this article suggest that excellence in legal scholarship should be gauged by the following six factors: (1) Is the issue important and timely? (2) Is there thorough knowledge of the topic, often demonstrated through a literature review? (3) Is a critical or in-depth analysis of the topic using an appropriate analytical framework or appropriate empirical methodology provided? (4) Is there a contribution to knowledge in the field? (5) Is the communication effective? (6) Is it supported by appropriate authority?

These six factors should be considered in light of the mission of the scholar’s educational institution and the intended audience. Just as the mission of educational institutions vary, depending on the importance placed on research and teaching, the audience for legal scholarship varies from academics, to judges, to practitioners, to students, to business persons, etc. Depending on the audience, the scholarship will serve a different purpose and take a different format.

The legal studies policies and practices for researching and disseminating knowledge are distinct in many ways from those of other disciplines, even those within the same department or college. Because of these differences, it is incumbent on the legal studies discipline to educate members of other disciplines regarding its policies and practices.

Although selection of articles for a law review is not accomplished in the same way as in a peer-reviewed journal, the process can be just as, if not more, selective.

Professors in disciplines other than legal studies can be educated to view legal studies scholarship as integral to higher education because of its pivotal role in business and society at large. They should also be educated to evaluate publication based on the substance of the publication rather than its form.