Advice From Counsel: Can Predictive Coding Deliver On Its Promise?

Monday, December 17, 2012 - 08:52

The Editor presents the following summary of a whitepaper with the above title that was commissioned by FTI Consulting Inc. and authored by Joe Looby, Senior Managing Director with FTI Consulting, Inc. (“FTI”), and Ari Kaplan, Legal Consultant and Principal of Ari Kaplan Advisors. Please download the complete whitepaper here

 

Download the Whitepaper Now!

 

Introduction

While the promise of predictive coding is alluring, many questions remain for corporations and law firms. Where does the software end and the importance of workflow begin? What can lawyers do to effectively defend its use? Are companies using it successfully? How much money can it save the client? Is the technology well-suited for all types of legal matters? Do law firms and corporations have similar perceptions of the technology? What is the perspective of the courts? Will future rulings impede or accelerate adoption of machine learning and classification technology in legal review?

Predictive coding has sparked discussion among e-discovery practitioners as no other technology has. From conference panels to blog posts, attorneys debate whether it will eliminate the role of human reviewers and question its capacity to defensibly automate and reduce the cost of e-discovery. Numerous providers of e-discovery software and services add to the industry hype by touting competitive software classifiers. Yet for all of the excitement surrounding predictive coding, the technology itself is not new. Patents for machine-assisted document classification tools have existed since the 1970s, and machine-learning patents relating to e-discovery are a decade old.

FTI Technology commissioned an interdisciplinary survey of law firm leaders and senior corporate counsel to identify key trends and perspectives on the emergence of predictive coding. This report summarizes the findings collected via phone interviews in April and May 2012. Thirteen in-house counsel from Fortune 1000 companies and 11 Am Law 200 (American Lawyer) law firm partners and senior counsel were interviewed for this report. More than half of the respondents have used some type of predictive coding technology, and all expressed an interest in the industry discussion around predictive coding.

Respondents were asked a wide range of questions relating to predictive coding, including their thoughts on high-profile court rulings, cost savings estimates, and adoption inhibitors. The feedback reveals widespread interest in the technology, evidence of successful predictive coding pilot projects, and optimism about its ability to reduce e-discovery costs in the long term. However, skepticism and uncertainty remain. Respondents expressed a healthy amount of concern about the reality of cost savings, the defensibility of the technology and its usefulness for certain types of legal matters.

Participants

FTI Technology focused on the perspectives of large corporations and law firms. The 13 participating in-house lawyers represented Fortune 1000 companies within the manufacturing, insurance, life sciences, energy, financial services, retail, technology, and transportation sectors. All 11 of the law firm participants practice at Am Law 200 firms. All responses were provided under condition of anonymity.

Key Findings

The whitepaper identifies 11 recurring themes and provides a detailed and quantitative discussion of each. Readers will appreciate the numerous quotes from survey participants, which enrich this discussion by providing insights into actual experiences in the field. They are well worth reading in full. The themes are very briefly presented below:

Lawyers are trying predictive coding with some promising results. While there is evidence backing up the assumption that adoption has been light to date, there are positive signs for future adoption. Participants who used predictive coding – often through pilot projects with vendors – report positive feedback and successful results. One participant also expressed caution that “it is not a magic bullet on its own,” particularly noting the necessity for and cost of manual review.

For most respondents, predictive coding really means fast culling and prioritizing. As the above respondent indicated, predictive coding is not a silver bullet for automatically coding an entire data set. In fact, very few respondents appeared to use the technology for actual document coding. The majority discussed using predictive coding technology as advanced keyword search functionality so that reviewers could focus in on important materials faster.

The verdict is still out on cost, as well as on potential savings. Perhaps because the majority of the participants’ reported experiences were relating to pilot projects, no clear consensus emerged on real-word, or even potential, costs. Instead, the whitepaper provides ranges of reported costs, and quotes from participants offer useful insights into the logic behind this data.

“Garbage in, garbage out.”  Predictive coding is valued for its potential to eliminate or reduce costly human review. However, the respondents overwhelmingly agreed that human input is even more important during the predictive coding process. Successful results depend on the quality of the seed set and the quality control, which is driven by human reviewers, not by a push-button process.

Some matters are better suited to predictive coding than others. Eighty-eight percent of respondents agreed that certain case parameters support the viability of using predictive coding. The whitepaper presents a substantial discussion here, including specific parameters to consider, concrete examples of where predictive coding should be considered and similar examples of situations in which predictive coding may be less effective.

Top-ranked benefits of predictive coding. On a related theme, the respondents were asked to rank the top reasons they would or would not use predictive coding. Four top benefits are discussed in detail, including prioritized review, reduced costs by eliminating irrelevant documents, testing capabilities to ensure accuracy, and the ability to find more responsive documents. Other benefits are briefly mentioned to round out the results.

Top-ranked concerns about predictive coding. The corresponding concerns are discussed in equal detail and measure. They include: the “black box” nature of the technology, not being well suited for investigations (finding a “needle in a haystack”), fear of inadvertent productions, and the fact that respondents don’t want to be early adopters. Among the other concerns listed is cost, which interestingly appears as a benefit and a concern. Please see next theme.

In certain circumstances, predictive coding can cost more. While cost concerns barely missed ranking in the top four, they are a running theme throughout the survey results. This section of the whitepaper breaks down this issue along substantive criteria, such as urgency or scope of the case and long-term versus short-term cost factors.

Experts needed. Because predictive coding is just as much a process as a technology, there was widespread acknowledgement that experts, both legal and IT, are an integral part of the process. According to one respondent, “You will still need lawyers who run those machines correctly.” Further, many participants openly discussed their lack of technical knowledge, with one stating that “understanding the different predictive coding algorithms and classifiers is for the technology geeks to deal with.”

Emerging case law is having an impact. One hundred percent of the respondents have heard of and are following developments around two predictive coding matters, Da Silva Moore v. Publicis Groupe; 11 Civ. 1279 (S.D.N.Y. Feb. 24, 2012) and Kleen Products, LLC v. Packaging Corp. of America; 1:10-CV-05711 (N.D. Ill.). Fifty-eight percent said they were more likely to use predictive coding because of these cases. The whitepaper continues with an interesting discussion of the different responses to such judicial commentary from law firm versus from in-house participants.

Corporate and law firm attorneys are largely in alignment. Notwithstanding those different responses to case law vis-à-vis anticipated use of predictive coding, corporate and law firm responses were largely aligned on the majority of other issues raised.

Looking Ahead

Based upon the survey responses, it is possible to forecast coming trends that legal teams should be aware of. Summarized very briefly here, the first trend points to the anticipated demand for predictive coding experts. Second, predictive coding will be less “black box.” Users will increasingly demand transparency and metrics, and such demand will only increase should we start to see adverse court decisions. Finally, because technology solutions constantly evolve and should be adopted based on the needs of each matter, corporations and law firms may avoid purchasing a predictive coding technology outright and, instead, select the right tool for each job, as needed.

Summary

Corporations and law firms are taking a measured and pragmatic approach to the adoption of predictive coding technology and workflow. Pilot projects, a reliance on “techno-lawyer” experts, as well as a focus on the types of matters best suited for predictive coding demonstrate a keen understanding of the technology’s promise and potential hazards. As organizations transition from pilot programs to regular use, the industry as a whole will benefit from broader focus on and awareness of these potential hazards, especially around cost calculations and defensibility. With this focus, predictive coding can deliver on its promise.

 

 

Joe Looby is a Senior Managing Director with FTI Consulting, Inc. (“FTI”) and has more than 20 years of combined experience in the military, regulatory enforcement, investigations and technology, much of which has involved the research, documentation, explanation and validation of proprietary software technology, computer models, and technical processes. Mr. Looby is a certified fraud examiner (“CFE”); he holds a bachelor’s degree in economics and a juris doctorate.

Ari Kaplan is a legal consultant, Principal of Ari Kaplan Advisors and a prolific author of articles as well as numerous highly acclaimed books on the professional services market. After nearly nine years practicing with large law firms in Manhattan, Mr. Kaplan, named to the inaugural Fastcase 50 list of innovators in the law, has become the leading copywriter and industry analyst in the legal community.

Please visit www.ftitechnology.com for more information about FTI.