Toward More Effective Search Terms For E-Discovery

By Ryan Davis

Law360, New York (July 28, 2009) -- Judges are beginning to lose patience with lawyers who submit careless, ineffective lists of e-discovery search terms that return mounds of useless results, as a handful of recent rulings makes clear. However, experts say there are some relatively easy steps lawyers can take to make sure the keywords they use to search e-discovery data generate useful results and avoid bringing down a judge's wrath.

In March, Magistrate Judge Andrew Peck of the U.S. District Court for the Southern District of New York handed down a ruling that he wrote should serve as a “wake-up call” to the bar about the problems caused by lawyers “designing keyword searches in the dark, by the seat of the pants.”

The case, William A. Gross Construction Associates Inc. v. American Manufacturers Mutual Insurance Co., involved a multimillion-dollar dispute over alleged defects and delays in the construction of the Bronx County Hall of Justice.

One side in the case had suggested searching through the e-mail of one of the construction management companies using very specific keywords like “Hall of Justice,” while the other side sought a list of terms that included generic terms like “elevator” and “sidewalk” that could presumably produce every e-mail at the company.

The management company provided little guidance about what it thought the appropriate terms should be, so Judge Peck was forced to construct the search terms for e-discovery in the case himself.

He tried to strike a balance between terms that were too broad and those that were too narrow, but criticized the parties for not doing more to come up with effective terms themselves.

“This opinion should serve as a wake-up call to the bar in this district about the need for careful thought, quality control, testing and cooperation with opposing counsel in designing search terms,” the judge wrote, citing other recent rulings that highlighted the importance of crafting effective search terms.

In only a few words, that sentence actually gives lawyers a succinct road map for crafting more effective search terms, said Farrah Pepper, chair of the Electronic Data Discovery Initiative task force at Gibson Dunn & Crutcher LLP.

Crafting search terms “is not a game of chance,” she said. “You don't just throw out a string of search terms and hope you hit the jackpot. The plan needs to be well thought-out and grounded in fact.”

In every case involving an e-discovery search, experts said, lawyers should be sure to consult with members of the organization whose data is being searched, make at least a good faith effort to collaborate with opposing counsel and test out the search terms by running a trial search before settling upon them.

“Of course, it's impossible to come up with a perfect list of search terms,” said Thomas A. Gilson, a partner at Lewis and Roca LLP. “Our use of language is almost infinitely varied; it's impossible to predict in the abstract how witnesses are communicating about the issues that are important to your case.”

Key players at the organization are clearly the experts on their own data, so it's foolish to try to design a way to search that data without consulting them, said Craig Ball, an electronic evidence consultant and court-appointed special master based in Austin, Texas.

Lawyers should ask the organization's data custodians how they would do the search and what technical terms, names and acronyms would be often used in e-mails or documents discussing the matter at issue in the litigation, he said.

Without talking to the people who created the data, attorneys may miss out on a number of potentially very useful search term ideas.

Collaborating with opposing counsel on search terms is probably a less appealing option for some lawyers, given the adversarial nature of court proceedings and the fact that “the same people responsible for getting you evidence are also responsible for frustrating your efforts,” Ball said.

Nevertheless, “there's no other way to do it. Collaboration is a significant cost-saver and the single greatest insurance against spoliation,” he added.

Collaborating with opposing counsel on search terms usually involves presenting your own list of suggested keywords, listening to their suggestions, even if they are light-years removed from your own, and trying to reach an agreement on some middle ground.

“Dialogue allows the parties to either reach an agreement or agree to disagree and get guidance from the court,” said Paul Weiner, the national e-discovery counsel at Littler Mendelson PC. “It benefits both parties and ensures both parties find the information they need.”

Some lawyers may have a lingering sense that there's an advantage to not collaborating with opposing counsel and withholding proposed search terms from them, but working together is really the best way to craft effective search terms, Pepper said. Failure to collaborate just increases the chances that there will be a conflict later on in the litigation.

“Cooperation isn't just a utopian concept with no place in successful advocacy,” she said. “It can end up saving clients hefty fees and expenses. Not everything in the adversary process needs to be adversarial.”

One common problem that arises when attorneys meet with opposing counsel is that they discuss search terms in the abstract, sometimes without the input of junior lawyers who will actually be doing the document review, said H. Christopher Boehning, a partner at Paul Weiss Rifkind Wharton & Garrison LLP.

As a result, lawyers may leave the meeting with some sort of agreement on search terms that ends up not being practical, so it's important for both sides to have done their homework about the case and have a discussion that's grounded in reality, he said.

Even if the other side is not responsive to your requests, the fact that you made an effort to discuss the issue with them can be important in litigation, Ball said.

“Being able to say the other side was approached and had an opportunity to object protects against what lawyers fear most, which is sanctions,” he said. “With e-discovery, even communicating with a stone is better than not communicating at all.”

Once a list of acceptable search terms has been proposed, but before the parties have reached a final agreement on them, it’s important to run a test search using the terms and a representative sample of the data that will be searched, Weiner said.

A test run can eliminate problems such as the potential for “noisy” hits far removed from what you're looking for, and until you test the terms on a data set and see what kind of results they generate, it’s difficult to refine the search terms, he said.

For instance, searching for the word “confidential” may seem like a good idea when a case involves disclosure of confidential information, but a test run may illustrate that a confidentiality footnote automatically at the end of each e-mail at the company renders the word useless as a search term.

Testing and refining a hypothesis “is a natural response in any other science, but it's often not done in the science of e-discovery,” Ball said.

Gilson noted that reviewing a portion of the “discard pile” of data that was not retrieved by your search terms can turn up relevant documents containing terms that should be added to your list. It's also important to document all your steps in writing, so you can demonstrate to the court that you made a reasonable effort to identify effective terms, he said.

Lawyers should ensure that any agreement with opposing counsel includes a provision allowing for the list of terms to be modified, Boehning said. For instance, changes should be allowed if the terms end up generating a slew of irrelevant results, or important information arises later that merits the addition of more search terms.

In addition to simple keyword searches, there are an increasing number of advanced search tools coming on the market that can generate even more effective and narrow results, Weiner said, and tech-savvy lawyers are becoming more comfortable using them.

For instance, “concept search” attempts to locate information that relates to a desired concept, rather than a certain word, so it can recognize that Eskimos and Sarah Palin are related to the concept of “Alaska,” even if the state's name is not mentioned.

Other tools that use mathematical probabilities and commercially available information “search data in a way humans never could,” Weiner said.

Despite the recent rulings highlighting the perils of ineffective search terms, it’s unlikely most lawyers will treat them as the wake-up calls the judges intended, Boehning said.

“Sadly, I think it's going to take a while” for lawyers to change their habits in favor of more effective search terms, he said. The general rules of civil procedure concerning e-discovery “have been with us for a while now, and people are still acting as if nothing has changed,” he added.

“It remains to be seen whether these opinions will have a trickle-down effect on counsel and force them to take the steps judges want them to take,” Pepper said. “But if there's anything that is going to make attorneys sit up and pay attention, it's the specter of an angry judge issuing chastising opinions against those who fail to heed their warnings.”