ACC Law Firm Scoring Seeks 'Wisdom Of The Crowd'

By Jocelyn Allison

Law360, New York (December 07, 2009) -- The Association for Corporate Counsel's new online system for evaluating law firms embodies one of the key factors in-house counsel consider when hiring time comes: what other corporate counsel think.

Since its launch in October, the ACC Value Index has been met with a fair amount of buzz in the legal industry, prompting criticism from some who question the adequacy of the scoring system and the decision to keep the data private, at least for now.

But in some ways, the system, which allows corporate counsel to share evaluations of law firms with other ACC members, is an expansion of a social network that some in-house counsel already rely on when making hiring decisions.

“It's taking advantage of the Internet and the ability to collect information and to share it,” ACC President Fred Krebs said.

“What we hope to achieve is very much the wisdom of the crowd, shall we say, in terms of our members and in terms of how they make use of law firms and which firms they have good experiences with, and perhaps which firms they had not so good experiences with,” Krebs said.

“I wouldn't say it would become the sole means someone would use, but it would become a valuable data point,” he said.

The top way corporate counsel find new firms is through a recommendation from a trusted source, often fellow inside counsel, industry observers said.

The Value Index, which allows anonymous submissions, won't take the place of a trusted colleague, but it can be a worthwhile data point, proponents said.

“When counsel are looking at what they want to know about a law firm or how to find one, our goal is to be one of the factors that they will take a look at,” Krebs said.

Under the system, corporate counsel evaluate firms in six areas: understands objectives/expectations, legal expertise, efficiency/process management, responsiveness/communication, predictable cost/budgeting skills and results delivered/execution.

They rate the firms from 1 to 5 — with 1 being poor and 5 being excellent — and then indicate whether they would use the firm again. There is a spot for comments, and respondents can choose whether to keep their name and information anonymous.

So far, 1,800 evaluations have been submitted for 600 law firms. The average overall score is 4.1, with the average score for cost/budgeting coming in lower than the other areas, Krebs said.

Mixed Reviews

The anonymity of the submissions has raised eyebrows among some in the legal profession, including legal consultancy the Zeughauser Group, which worries it could leave too much room for disgruntled partners and associates who have gone in-house to skew the ratings.

Zeughauser Group issued a client alert in November raising several issues with the Value Index's methodology and questioning a system that evaluates a firm based on respondents' experience with particular lawyers on a single matter.

The alert said the criteria should be more refined and the options for practice areas more specific, taking into account the fact that a single matter could span multiple practice areas and offices. But the biggest issue seemed to be lack of transparency.

“I think they need to stop talking to themselves and start talking to their outside lawyers,” said Zeughauser Group chairman Peter Zeughauser, who criticized the ACC for hesitating to share the evaluations with firms.

“They're trying to improve the dialogue between inside and outside counsel and better align costs and value,” he said. “There's a lot of talk, but they've reverted to their old ways of talking past each other. I think it's an odd turn of events after so many years of professing to encourage a better dialogue.”

The ACC plans to share the data with law firms in an aggregate form once more evaluations are collected, Krebs said.

They'll also be adding an option to the evaluation form that allows respondents to have it sent to the law firm if they choose, he said.

Krebs said the ACC modeled the Value Index after an evaluation form offered in the Serengeti Tracker matter management system, because a large number of corporate counsel are already familiar with the criteria.

The response in the legal community has been mixed, he acknowledged, saying the organization will continue to make improvements where needed.

The ACC welcomes all comments because it shows people recognize the potential of the index, Krebs said.

“The point is that people are paying attention to it and it is causing a discussion and a dialogue and conversation about expectation and value and client service and law firms,” he said. “All of these things are what we encourage to happen as part of the ACC Value Challenge.”

Client-Driven Ratings Rise

Based entirely on client feedback, the ACC Value Index is part of a move in recent years toward customer-driven ratings, the counterpoint in the marketplace to rankings based on revenue or profits per partner or ratings that rely on a lawyer's peer reviews, industry observers said.

“If you use the Internet today to inform your buying decision, you actually rely heavily on client ratings and feedbacks,” said Alfredo Sciascia, a vice president at LexisNexis, which publishes the Martindale-Hubbell ratings.

“The professional services industry was affected by this trend,” Sciascia said, “and I think we are starting to see that now with the number of providers that offer client ratings.”

LexisNexis Martindale-Hubbell, which had previously based its ratings on peer reviews, began offering client reviews in April. So far, 2,000 to 3,000 lawyers have been given a client score, which, along with peer reviews, becomes part of their overall profile.

“We've discovered over the many research initiatives that we've done that folks want different things,” said Carlton Dyce, a vice president with LexisNexis. “Some people want to rely on the peer review ranking or ratings in order to make their decision, and some want to rely on the client review, and some want to use them in conjunction.”

When corporate counsel are hiring outside counsel, the law firm's position in the various ratings and ranking systems is more likely to play a confirmatory role, industry observers say. But they also can be used as a discussion point when it comes to managing existing outside counsel.

“There are a fair number of corporate counsel that take the rankings and challenge their law firms and say, 'Why aren't you in the top?'” said Michael Rynowecer, president of BTI Consulting Group Inc., which has been conducting corporate counsel surveys for nine years.

“They may not say, 'You're fired,' but they're put on notice that the client expects more, so that can be valuable for both corporate counsel and the law firms,” Rynowecer said.

Jeff Carr, general counsel for FMC Technologies Inc., has used an evaluation system similar to the ACC Value Index since 2003 as a way to provide feedback to law firms and form the basis for the performance-based bonuses it pays firms.

Such systems provide a way for corporate counsel to communicate with their outside lawyers instead of giving up and calling the next firm when they're dissatisfied, Carr said.

“How can we expect outside counsel to perform well if we don't tell them what we want and we don't tell them how they're doing?” he said. “It's easier to just call someone else up because there are so many competent lawyers. I think that's wrong. We owe it them to have that discussion.”

Carr doesn't care much for ratings based on lawyers' peer reviews, and is leery of providers that generate part of their revenue by selling the results to the law firms or by soliciting advertising from law firms, he said.

“It's not that I don't trust them,” Carr said. “I think that having customer-provided, neutral data is far more relevant and far more reliable than some of the other systems that are out there.”

The world of law firm ratings and rankings is a crowded one, but some say the growth of customer-driven rankings like the ACC Value Index could be indicative of the balance of power in the legal services market shifting to corporate counsel.

“It is a competitive marketplace, but we don't act like it is,” Carr said. “I think some of this is reflective of the fact that the buyers of legal services are coming to grips that it is a competitive market and there is a fungibility of the legal services.”

Years ago, when the demand for legal services outweighed the supply, corporate counsel might have tolerated mediocre client service, but the shift in the market has given it new meaning, Rynowecer said.

“Now there is more supply than demand, and clients can be more selective, and client service becomes one of the few things out there that differentiates law firms, so any client-driven ranking takes on new meaning,” he said.