Jim Greiner, director of Harvard University's Access to Justice Lab, brings a statistician's eye to the practice of law.
So after repeatedly encountering “heavy numbers” in his case load, the man who would eventually launch Harvard Law School’s Access to Justice Lab took a leap — from the practice of law to the study of statistics.
“I started to become frustrated with how I was unable to follow the experts,” he said. “And I was also finding out at the same time that I cared a whole lot more about facts than I did about political theory and rhetoric.”
The focus on facts propelled Greiner toward a 2007 Ph.D. from Harvard’s Department of Statistics, earned the same year he joined the Law School faculty. Since then, he’s worked to bring a concept called “randomized controlled trials” to the world of legal services.
Known as RCTs, these trials randomly assign treatments to different groups in an effort to determine which ones work better than others.
His quest to bring RCTs to the legal world is embodied in Harvard’s A2J Lab, and Greiner told Law360 that its goal of introducing a more evidence-based approach to the world of law “would most benefit and be most relevant for folks who cannot afford to hire lawyers to interface with the legal system for them.”
The lab’s goal is to have 22 studies completed or in progress by 2022. Here Greiner talks about the importance of evidence-based thinking in legal services and the ethical concerns of randomly assigning parachutes to people in a crashing plane.
Why do you think the practice of law is not evidence-based enough?
There’s this idea of “I’m a lawyer: I have experience, I have training, I have seen many things and I have talked to very, very smart people who also have lots of experience and training. And from all that, I know everything I need to know in order to make the best possible choices.”
Medicine thought the same way up until around 1938 or 1940. But then it began to realize that there were things that entirely human-based thinking could not tackle. And so it made a turn to a randomized controlled trials to try to find out things like which drugs work and which don't or which drugs work better or which drugs work with fewer side effects or which drugs are more cost effective.
And it made that turn — it wasn't an easy or a quick turn — but it made that turn as to drugs and medical devices and more recently has begun to make that turn to things like medical malpractice law.
Law never made that turn.
Why is it somewhat controversial or unusual to institute randomized controlled trials in law?
The reason that this kind of experiment is a challenging idea is because it requires an official — usually a lawyer or a judge — to temporarily give up control over a choice that she would otherwise ordinarily make, and give that choice to a randomizer for a particular period of time. And that challenges the idea that the lawyer or the judge is the greatest possible decision maker as to that choice.
What’s an example of how evidence-based law practice is different that current norms?
Let’s say a legal aid lawyer is confronted, daily, with the problem of too many clients and not enough time to provide each one with full representation.
The lawyer says, “I could spend all my time providing full representation, in which case I might be able to represent 20 percent of the people who walk through my door. Or, I could spend all my time writing self-help materials and updating people on their cases and providing the occasional phone call.
“Or, I can have some mixture of those two. I’d be able to tell which people are okay with self-help materials, if in fact there is anyone who is okay with just self-help materials, and which people really have to have individual representation or they won’t succeed.”
An evidence-based thinker might look at that problem and say, “Those are very hard allocation decisions to make. They depend on a lot of knowledge about how many people would succeed with self-help alone, or how many people will fail even if given full representation.
“Let’s do some randomized controlled trials: randomize whether people receive self-help materials or full representation for a set period. Follow everyone, observe their results and compare the costs and benefits of one program versus another.
“Meanwhile, let’s try to collect as much background information as we can on the folks that come in so that if it seems like we might want to have some full representation and some self-help, we can have some predictors based on scientific measurements as to who ‘needs’ the full representation in order to succeed (those may be the folks that you want to target for your full representation cases.)”
What are some of the lab’s biggest projects?
It’s important to note that these studies are extensive, multiyear projects that compare self-help with whole representation on a cost basis, and they include long-term follow-up on things that amount to, even though they aren’t technically, credit scores and credit reports as well as legal results.
So far, we have received a grant from the National Institute of Justice to pursue a large scale randomized trial on expungements of criminal records. Hopefully the design will tell us whether expungement provides a benefit to individuals who are successful in obtaining it, in employment and housing outcomes.
We also have trials in the field right now on pretrial risk assessment scores, which is a highly contested issue in criminal law. We are randomizing whether the judges have them in some cases and not in others, to see the varying effects of these scores.
Are there ethical concerns about randomizing people’s access to justice?
The really jolting way of expressing this is "when is it okay to randomize who gets a parachute coming out on an airplane that's going down?"
You might think "Oh my Gosh, It’d never be okay to randomize parachutes!"
Well if there aren’t enough parachutes to go around, that may be a fair way to do it. And also, the legal services situation we’re in, it’s not like parachutes — we know parachutes work, we know how many lives they’d save if you gave them to people in an airplane that’s going down.
We don’t know the success rates in terms of full legal services as compared to something like self-help plus the availability of a phone call. We don’t know that right now because quite frankly we ordinarily don’t look.
All Access is a series of discussions with leaders in the access to justice field. Questions and answers have been edited for length and clarity.
Have a story idea for Access to Justice? Reach us at email@example.com.