Judges are odd and uniquely frustrating creatures. Most have the demeanor of an especially hidebound DMV attendant, but their power and prestige gives them even more control over our day-to-day lives. Judges are often the last thing between you and imprisonment, deportation, homelessness, poverty, or the loss of your children. Their courts are tiny fiefdoms, and everyone who enters must cater to their whims. For lawyers—especially ones who actually care about their client—it’s almost malpractice not to.

A “good” attorney is forced to spend years learning the weird idiosyncrasies of any particular judge. If your client has been arrested for, say, sleeping in the atrium of a shopping mall, you must carefully calculate whether you will be met with sympathy or derision if you offer, say, an argument that “trespassing” shouldn’t apply in cases where the person was simply looking for a place to pass a cold night. Depending on which “legal values” a judge chooses to prioritize in any given case, you could end up with completely opposite outcomes. A lot is left to the whim of whatever particular judge you happen to be in front of. Perhaps that judge has some opinion about the issue that you’d better know. Or maybe you’ll just be more likely to win if you appear extremely deferential, or if you quote scripture as part of your argument—who knows?

Knowledge of the individual personalities of judges is such an important feature of the legal system that it operates as a skill in a lawyer’s toolkit, one that can be paid for. Supreme Court clerks who choose to go to big law firms after clerking on the court receive massive bonuses, often hundreds of thousands of dollars. The main asset they bring to their new firms is their firsthand knowledge of the intricacies of a particular Supreme Court justice’s mind. Large firms understand the strategic value of knowing a judge personally—what they like, what they dislike, what considerations will make them most likely to agree with you.

With judges wielding such concentrated and individualized power over cases, courtrooms quickly become stages for bizarre legal farces. Lawyers make arguments they don’t believe, that the judges know the lawyers don’t believe, but everyone has to play along. Only the judge has the power to decide when the game will end, and how. Let’s say, for example, that your client lost a case because they didn’t show up for a previous hearing. They likely missed that hearing for some reason that a normal person would find totally understandable: They didn’t have a lawyer at the time, they don’t speak English that well and didn’t understand what the hearing was for, they couldn’t get time off work, the bus got stuck in traffic on the way to the courthouse. But under the applicable statute, it’s likely that none of these perfectly rational and comprehensible explanations are admissible. In this situation, you know, and the judge knows, the real reasons the client missed the hearing. But you’ll have to try to make an argument about something totally different, something that this particular judge might choose to accept, even though they know that your argument has little to nothing to do with the reality of the situation.

At times, this peculiar trade in niche arguments feels thoroughly demented. If the judge wanted a bribe, that would at least feel normal. Everyone wants money. But what judges want is some strange intellectual product. Maybe they want you to cleverly contort the facts into some tiny legal box. Maybe they want to be convinced that doing whatever you’re asking them to do will quickly vanish the case from their docket and free the judge up to go to lunch. Or maybe the judge made up their mind about the case the second they glanced down at the paperwork, and is now simply idly watching you dance.

The fact that legal arguments are usually completely divorced from reality is partially a function of the law itself, and not solely the judges. That said, nothing prevents judges from acting like rational, normal people instead of playing games with people’s lives and making lawyers jump through hoops. Yet they often play these games, especially at the Supreme Court. They will straight-facedly ask lawyers, for the sake of argument, to justify things that are clearly insane.

Let’s take an example from a recent Supreme Court case: a U.S. border guard, standing on the U.S. side of the U.S.-Mexico border, shot and killed a child who was on the Mexican side of the border. The lawyer for the child’s family, in attempting to sue the border agent, argued that U.S. officials, when they kill people from inside the U.S., should be held liable. Now, it’s already ridiculous that the lawyer’s liability argument had to hinge on which side of the border the officer happened to be standing on, and not on the simple fact that a child was murdered and the person who killed him should obviously be responsible for compensating the family (I rest my case!). But it gets nuttier. The Court asked the attorney (paraphrasing), “Well, what about drone pilots who sit in Nevada and murder people in Pakistan? Are you saying we should hold them liable?” The lawyer—knowing that no court thinks drone pilots are liable for anything, knowing that if he says, “yes, they should also be liable” his client’s case will be lost—felt forced to make an argument that of course drone pilots are different, for… for some reason. In reality, of course, there is no substantive difference between a drone pilot who murders people from inside the U.S. and a border guard who murders people from inside the U.S. The lawyer knew this, and the Court likely knew it too. Yet the Court forced the lawyer to go through the exercise of attempting to draw an insincere distinction, making the lawyer look silly and further distancing the Court from the actual important questions.

Now, it’s possible you might think that we’re being unfair to judges here by expecting them to care about the same things (children being shot to death, poor people having their lives irreparably ruined by filing errors, etc., etc.) that laypeople care about. The law is complicated, after all, and judges are legal experts. Isn’t it natural that judges would care about abstruse legal arguments that the rest of us can’t see the relevance of, because they understand the larger ramifications of each individual case for our great legal system? Without this far-seeing vision, wouldn’t our entire civilization go down in flames?

To this, we can only say that A) this is a terrible way to run a society, and B) this is not what actually motivates judges in 99 percent of cases. Judges are just ordinary people like you and me. They do not behave in the strange way they do because they are wise beyond the ken of mere mortals. They behave this way because our legal system has created a climate calculated to warp judges’ minds. We once spoke to an immigration judge presiding over a court that was attached to a detention center. Her docket consisted largely of asylum-seekers who had been detained after crossing the border, and would appear before her (in jumpsuits and shackles) to try to argue their own cases without lawyers. The denial rate at this detention center was high. Most of these asylum-seekers would be sent back to countries where they claimed to fear for their lives, and the judge would never know their fates. We asked the judge: “What’s the hardest part of your job?”

She seemed to think for a moment.

“Our computer system crashes a lot,” she said. “It’s really annoying.”

How did we get to this point? What are judges actually for? Historically, judges were basically people who were good at dispute resolution at a time when state enforcement wasn’t really a thing. They needed to be able to come up with judgments that parties would actually follow, instead of just, say, murdering each other. This judicial role makes some kind of abstract sense. In less regulated times, judges were probably pretty useful practical agents of community order, and even sometimes of justice. In the Bible, for example, there’s a lot of language about the ideal judge being someone who defends the interests of the weak and downtrodden against the tyranny of the powerful. (The number of verses invoking fire and brimstone against “unjust judges” who “even rob the widows and fatherless children,” though, tells us that this ideal probably wasn’t always borne out in real life.)

But then we come roaring into the present. Here in the U.S., we no longer live in a time where it’s typical for vigilantism and blood-feuds to break out on a large scale if the judge doesn’t make a decision the parties will willingly accept. (Although in the case of, say, Bush v. Gore, that would have perhaps been a preferable outcome). So what function do judges perform now? Rather than just making pronouncements on what would be equitable in a particular dispute, judges are—theoretically—supposed to faithfully apply codified law. We’ve made a policy shift over the years that strongly favors “predictability,” in place of “justice,” as the chief virtue of the legal system. It’s a well-accepted principle of the judicial craft that the important thing is to have a rule, rather than to necessarily have the right rule. (Having the right rule is merely a subsidiary concern, if it’s a concern at all.) We’d much rather have all “similar” cases or issues get decided the same way, instead of different ways, even if this leads to worse practical, real-world results overall for more individual people. Under this system, say, it’s much better if all U.S. agents who kill people from inside U.S. territory are deemed categorically ineligible to be sued, because this is predictable, and predictability is fair. To have a situation where some border guards are sueable and others aren’t, by contrast, is unfair. People would never know what to expect in a world like that! (Except that there might, you know, occasionally be some consequences for people who slaughter other humans from a distance, but again, what do we know, we’re not judges after all.)

But even in a world with “predictability” and “uniformity” as the highest legal values, we still have pretty confused notions about the appropriate role of a judge. Some people believe that judges should apply law “impartially” to the facts and not let personal or political feelings interfere with their interpretation. Other people, who see the judiciary as a tool for social or legal change, believe that judges should be (what’s sometimes pejoratively described as) “activist judges,” who push and twist the law to fit their own political ends. A lot of the public argument around judges revolves around which of these two understandings of the judge’s role is better for democracy, in the abstract and/or for particular policy outcomes.

But this binary between “impartial” and “partisan” judges is actually pretty nonsensical. First, evidence suggests that judges are not capable of being impartial at all. Factors as insidious as racial and class bias, as reasonable as background and knowledge, as mundane as when the judge last ate a meal, and as calculated as the judge’s ambitions for career advancement or political office, all clearly influence their choices. Numerous studies have shown the way many of these factors (not legal standards or arguments) lead to predictable patterns in rulings. And the reality is, most judges, and most people who closely study the activities of judges, know they aren’t impartial. When judges exalt impartiality as the highest virtue of legal adjudication, they are usually doing so disingenuously, or with significant psychological blinders raised. Judges who most vocally pretend to be impartial are often those who practice it the least.

Take the late Antonin “Nino” Scalia, who talked all the damn time about how one should just read and apply the “ordinary meaning of the plain text of the law.” As Scalia once wrote: “The good judge must suppress his personal views and must decide each case as the law dictates, not as he would have resolved the matter if he had drafted the law or the constitutional provision at issue.” The reason for this, Scalia continued, is because “when the vocation of a judge is reduced to simply selecting the best rule, remarkable power is placed in the hands of a few persons who are barely accountable for their decisions.” Superficially, this sounds reasonable. But this simple presentation of the issue neatly sidesteps the reality that the “ordinary meaning” of statutes is very often so unclear—both within the statute itself and across different statutes—or so tenuously-connected to the practical question under debate, that a judge can easily come up with several equally plausible interpretations. For Scalia, the supposedly neutral “servant of the law,” the interpretation that best aligned with his socially conservative views almost always turned out to have been the “ordinary meaning of the plain language” all along! Funny how that happens! (This sometimes required Scalia to be pretty creative about what constituted “ordinary meaning”—when he didn’t like the “a well-regulated militia, being necessary to the security of a free state” part of the Second Amendment, for example, he simply declared that it was a purely decorative “prefatory clause,” and when he didn’t like the fact the “plain meaning” intended by the Second Amendment’s drafters could not possibly have anticipated the semiautomatic handgun, he adopted a belief in a kind of evolving Constitution. And for someone who professed to dislike the idea of unaccountable judges thwarting the popular will, he certainly had no qualms about invalidating campaign finance reform legislation passed by the democratically-elected legislature.)

All right, so Justice Scalia was a hypocrite whose impartiality was a sham, that’s not exactly Breaking News. But even if judges could be impartial, in theory, is that something that anyone even wants? Laws are inextricably linked with politics, so even the most neutral application of the law has a political dimension. And when people talk about who they want as a potential Justice it’s clear they base their preferences on who shares their opinion on issues like free speech or abortion. We rarely hear a conservative complain about Scalia’s judicial activism, or a liberal bemoan that RBG wasn’t impartial enough. This is because, deep in our core, we want partiality. Even where we have different conceptions of what “justice” means, we do actually want judges to advance justice rather than merely law.

The other problem with impartiality (besides being near impossible and possibly undesirable) is that a lot of seemingly “impartial” legal standards—like the famous “what would a reasonable person do” standard—are inherently subjective, so that it’s hard to say what an “impartial” application would even mean. The law is full of attempts to determine what “reasonable” behavior would be in a particular situation. It should shock no one (except lawyers) that people often have wildly divergent views of what “reasonableness” means in any given situation. For courts, the “reasonable person” standard has a disturbing tendency to align with whatever best suits the positions of those in power. Think of all of the police officers whose shootings of unarmed black people have been deemed “reasonable”—and then say you want a judicial system run by “reasonable” or “impartial” judges.

Even where judicial discretion isn’t explicitly authored into the system, there are vast areas of law where a judge has the choice between several equally plausible legal arguments, each leading to different outcomes. This, in effect, empowers the judge to make whatever decision they want. This view of the judicial decision-making process was expounded by Richard Posner, one of the U.S.’s most well-known jurists, in an interview with the New York Times after his sudden retirement in September 2017. In Posner’s estimation, it’s rarely difficult for judges to do as they wish: His own modus operandi was to decide what a “sensible” resolution of the case would be, and then look to see if there was any precedent that explicitly barred him from implementing his preferred solution. “And the answer is that’s actually rarely the case… When you have a Supreme Court case or something similar, they’re often extremely easy to get around.” (In the same interview, Posner characterized his judicial career as “a slumber of 35 years,” during which he failed to realize how much poor people without lawyers are screwed over by the legal system. It’s unclear why it took Posner 35 years to experience this revelation, but at least he got there eventually.) As an example of what this kind of judicial freedom can look like in practice, take a look at the disparate grant rates for asylum cases in different parts of the country. Immigration judges in New York grant 88 percent of asylum cases. Immigration judges in Atlanta grant 2 percent. Both courts are nominally employing the exact same legal standard. The difference is that most judges in New York are looking for reasons to grant cases, and most judges in Atlanta are looking for reasons to deny them. Usually, the judge will find whatever they are looking for.

Given the basic incoherence of the whole concept of “impartiality,” then, is the answer that all judges should be “activists”? Would it at least be better for a judge to be honest about what they’re doing, rather than hide behind a screen of impartiality? If a heartless but “honest” judge is still doing substantively the same thing they were doing back when they pretended to be “impartial,” that might be slightly better, but not by much. (Our current president, after all, is very honest about the terrible things he’s doing, and in a lot of cases, this hasn’t seemed to make his plans any easier to combat.) But this ignores what people see as the main benefit of an “activist” judge. It’s not just that they’re more honest about what they’re doing, it’s that maybe they’ll be able to use the law to “save us”. This is the other problem that arises when you champion open judicial “activism”—you tend to over-emphasize the extent to which judicial rulings actually result in social change.

Take Brown v. Board of Education, which the general public often thinks of as a great victory of legal justice against racist tyranny, and a watershed moment for civil rights. (It’s certainly one of the only Supreme Court cases that a lot of non-lawyers could name off the bat.) But it’s a very clean, simplistic view of history that thinks “well, at one point there was segregation, and then there was Brown v. Board of Education, and suddenly there wasn’t.” In fact, this isn’t true at all. In post-Civil War America, segregation fluctuated and was fought against over decades of social movements. In Louisiana, for example—a few years before the infamous 1896 Plessy v. Ferguson decision which legally justified segregation—ox-carts were desegregated after a campaign of sustained protest and sit-ins. Social change doesn’t happen in the courts: Courts just eventually catch up to what’s been happening on the ground. If we instead depend on activist judges or lawyers, then social change will never come, and if it does, it will likely not be sustained. Try telling students in Boston’s deeply-segregated public school system that segregation in education doesn’t exist because of some Supreme Court case. They’ll laugh you right out of their crumbling gymnasium.

A very clear example of how legal argumentation doesn’t drive social change comes from the gay rights context. In 1971, in Baker v. Nelson, the highest court in Minnesota ruled that a state law banning gay marriage did not violate the U.S. Constitution. The couple seeking to be married tried to appeal to the U.S. Supreme Court. The Court at the time was one of the most liberal we’ve ever had, just on the tail end of the famed Warren Court. The Court denied review of the case, stating that there was no Constitutional issue to review. Fast forward to 2015, when Supreme Court struck down a gay marriage ban in Obergefell v. Hodges. What happened in the intervening forty years to make this possible? The Constitution didn’t change regarding these rights. The legal arguments didn’t change either: If you read the briefs from both parties, the arguments advanced in 1971 and 2015 were pretty interchangeable. And the court in 2015 was more conservative, so the deciding factor wasn’t some new critical mass of “activist” liberal judges. The only thing that explains this landmark shift is that, over the past four decades, both mainstream and radical LGBT rights movements had taken to the streets and created a cultural shift. Obergefell is due much more to the activities of ACT-UP and the post-Stonewall gay rights movement than to Anthony Kennedy suddenly being a pro-gay rights “activist judge.”

Given that this is the reality of the situation, what should be the appropriate role of judges in determining the direction of public policy? How much discretion should they have to delay or override actions by the legislature, or the executive? At the moment, it’s hard not to cheer when a liberal judge from the Ninth Circuit issues an injunction against Trump’s Muslim ban, or pushes back the imminent termination of Dreamers’ DACA status. But this, of course, can go either way: In 2015, a conservative judge from a court in the Fifth Circuit was responsible for hamstringing Obama’s DAPA program—which was supposed to extend relief from deportation to parents of U.S. citizen and permanent resident children—using the exact same judicial maneuver. If you are mostly indifferent to The Separation Of Powers in the abstract and simply favor whatever set of conditions seem to produce better moral outcomes, it’s easy to feel conflicted about judicial power. Perhaps we rather like having a judiciary that, with a colorable legal pretext, can periodically throw a wrench in the governmental works on the theory that it’s worth facing occasional legal obstacles to implementing good policies, so long as we retain the ability to sabotage truly egregious policies when they come along. But even this calculus depends on there being some critical mass of judges whom we believe to be our ideological allies (and/or basically decent moral actors) occupying high positions in the judiciary at any given time. This is far from a certain proposition. As with most checks and balances, the judiciary’s ability to thwart the other branches of government isn’t so much a reliable safeguard against tyranny as a wild-card element that occasionally works out in our favor.

More broadly, the temptation to view judges as potential saviors often seems to sap progressive will for reform efforts through electoral and legislative channels. We currently have a legislature that is chronically unresponsive to genuine public concerns, and an ongoing concentration of de facto governing power in the president and his executive agencies. In this context, it’s understandable that people want to think of the judiciary as the last, best hope of American democracy. But even an optimally moral and courageous judiciary can usually only engage in obstructionist tactics, which a sufficiently determined executive will then maneuver around, unless the public finds some other way to make this politically inexpedient. The danger of reposing too much power in the judiciary, too, is amply illustrated by the Supreme Court’s Citizens United decision, where the Supreme Court declared unconstitutional the exact kinds of reforms that would have made the legislature more beholden to the real interests of the majority of their constituents—and which would thus have reduced the importance of the judicial deus ex machina. If we aspire to a form of democracy where there is an actual connection between the organizing efforts of the general public and the subsequent behavior of our elected officials, pushing for reforms to make our elected government more responsive to popular concerns is a better route than relying on distant elites to undo the mistakes of other elites. When you put power in the hands of unaccountable elites, you never know what they will do with it.

So much for judicial impact on public policy writ large: What kind of power should judges have to decide the fate of the ordinary people, most of them poor, who come before them with criminal, domestic, housing, and immigration issues? Lower court judges—as opposed to appellate judges, who are often dealing with weird standards of review—have considerable discretion to reach whatever decision they wish to reach. (And in the sense that law is important in the impacts that it creates on one’s life, district and municipal judges have much more legal power than the higher courts since they decide most of the cases). In some sense, we might think this is a good thing: If a plaintiff or defendant’s situation is complicated, we could want the adjudicator to have the flexibility to weigh whatever factors seem most relevant, rather than being totally bound by one-size-fits-all rules. (Consider, for example, the disaster of mandatory minimums, which were originally touted as a means of curtailing the improper and inconsistent exercise of judicial discretion in sentencing, or consider risk evaluation metrics that turn peoples’ bail determinations into an algorithm in an effort to reduce inconsistency.) But if you have a judge who fundamentally does not care about the person in front of her, or grossly misunderstands the actual circumstances of their life, this broad discretionary leeway will be, at best, useless. At worst, it will empower the judge to make a unusually bad and biased ruling.

Part of the problem, of course, is that judges are separated from poor litigants by class and, often, race. If we want more judges to exercise discretion in an empathetic direction, it seems crucial to diversify the pool of judges, perhaps directly through quotas, or indirectly by reducing social and financial barriers to entry in the legal profession. Also important is changing the dominant ethos of legal education, which overwhelmingly privileges the pet concerns of legal academics and corporate clients over the kinds of issues that affect the vast majority of the people caught up in our courts. But these initiatives by themselves likely wouldn’t be sufficient to cure the defects in our judiciary. Just as diversifying the police force hasn’t been a silver bullet in eliminating police brutality, we can’t simply address who is on the bench and ignore the problem of the power that judges have in and of itself. Since there’s no good way to actually measure judicial “empathy,” you run the risk of simply adding a diversity gloss to a fundamentally unjust system.

This problem does not have a simple solution, but there are perhaps three clarifying ways we can think about the Problem of Judges. One, we should break any habit of looking to the law as a primary source of social change. Sure, we want to eliminate bad laws and enact good ones. But changing the law is usually the middle- or end-point, rather than the starting-point, of on-the-ground changes in human social behavior. Most of the real work will happen not in the courtroom, but in the streets: Social justice-minded lawyers play, at best, a bit part or supporting role, formulating legal avenues for the changes already underway to become more solidified in formal practice.

Secondly, when we do seek to enact legal change, one of our concerns should be to craft legal standards with an optimum humane baseline, so that the role for judicial discretion, so far as possible, is forced in the direction of mercy. Attempting to make perfect legal rules for all cases is a futile exercise, but we can at least shut off some possibilities for bad exercises of discretion. (Sometimes, the best way to do this is to get rid of the stupid law that would have thrust a person into the back of a police car, and then in front of a judge, in the first place.)

Thirdly, it’s naïve to expect judges to be unusually moral people. That said, basic morality remains the only proper standard by which to assess whether someone is a “good” judge or not. A good judge is someone who uses whatever discretion and whatever legal tools are at their disposal to reduce human suffering. They actively seek to understand the human impact of the cases in front of them. They are humble enough to admit what they don’t understand, and to solicit whatever information or advice they think will help improve their understanding, so that they can make a decision that they believe will be really helpful to people. They care about what happens to the parties in the case. Sure, inasmuch as we disagree about what “goodness” is, this standard for assessing judges is squishy. But we’d rather have that argument any day of the week than 99 percent of the arguments lawyers are forced to make in court.

The solution may not be clear, but what is apparent is that judges are more concerned with law than justice and that they have far too much power to ruin peoples’ lives. Ideally, no one would actually be able to authorize your ouster from your home or the prolonged caging of human beings—so to the extent that we can limit that, we ought to. Beyond that we must strive to make the system and the people within it more just and reduce our dependence on that system for justice and morality in the first place.

Be sure to take our accompanying quiz, “You Be The Judge.”