When algorithms meet debt...
Everyone must be aware of four (no, five) problems. Especially lawyers.
We “First World” countries love to hate the so called “China’s Social Credit System”. And surely there is a lot to object to such system. The problem is being unable to recognize the same thing, when it happens at home. Especially, maybe, when it comes to debt.
Debt has been in society probably since well before the formal adoption of “money”, but (both as individuals, and at the social level) we haven’t quite managed how to deal with it yet. That is why, probably, debt and its “management” are a growing problem today. The snapshot below is about how Indians could land in a debt trap without even realizing it, and the thumbnail of this post comes from a 2018 report on Silicon Valley startups working on debt collection algorithms.
One exception to this situation is some US lawyers have recognized the system, and are actively helping people that fall in poverty because of it. I am talking of the lawyers who declared “war on the hidden algorithms that trap people in poverty”.
Land of the Free, Home of Debt Traps
What follow is a description of the current situation in the US, that even this time host the best and worst of some issue. But the same thing, basically, is happening or about to happen everywhere, so better be prepared, lawyer or not:
- Credit scoring has been largely automated. The same is happening to public administratin procedures that affect access to e.g. health care, unemployment, and child support services.
- When it is passively adopted, it is the software that implements these procedures that “increasingly affects whether you can buy a car, rent an apartment, or get a full-time job”.
- Once the same software makes some mistake, it can be nearly impossible to recover
If, or: until you do have money, no problem. You just won’t see any of this. If you barely makes ends meet, instead, this is a “hidden web of interlocking traps”.
Modernization is necessary. But it must be done well, and never left alone
Mind you, I’m all for digitization of many services. It can, and really should save tons of money and other resources, from energy to human well-being. But it cannot just have speed and cost savings as metrics. Accountability and recovery of errors are at least equally important. To steer things in that direction, it is necessary that the general public knows how things really work, or should work. The subjects of this post, that is lawyers, should know much more, and quickly. The starting point, as that article explains, is awareness of four problems. Plus one, I think.
Problem 1: obscurity, and poor quality of inputs
These algorithms are fed data from a too wide range of sources (soome of which should not even exist in the first place, regardless of their uses): public records, social media, web browsing, banking activity, app usage, and more"_. But too much of this stuff is collected just because it exists, without any context, or quality checks.
Problem 2: procurement ignorance, and abdication of responsibility
Public agencies often buy automated decision-making tools directly from private vendors, without having a clue of what they are really buying, and how it really works.Even worst, usually those private companies “don’t divulge how they come to their decisions”. That is one important reason why, when something goes wrong, both the individuals affected, and their lawyers, if they ignore these problems, are left in the dark.
Problem 3: temporary problems become permanent
This is a consequence of the previous problem, of course: “Coronavirus-related debts and evictions will get codified into credit scores, making it permanently harder for people to get jobs, apartments, and loans."
Even without pandemics, the stupid, STUPID fragility of our “advanced” way of living greatly increases the probabilitiy that "one, temporary economic hardship can lead to a vicious cycle that sometimes ends in bankruptcy or homelessness." Unsurprisingly, “the consequences are much more harsh for poor people and minorities." For an example, check out in the article the full story of the person who “lost work because of the pandemic and was denied unemployment benefits because of an automated system failure”.
Problem 4: it’s going to get worse. Invisibly
Many laywers still don’t realize any of this is going on, “because a lot of these systems are invisible”, and are unable to ask the right questions to uncover the algorithms. But, at least in this field, “ultimately it’s like every case is going to turn into an algorithm case."
Lawyers must quickly prepare themselves for this. Everywhere
“Poverty lawyer” is one of the names for lawyers who represent people who’ve lost access to basic needs: housing, food, education, work, or health care. Increasingly, the fight over a client’access to those needs is against algorithms: “Basically every civil lawyer is starting to deal with this stuff… We need to wake up, get training."
Michele Gilman, a clinical law professor at the University of Baltimore, is working just on that. In September 2020 she released a report documenting all all the problems above, called“Poverty Lawgorithms”/ The report is meant as a guide for lawyers in the field, but should probably be read by many more people. I do not exclude making more posts in the future, to popularize some of its part. In the meantime, check it out as it is, without forgetting the fifth problem.
Problem 5 (by me): cashless makes things even worst
In and by itself, the story above is about the consequences of letting software decide unchecked how you can get money or services. Thing become even worst, when software also gets the unaccounted, non-circumventable power to prevent you from spending money that is already “yours”. When, that is, society becomes fully cashless. Think about that, starting from: