When a 'Wildly Irrational' Algorithm Makes Crucial Healthcare Decisions

"Thousands of disabled and elderly people in more than a dozen states have had to fight against decisions made by an algorithm to get the support services they need to remain in their homes instead of being institutionalized," reports the U.S. edition of the Guardian: The cuts have hit low-income seniors and people with disabilities in Pennsylvania, Iowa, New York, Maryland, New Jersey, Arkansas and other states, after algorithms became the arbiters of how their home health care was allocated — replacing judgments that used to be primarily made by nurses and social workers. In Washington D.C., "on the worst end, we've had clients who actually died, because their services were cut and they were not receiving the care that they needed" Tina Smith Nelson, supervising attorney with AARP Legal Counsel for the Elderly, said about the effects of a new algorithmic system introduced in 2018. Over 300 seniors have had to file administrative appeals after their home care was cut by a new algorithmic system. "I think as a society we move into unsettling territory when we rely solely upon algorithms and data to make determinations about health care needs," Nelson said. "We reduce a person's humanity to a number...." The situation is reflective of a reality increasingly affecting all users of American healthcare: algorithms — ranging from crude if-then charts to sophisticated artificial intelligence systems — are being deployed to make all sorts of decisions about who gets care. Government officials have touted algorithmic decision-making systems as a way to make sure that benefits are allocated even-handedly, eliminate human bias and root out fraud. But advocates say having computer programs decide how much help vulnerable people can get is often arbitrary — and in some cases downright cruel. The underlying problem, experts say, is that neither states nor the federal government provide enough funding to allow people needing health assistance to remain safely in their homes — even though these programs usually end up being much less costly than putting people in institutions. The algorithms resort to divvying up what crumbs are available... Kevin De Liban, an attorney with Legal Aid of Arkansas, began fighting the cuts after severely disabled patients started calling "en masse" in 2016.... De Liban's legal team revealed flaws with the algorithm in court. It turned out, De Liban said, that the calculations had failed to factor in things like whether a patient had cerebral palsy or diabetes. A single point in the scoring system — for instance a point added because the patient had had a fever in the last three days or had open pressure sores — could make a huge difference in how many hours they received for the entire year... "As the algorithm worked, it was, to our eyes, pretty wildly irrational," said De Liban... After years of court battles, Arkansas' use of the algorithmic system was finally thrown out in 2018... But across the nation, the battle continues. In Washington D.C., Pennsylvania and Iowa, legal services attorneys are plagued with calls from seniors complaining they have lost their care because of the algorithms recently adopted in those states. The Guardian ultimately tracked down the designer of the algorithm, University of Michigan Professor Emeritus Brant Fries, who acknowledged that the system isn't even designed to calculate how many hours of care people actually need, but to try to allocate whatever scarce resources are available in the most equitable way. "We're not saying that the size of the pie is correct... But whatever the money is there, I'm dividing it more equally!"

Read more of this story at Slashdot.

from Slashdot https://ift.tt/3heTSmn

Share on FacebookPlus on Google+

SUBSCRIBE TO OUR NEWSLETTER