‘How Is This Even Possible?’: Racial Justice Writing Competition Winner Examines AI’s Impact on Credit Scores

“You need money? Borrow it.”

That’s how Jay Esparza Castillo ’24 defines the current financial landscape in the United States in his paper titled “Fair Lending in the Age of AI: Regulations to Combat Discrimination in United States Credit Scoring Systems.”

The paper helped Castillo become the fourth winner of Chicago-Kent College of Law’s A Perfect Union Racial Justice writing competition.

“With inflation, people are reverting to using their credit for things like groceries,” says Castillo, demonstrating the need for ordinary people to have access to a line of credit.

The rise in artificial intelligence has worried Castillo that more consumers may be cut off from that  essential financial lifeline: credit.

“Traditional credit scoring takes into account the number of accounts you have open, your usage, your credit usage, your age, your address, and credit length,” he says. “But with the advent of AI, what a lot of software engineers have developed is a system where it’s going to take into account not only those traditional factors, but also things like your social media usage, your social media communications, as well as your socioeconomic status.”

If algorithms can start to refine consumers’ credit worthiness with such painstaking precision, Castillo makes the case that there is a possibility of discriminatory lending practices.

In his paper, he uses lender ZestFinance as an example.

“Their method involves employing machine learning algorithms to analyze thousands of data points per applicant, including unconventional factors such as the time taken to complete the application and applicants’ interactions with the terms and conditions on
ZestFinance’s application website,” he writes.

That means that people could be approved or denied based on how long they spent reading the terms and conditions of the potential loan.

He compares AI’s credit scoring potential to software that is used in the criminal justice system to determine how likely it is that a person would re-offend once released from prison.

“A lot of the factors that go into it are factors such as their relationship circles, the people who they surround themselves with,” says Castillo. “Who are my friends? Who’s my social circle? What are they posting? What kinds of pictures are they posting? What are the messages they’re insinuating when they’re posting something?”

In his paper, Castillo muses about how AI could use social media activity to determine credit worthiness. He highlights how difficult it is to believe that “someone posting a picture of themselves on Facebook or Instagram, showing off their jewelry, fancy dinners, or family lavish trips, could affect their credit score,” he writes in the paper.

Currently, there’s no indication that the major credit bureaus are using AI to determine credit scores, and there are laws in place to prevent discrimination in lending and credit scoring practices. But Castillo worries that lack of transparency in AI algorithms and scoring models could make it difficult to understand if these advanced technologies are leaving certain groups of people in the dust.

“FICO, TransUnion, all the other big corporations, they’re finding ways of refining the way you’re scored,” he says. “It’s a big threat and we have to know how to manage it.”

In the paper, Castillo critiques the current legislative attempts to control AI, but he says that none of them have yet tackled this issue. Castillo presents possible legislation that could confront this head-on before it becomes a problem.

“It comes down to laws being in place so that this technology is being able to be used in an ethical manner,” he says.

Castillo says that winning the prize felt “amazing,” especially considering that he was facing extraordinary obstacles in his personal life at the time he was writing the paper.

In October 2023 he was diagnosed with a brain tumor. Fortunately, the tumor was benign and surgery to remove it was successful. However, Castillo was going through a long, brutal recovery during his final semester at Chicago-Kent when he wrote this paper.

“You have to be thankful for everything that you have, and you learn value everything even more,” he says.

Castillo is currently clerking at Dinsmore’s Banking and Financial Services department, with “associate” status pending his admission to the Illinois Bar.

Photo: provided