A Survey of Fair and Responsible Machine Learning and Artificial Intelligence: Implications for Consumer Financial Services

by Stephen C. Rea, PhD, Research Assistant Professor at Colorado School of Mines and former IMTFI Research Assistant

Capital One Eno Chatbot

Stephen Rea recently published a white paper surveying literature in computer science, law, and the social sciences on developments in machine learning and artificial intelligence, with special focus on their implications for consumer financial services in the United States. This project grew out of a joint collaboration between IMTFI and Capital One's Responsible AI Program. We present here an excerpt from the introduction to the white paper, followed by some updates about recent developments in this space. 

Excerpt
Machine learning (ML) algorithms and the artificial intelligence (AI) systems that they enable are powerful technologies that have inspired a lot of excitement, especially within large business and governmental organizations. In an era when increasingly concentrated computing power enables the creation, collection, and storage of “big data,” ML algorithms have the capacity to identify non-intuitive correlations in massive datasets, and as such can theoretically be more efficient and effective than humans at using those correlations to make accurate predictions. What is more, AI systems powered by ML algorithms represent a means of removing human prejudices from decision-making processes; since an AI system renders its decisions based solely on the data available, it can avoid the conscious and unconscious biases that often influence human decision-makers.

Contrary to this rosy picture of ML and AI, though, decades of evidence demonstrate how these technologies are not as objective and unbiased as many perhaps wish they were. Biases can be encoded in the datasets on which ML algorithms are trained, arising from poor sampling strategies, incomplete or erroneous information, and the social inequalities that exist in the actual world. And since ML algorithms and AI systems cannot build themselves, the humans who construct them may, however unintentionally, introduce their own biases when deciding on a model’s goals, selecting features, identifying which attributes are relevant, and developing classifiers. Additionally, the inherent complexities of ML algorithms that defy explanation even for the most expert practitioners can make it difficult, if not impossible, to identify the root causes of unfair decisions. That same opacity also presents an obstacle for individuals who believe that they have been evaluated unfairly, want to challenge a decision, or try to determine who should—or even ​could​—be held accountable for mistakes.

Compared to other fields, the financial services industry has taken a relatively conservative approach to ML/AI integrations. Consumer-facing applications like robo-advisors for portfolio management, AI-powered banking assistants, algorithmic trading programs, and proactive marketing tools, as well as harnessing the power of ML to do sentiment analysis of social media feeds and news stories in search of trendlines, have garnered a lot of media attention. However, the visibility of initiatives like these in press releases and news items exaggerates their role in financial services today, as they represent less than one-tenth of the funding received in the financial technology, or “fintech,” vendor space. Thus far, financial institutions have primarily invested in ML and AI for automating routine, back-office tasks, improving fraud detection and cybersecurity, and making regulatory compliance easier. 

The current state of ML and AI in consumer financial services, then, is one in which there is still enormous opportunity for innovation, but also reasons to be cautious. To paraphrase the feminist geographer Doreen Massey, some individuals and groups are more on the “receiving end” of these technologies than others. In other words, ML and AI’s advantages and disadvantages are not equally distributed. Nor are the vulnerabilities entailed by digital surveillance techniques for data creation and collection, the sorts of harm that can occur from an erroneous data entry and the burden for correcting it, or the ability to affect how an algorithm interprets one’s individual attributes and characteristics. In many ways, ML/AI research’s most important contributions have been demonstrating the extent to which structural inequalities—that is, conditions by which one or more groups of people are afforded unequal status and/or opportunities in comparison to other groups—persist by providing quantifiable, documented evidence of social disparities. If an organization’s reason for integrating ML- and AI-powered systems is to improve its decision-making procedures so as to make them both more accurate and fairer, then it is imperative to understand and account for persistent inequalities in the social contexts where those systems are embedded. Furthermore, assessing how exactly an algorithmic and/or automated decision-making system could impact specific populations, the risk that it could violate legal standards prohibiting discrimination, and the extent to which the system could perpetuate structural inequalities are of the utmost importance when deciding whether or not to make the integration in the first place.

You can read the rest of the white paper on SSRN.

Updates

Work in ML and AI is fast-moving, and in the time since this paper was published, there have been a number of developments that will affect how these technologies are integrated with the consumer financial services industry and beyond. Two in particular merit attention here:

1) Congressional action: On February 12, 2020, the U.S. House Committee on Financial Services' Task Force on Artificial Intelligence heard testimony from experts on AI, ML, and race and inclusion in a panel titled “Equitable Algorithms: Examining Ways to Reduce AI Bias in Financial Services.” The Committee acknowledged the usefulness of standards for the fairness and accuracy of AI applications in financial services, while also noting that existing laws such as the Equal Credit Opportunity Act, the Fair Housing Act, and the Fair Credit Reporting Act are inadequate in many respects for regulating AI's impact. The panel of experts recommended drafting a definition of "fairness" that could be used for evaluating ML, developing audit and assessment methods for locating biases in data and models, and requiring ML/AI developers to implement and report upon continuous monitoring plans that can detect new biases as they emerge. They also voiced concern regarding the Department of Housing and Urban Development's plans to revise the Fair Housing Act's disparate impact standards, and how such action might exacerbate the discriminatory effects of AI in home lending. 

2) Sandvig v. Barr decision: In March 2020, the U.S. District Court for the District of Columbia delivered its ruling in Sandvig v. Barr, which challenged a provision in the Computer Fraud and Abuse Act (CFAA) that made it a crime for researchers and journalists to use "dummy" accounts for the purposes of auditing algorithms in order to identify possible discrimination. The American Civil Liberties Union had initially brought the lawsuit in 2016 on behalf of a group of academics and journalists led by Christian Sandvig of University of Michigan's School of Information. The plaintiffs argued that the CFAA violated their First Amendment rights, and noted that comparable research activities were not illegal in offline contexts. The Court ruled in favor of the plaintiffs, thereby opening the door for more independent review of ML/AI applications and scoring an important victory for researchers' ability to hold algorithms and the institutions that use them accountable.

Additional Resources

AI Now Institute: https://ainowinstitute.org
Data and Society's AI on the Ground initiative: https://datasociety.net/research/ai-on-the-ground/

The Use of Money to Maintain Connection (and Toilet Paper)

Pandemic Insights
Bill Maurer in Anthropology News, American Anthropological Association(AAA)

When I received an email containing one of the first Covid-19 payment memes—“How do you wish to pay?”, with an image of Visa, MasterCard, and a roll of toilet paper—I thought, OK, there might be some interesting payment and money things I should pay attention to during this pandemic. Students and colleagues sent me lots more toilet-paper themed memes and made jokes: about scenes of barter, the transactors trying to determine an exchange rate; toilet paper on a golden platter; stores accepting toilet paper rolls for one cup of coffee or one scoop of ice cream (“must be new; one roll per visit”). A satirical (I think!) account of a churchgoer placing a roll of toilet paper on the collection plate. One Sunday afternoon my family ordered delivery. Every order over $40 came with a free roll of toilet paper.

In between all the new kinds of work involved in transitioning a large university to fully-online operations, I filed away the toilet paper stuff in a special email folder. Toilet paper economies.

But when the University of California, Irvine Graduate Division announced that it would henceforth accept scanned forms and digital signatures (it has long been a preserve and holdout of paper) and more than that, suspend all fee payment associated with those forms—which heretofore had to be paid with paper check—I thought, game on! This pandemic is changing payment.

Some establishments—my local ice creamery, for instance—have suspended all in-store payments and require customers to pay online for curbside pickups. Some places ask you to Venmo. CNN aired a segment on cash as a possible vector for viruses, and a company offering contactless tap-and-pay card readers has launched a marketing blitz. But cash is still there, too. In fact, cash demand is way, way up. It is at the restaurants offering take-out whose proprietors prefer the one-way transmission of cash to the back-and-forth passing of card or card terminal between customer and vendor. It is in the cash stashes people are starting to build up. A reporter from the Financial Times called me asking why people are stockpiling cash (not just toilet paper) when the cash distribution system is not likely to be disrupted the way it has been when hurricanes and floods knock out servers or the electricity grid.

Continue reading original post here:
https://www.anthropology-news.org/index.php/2020/04/09/the-use-of-money-to-maintain-connection-and-toilet-paper/

Because money makes the world go 'round

Bill Maurer, PhD, IMTFI Director, Dean, UCI School of Social Sciences explains money's many meanings, particularly amid the COVID-19 pandemic and times of crisis, in this piece for the Consortium of Social Science Associations (COSSA).

I often get asked questions like, “What is an anthropologist like you doing studying money? I thought that was the domain of economists!” The archaeological and ethnographic record is full of objects, texts, and records of promises humans have used for millennia to mark transactions with one another and figure value. It’s true that I enjoy working with and thinking about those objects, and among my favorite places are the money galleries in museums around the world and at the regional branches of the U.S. Federal Reserve. But the anthropology of money is more than an archive of the arcane. Understanding practices like bridewealth, involving objects like the tevau of the Santa Cruz Islands, can shed light on how contemporary money is far more than a neutral medium of exchange. This matters for product design, financial literacy programming, and macroeconomic policy, too.

Indeed, now that the world is in a global pandemic caused by the rapid spread of the COVID-19 virus, what people do with money and its technologies has acquired a new kind of significance. Although the virus apparently does not survive for long on fibrous materials like cloth and paper, reports have surged of Chinese and other officials ordering the disinfecting of banknotes to prevent its spread. The fintech industry conference organization Money 2020, promoting its (almost certainly to be cancelled) next event, proclaimed in an email that the pandemic would usher in the end of cash and the era of digital payments—despite the fact that most in-person digital payments (at your local take-out restaurant now, for example) rely on plastic and metal cards and point of sale devices, touched by many hands, on which the virus can survive for several hours. And in Kenya, the authorities are recommending all Kenyans use mobile phones to pay—about which, more below.

Economic uncertainty is leading many to horde cash, which may intensify a trend that had already been in motion given historically low interest rates—now even lower—and shifting behaviors related to digital payment. The Federal Reserve reports a steady increase in cash demand, despite a decline in cash transactions. Since the rich are putting their money in things like money market funds and CDs, however, this cannot be due to their behavior. With all the ways to pay via laptop, mobile phone or smart watch, why should the Fed need to supply banks with more and more banknotes? We found a clue in some recent research on young people’s use of budgeting apps like Mint. We set out to understand what financial literacy lessons—if any—such apps taught their users. But along the way we discovered that some of our subjects were withdrawing cash from the ATM on payday and stashing it away, because services like Venmo had made it so easy to spend the money in their bank account. Whereas, for me, the money in the bank is my store of value, for them, Venmo had turned bank money into current-use funds. Storing cash was a means of controlling spending. Cash is not dead or dying—its uses are just changing.


Read the full piece, courtesy of COSSA: https://www.whysocialscience.com/blog/2020/3/24/because-money-makes-the-world-go-round

Why Social Science? is a project of the Consortium of Social Science Associations, a 501(c)(6) non-profit organization based in Washington, DC. Our goal is to share the benefits and contributions of federally-funded social and behavioral science research with the public and encourage its widespread use for tackling challenges of national importance. To learn more about COSSA visit www.cossa.org. #whysocialscience

Virtual Currencies and the State - B. Maurer, Money at the Zero Lower Bound

Bill Maurer in Just Money's Roundtable 2: Virtual Currencies and the State

I picked up a copy of the Financial Times in the Munich airport on my way home from keynoting the Bundesbank’s biannual International Cash Conference. The lead article, headlined “Draghi calls for urgent spending as he relaunches stimulus,” reported that the European Central Bank had lowered interest rates deeper into negative territory, to -0.5%. In the opinion pages, anthropologist and regular columnist Gillian Tett observed that negative interest rates were constraining policy options to stimulate growth, which might compel central banks to coordinate more directly with fiscal policy makers—thereby lessening, if not abandoning, central bank independence. Lack of monetary policy options was  leading to a “changing zeitgeist,” she wrote.

At the Bundesbank conference, attended by people affiliated with the cash payments divisions of central banks and others, researchers presented data on the increase in cash demand despite the decline of cash transactions at the point of sale. People are increasingly paying with their mobile phone or cards, but at the same time, negative interest spotlights the cost of bank deposits, suddenly making cash a smarter option for savings. At the conference, lighthearted disagreements over whether to call this “cash hoarding” gave way to more insistent pleas for what some called “non-transactional” cash to be recognized as a rational response to negative interest with consequences for commercial banking and banknote design. If people are going to hoard cash, then perhaps banks need to get into the business of building vaults. And if people are going to want cash as a store of value resistant to negative interest, perhaps innovative banknote design should support hoarding: the cash should be more durable, stackable, maybe smaller than a standard banknote, and able to be kept in a cupboard and easily stashed in a backpack, should one need to escape a natural disaster, political instability, or war.

These were European designers, talking about European banknotes. This is a changing zeitgeist indeed.

Cash limits just how low interest rates can go, unless governments find a way to levy and enforce a tax on cash. Cash holdings are an alternative to paying the bank to hold your deposits—at least until the cost of storage, security and insurance approach the cost of paying negative interest. Hence: vaults. If for everyday transactions cash serves as a control mechanism for consumption (the pain of seeing your cash go away introduces a mental speed-bump in your spending), at the monetary policy level cash is a control mechanism defining a limit to the “innovative” monetary policies we have seen since the global financial crisis.

To read the full discussion, please visit https://justmoney.org/b-maurer-money-at-the-zero-lower-bound/.

New bank account, who dis?

IMTFI Director Bill Maurer talks about fintech apps in this Dope Labs podcast by Zakiya Whatley, Titi Shodiya & Jenny Radelet Mast

Interview Transcript Excerpt:

Zakiya: FinTech is all around us and it can feel like these apps have been around forever, but that's not exactly the case.

Titi: Yes, so our first question for Dr. Maurer was when did all of this stuff start popping up?

Dr. Maurer: This really kind of hit the scene in a big way in 2008 and 2009 with the launch of the iPhone.

Zakiya: Yes, the iPhone launch in the U.S. in 2007. But the iPhone 3G came on the scene in 2008.

Titi: Revolutionary.

Zakiya: Changed everything.

Dr. Maurer: All of a sudden people have this really cool device that fits in their pocket and is basically a terrific interface into a whole bunch of different applications are a whole kind of appecology grew up around the iPhone and similar devices. And people in Silicon Valley and in the banking industry and payments industry started realizing, hey, this device, this suite of devices and apps could really change people's relationship to their money into the existing financial and banking infrastructure.

Zakiya: I love that he describes this as ecology because we've talked about ecology on a couple of different episodes.

Titi: Yes.

Zakiya: And so the ecology is just how different things usually in biology we say how different organisms interact with one another. But now we're saying, how do these different apps talk to each other? How do people want to access the information now that they have a smartphone? So it's really cool to think about this in the context of more of a relationship.

Titi: Yes. So these apps have created their own special environment.

Zakiya: That's right. So the first category we're walking through is personal finance. These are apps like Mint, Credit Karma, You need a budget, and even your personal banking app. Generally, they help you manage your spending and saving.

Dr. Maurer: These apps basically serve almost as like training wheels. Right. So it's like training wheels on a bicycle that teaches you a little bit about budgeting and investing or savings. It's usually not enough to get you where you need to go, but it gives you basic skills and a basic vocabulary. So then you can start talking to other people, to friends, to your parents, or even walk into your credit union or bank and start asking better questions.

Titi: And these apps or services can be really good for helping people make smarter financial decisions. This help comes primarily in two ways. The first is by data visualization or giving people charts and graphs to show them what they're really doing with their money.

Dr. Maurer: For so many people. Money is just coming in and going out and they're not really being mindful about it or paying any attention to where things are going.


Listen to the full podcast and read full transcript to learn more about algorithmic bias, personal finance, and gender & credit. Links below.

Via podcast: https://megaphone.link/GLT5697101460 or online at https://www.dopelabspodcast.com/listen (Lab 021: New Bank Account, Who Dis?).

Full transcript:
https://static1.squarespace.com/static/5b38f40b7e3c3a94b302ba0d/t/5e347323e051ef5d568babaf/1580495652102/lab+021+transcript.pdf

For further reading, view Filene Research Report: "The Lessons of Fintech Apps: Design Matters for Personal Finance​"

Emotional Currency: How Money Shapes Human Relationships

IMTFI Director Bill Maurer interviewed on NPR's Hidden Brain, with Shankar Vedantam, Parth Shah, Tara Boyle, Jennifer Schmidt.

There's a story you may have heard before about what the world look like before money was invented. It's a story built on the idea of barter.

"It goes something like this: in the beginning, before there was money, if I had something that you needed, I would approach you with that thing and see if you had anything that I needed," says anthropologist Bill Maurer.

"The problem is that when we look around the world and in the historical and archaeological record for instances of this kind of direct barter, unfortunately we don't find it."

This week on Hidden Brain, we challenge established ideas about the origins of currency, and highlight the connection between money and relationships.

"Society is a thing of ongoing continuous relationships. The settling and unsettling of debts, on and on and on and on and on."

Listen to the interview podcast here: https://www.npr.org/2020/01/10/795246685/emotional-currency-how-money-shapes-human-relationships

The Lessons of Fintech Apps: Design Matters for Personal Finance​

by Bill Maurer, UCI; Melissa Wrapp, UCI; Chandra L. Middleton, UCI; Vivian Dzokoto, Virginia Commonwealth University; and Jenny Fan, UCI

This research report from Filene's Center of Excellence, the Center of Emerging Technology at UC Irvine provides lessons to credit unions that may help ​improve how their services are​ perceived and that may help ​activate deeper engagement ​with younger members.

Executive Summary
Fintech-based personal financial management (PFM) apps are providing new ways for budgeting and investing. In an effort to attract or retain Gen Z and Millennial members, financial institutions have been quick to adopt products similar to Mint or Acorns. In the race to provide these services, few financial institutions have reviewed and explored the value that such products provide, and how people interact with and feel about budgeting and investing with these new apps.

What Is the Research About?
Filene piloted a study that tested three PFM apps for budgeting (Mint, Wally, and You Need A Budget) and two for investing (Stash and Acorns). Twenty-seven participants between the ages of 18 and 34 used an assigned app for one month. Each participant was interviewed before and after using the app. Focus groups were held to provide additional insights.

We set out to learn three things from this study:

• How do people use these apps? And how does the design of the apps shape the user experience?
• What are the apps really teaching users? Are there conflicting messages to “buy” in budgeting apps? And how do users’ behaviors change as a result of using the apps?
• How effective are the apps at getting people to improve their financial behaviors?

Surprisingly, we learned that traditional financial institutions are still preferred over PFM apps. Although the apps have their benefits, people still want access to actual people in their financial institutions. In terms of design shaping the user experience, good design begets credibility. Poorly functioning or shoddy-looking apps eroded trust in the financial institution.

The apps tended to raise awareness of participants’ budgeting or investing behaviors rather than change behaviors. For some, using the apps reinforced preexisting frustration or stress participants felt about their financial lives. With investment apps, some participants felt that the gamification of the apps promoted addictive behaviors.

Read additional takeaways, download the full report, and view summary slides here: https://filene.org/487