A Survey of Fair and Responsible Machine Learning and Artificial Intelligence: Implications for Consumer Financial Services

by Stephen C. Rea, PhD, Research Assistant Professor at Colorado School of Mines and former IMTFI Research Assistant

Capital One Eno Chatbot

Stephen Rea recently published a white paper surveying literature in computer science, law, and the social sciences on developments in machine learning and artificial intelligence, with special focus on their implications for consumer financial services in the United States. This project grew out of a joint collaboration between IMTFI and Capital One's Responsible AI Program. We present here an excerpt from the introduction to the white paper, followed by some updates about recent developments in this space. 

Excerpt
Machine learning (ML) algorithms and the artificial intelligence (AI) systems that they enable are powerful technologies that have inspired a lot of excitement, especially within large business and governmental organizations. In an era when increasingly concentrated computing power enables the creation, collection, and storage of “big data,” ML algorithms have the capacity to identify non-intuitive correlations in massive datasets, and as such can theoretically be more efficient and effective than humans at using those correlations to make accurate predictions. What is more, AI systems powered by ML algorithms represent a means of removing human prejudices from decision-making processes; since an AI system renders its decisions based solely on the data available, it can avoid the conscious and unconscious biases that often influence human decision-makers.

Contrary to this rosy picture of ML and AI, though, decades of evidence demonstrate how these technologies are not as objective and unbiased as many perhaps wish they were. Biases can be encoded in the datasets on which ML algorithms are trained, arising from poor sampling strategies, incomplete or erroneous information, and the social inequalities that exist in the actual world. And since ML algorithms and AI systems cannot build themselves, the humans who construct them may, however unintentionally, introduce their own biases when deciding on a model’s goals, selecting features, identifying which attributes are relevant, and developing classifiers. Additionally, the inherent complexities of ML algorithms that defy explanation even for the most expert practitioners can make it difficult, if not impossible, to identify the root causes of unfair decisions. That same opacity also presents an obstacle for individuals who believe that they have been evaluated unfairly, want to challenge a decision, or try to determine who should—or even ​could​—be held accountable for mistakes.

Compared to other fields, the financial services industry has taken a relatively conservative approach to ML/AI integrations. Consumer-facing applications like robo-advisors for portfolio management, AI-powered banking assistants, algorithmic trading programs, and proactive marketing tools, as well as harnessing the power of ML to do sentiment analysis of social media feeds and news stories in search of trendlines, have garnered a lot of media attention. However, the visibility of initiatives like these in press releases and news items exaggerates their role in financial services today, as they represent less than one-tenth of the funding received in the financial technology, or “fintech,” vendor space. Thus far, financial institutions have primarily invested in ML and AI for automating routine, back-office tasks, improving fraud detection and cybersecurity, and making regulatory compliance easier. 

The current state of ML and AI in consumer financial services, then, is one in which there is still enormous opportunity for innovation, but also reasons to be cautious. To paraphrase the feminist geographer Doreen Massey, some individuals and groups are more on the “receiving end” of these technologies than others. In other words, ML and AI’s advantages and disadvantages are not equally distributed. Nor are the vulnerabilities entailed by digital surveillance techniques for data creation and collection, the sorts of harm that can occur from an erroneous data entry and the burden for correcting it, or the ability to affect how an algorithm interprets one’s individual attributes and characteristics. In many ways, ML/AI research’s most important contributions have been demonstrating the extent to which structural inequalities—that is, conditions by which one or more groups of people are afforded unequal status and/or opportunities in comparison to other groups—persist by providing quantifiable, documented evidence of social disparities. If an organization’s reason for integrating ML- and AI-powered systems is to improve its decision-making procedures so as to make them both more accurate and fairer, then it is imperative to understand and account for persistent inequalities in the social contexts where those systems are embedded. Furthermore, assessing how exactly an algorithmic and/or automated decision-making system could impact specific populations, the risk that it could violate legal standards prohibiting discrimination, and the extent to which the system could perpetuate structural inequalities are of the utmost importance when deciding whether or not to make the integration in the first place.

You can read the rest of the white paper on SSRN.

Updates

Work in ML and AI is fast-moving, and in the time since this paper was published, there have been a number of developments that will affect how these technologies are integrated with the consumer financial services industry and beyond. Two in particular merit attention here:

1) Congressional action: On February 12, 2020, the U.S. House Committee on Financial Services' Task Force on Artificial Intelligence heard testimony from experts on AI, ML, and race and inclusion in a panel titled “Equitable Algorithms: Examining Ways to Reduce AI Bias in Financial Services.” The Committee acknowledged the usefulness of standards for the fairness and accuracy of AI applications in financial services, while also noting that existing laws such as the Equal Credit Opportunity Act, the Fair Housing Act, and the Fair Credit Reporting Act are inadequate in many respects for regulating AI's impact. The panel of experts recommended drafting a definition of "fairness" that could be used for evaluating ML, developing audit and assessment methods for locating biases in data and models, and requiring ML/AI developers to implement and report upon continuous monitoring plans that can detect new biases as they emerge. They also voiced concern regarding the Department of Housing and Urban Development's plans to revise the Fair Housing Act's disparate impact standards, and how such action might exacerbate the discriminatory effects of AI in home lending. 

2) Sandvig v. Barr decision: In March 2020, the U.S. District Court for the District of Columbia delivered its ruling in Sandvig v. Barr, which challenged a provision in the Computer Fraud and Abuse Act (CFAA) that made it a crime for researchers and journalists to use "dummy" accounts for the purposes of auditing algorithms in order to identify possible discrimination. The American Civil Liberties Union had initially brought the lawsuit in 2016 on behalf of a group of academics and journalists led by Christian Sandvig of University of Michigan's School of Information. The plaintiffs argued that the CFAA violated their First Amendment rights, and noted that comparable research activities were not illegal in offline contexts. The Court ruled in favor of the plaintiffs, thereby opening the door for more independent review of ML/AI applications and scoring an important victory for researchers' ability to hold algorithms and the institutions that use them accountable.

Additional Resources

AI Now Institute: https://ainowinstitute.org
Data and Society's AI on the Ground initiative: https://datasociety.net/research/ai-on-the-ground/

Barriers to a single European payments market: Cultural-economic feedback loops

PERSPECTIVES By Erin B. Taylor, Canela Consulting, former IMTFI Fellow and co-creator of the IMTFI Consumer Finance Research Methods Toolkit

Look into the average traveller’s pockets today and you will find evidence of multiple means of payment. Debit cards, credit cards, traveller’s checks, several currencies, cryptocurrencies, and payment apps are now so common that it seems impossible to run out of ways to pay. Wherever we buy things—on the street, in shops, restaurants, at ticket machines—we have a way to pay. 

As cash falls out of favour, foreigners must switch between different debit 

and credit cards in order to pay. Photo By Erin B. Taylor.

Or so it would seem. In fact, as many travelers can attest, it is still possible to run out of ways to pay. 

Let me give an example. One fine winter’s day in early January 2016, I stopped at a kiosk at the University of Amsterdam to buy a coffee. It was the beginning of my six-month stint as a visiting academic, and the environment was brand new to me. 

I handed the teller some cash to pay for my coffee and croissant, and she looked at me in surprise: “We only accept PIN,” she said. She meant that the kiosk exclusively accepted payment via a Dutch debit card: no cash, no foreign cards—not even European ones. 

I was astonished. Not accepting foreign cards is bizarre enough, but who doesn’t accept cash? As it turns out, a growing number of retailers in northwestern Europe are turning away from hard currency, citing cost and safety reasons. Some stores don’t accept cash, but they accept virtually all foreign cards (debit and credit). Others accept cash and local debit cards, but not foreign cards. And a minority (like my kiosk) exclusively accept local debit cards. 

The unsuspecting traveller may encounter inconveniences not only when trying to pay in the odd kiosk, restaurant, or shop, but also when simply trying to get from A to B. In the Netherlands, an unusually cash-averse society, some parking meters and train ticket machines only accept Dutch cards, and many a traveller has been caught out trying to return to the airport but unable to pay for the fare. Even the simple act of making a meal can involve a complicated series of transactions (see text box at the end of this post, "A Recipe in pan European Payments"). 

This is not just a Dutch peculiarity: payments are a Europe-wide problem. The European common market is meant to deliver the “four ‘f’s”: freedom of movement in people, goods, services, and capital. Theoretically, this should endow people with far more choice as consumers, workers, and citizens. 

Yet despite decades of financial market integration, many consumer finance products and services cannot be readily used across national borders within Europe. This situation could worsen when Brexit is implemented. A diversity of financial systems and a willingness to experiment means that the consumer can never be quite sure what to expect when crossing national borders. Consumers who live, work, and socialize across Europe’s borders can encounter problems using a wide range of finance products and services (e.g., payments, mortgages, taxes, and pensions). Why is this the case?

Some ticket machines in the Netherlands only accept cash or
Dutch debit cards. Photo By Spoorjan (Own work)  CC BY-SA 3.0 

Barriers to integration

One major problem is that the process of financial integration is far from complete. Generally, this integration process is conceptualized as being primarily technological and regulatory. The Single European Payments Area (SEPA) has been largely rolled out across the continent, and the Target Instant Payment Settlement (TIPS)  service promises to abolish waiting times for transfers between European banks. European regulators are working to create legal solutions, such as developing Europe-wide pension schemes, and the Payment Services Directive 2 (PSD2) is due to be implemented next year, further deregulating payments and opening up the market to new players and products. 

However, there are also barriers to integration at the level of the firm and the consumer market interactions, and our understanding of these is threadbare. Some of these relate to market structures, such as pricing. For example, in some European countries, credit cards are not widely accepted because merchants consider the cost to be prohibitive. Other barriers have socio-cultural leanings, such as consumers’ preference for local services, which dissuades them from shopping around the EU, or a preference for using cash in Germany.

These barriers might appear to be either economic or cultural, but closer inspection often shows them to be both. Let me illustrate by way of an example. In an ECB Report, Kokola argues that the Dutch tend to be more averse to credit card debt than their neighbors, whereas Germans are more risk-averse. This kind of cultural heterogeneity influences how financial products and services are developed, marketed, and consumed. 

Such cultural predilections can have deep historic roots. In the Netherlands, there is a longstanding aversion to credit due to historical attitudes towards indebtedness, but bank cards were adopted early on. Because the Dutch are averse to credit, but used to debit cards, credit transactions are relatively rare compared with other countries. And because the Dutch don’t use credit cards much, the cost of credit card transactions remains expensive. Because they’re expensive, merchants don’t accept credit cards, and this reinforces the Dutch aversion to them. 

And so a cultural-economic feedback loop is created.

This lines up with what we know about the interplay between economy and culture globally. Social researchers have long observed that economy and culture are analytically inseparable, no matter what kind of economy people live in. This is easiest to observe in pre-capitalist societies, such as in the use of shell money in Melanesia. 

But economy and culture are intertwined everywhere. In Dreaming of Money in Ho Chi Minh City (2014), Allison J. Truitt discusses how money culture influences what banknotes people will accept (dirty or broken notes are rejected), how money is used for ritual purposes, and many more phenomena that cross the culture/economy divide. In Liquidated: An Ethnography of Wall Street (2009), Karen Ho describes how the decisions of investment bankers are  shaped by their sociocultural beliefs. Nobody, anywhere, is immune.  

The diversity of cultural-economic feedback loops has significant implications for the integration of consumer finance markets in Europe. It suggests that there are hard limits to what can be achieved through technological and regulatory means alone. As Sander, Kleimeier & Heuchemer note, “cultural distance limits international financial integration over and above what can be expected from economic trade and transaction costs.” Even if full integration is achieved, consumers will continue to face limits to their freedom of choice as they live, work, and socialize across European borders.

To understand why there is still no single market for financial services in Europe, it is not enough to look at technical or regulatory matters. But nor can we simply shift the blame to culture. Rather, a cultural-economic feedback loop comes into existence when an economic practice and a cultural practice reinforce each other’s existence. 

The standard EFTPOS machine is fast
being replaced by other POS devices.
Photo By Erin B. Taylor.

The EU’s problem is global

The globalization of payments and other financial services is also creating an imperative to figure out what happens when money cultures meet. Given that so many consumer finance products and services are now available over the Internet, consumers are no longer limited to what is available in their home town or country. Today, we can research and buy an increasingly wide range of savings, transfer, investment, credit, and money management services from anywhere around the world. 

Let’s stop for a moment to consider the irony here. A resident of one European country cannot use their bank card in a second European country, even though there is a single currency and theoretically an integrated payments system. But that same person can buy travel insurance from the U.S.A., invest money in a fund in India, exchange currency using a mobile app based in the United Kingdom, and trade cryptocurrency based in—well, anywhere really.

The problem we face is twofold. First, the integration of financial markets globally is proceeding at different rates in different places. This means that consumers are facing a rapid expansion of choice on the one hand, and the same old limitations on the other. (In fact, these limitations are becoming more problematic because people are more mobile across borders than they were before, and so they encounter these problems more often.) Regulators and financial services providers are over-providing services in some areas, and under-providing them in others. Corporate and government strategies for integrating financial markets need to find a balance between these extremes. 

Second, we have little idea what consumers do when faced with this strange situation. How do consumers work around obstacles to making financial transactions? Do any of the new products and services available globally fill gaps in local services? Why are some people willing to experiment and become “early adopters” of new digital finance products and services, while others remain “laggards” dependent upon traditional banks? And what will a more mature global market for financial goods and services look like in the future? 

Since consumers can now use financial services from around the world, we cannot assume that it is sufficient to approach any of these questions from a local or European angle. In the future, consumers are likely to care less and less about whether the financial services they use are local or not. This is particularly the case when brands that are already globally popular (such as Google, Apple, or PayPal) develop their own range of payments solutions, such as digital wallets. 

A Dutch ATM, fast becoming a rare commodity. 
Photo by Canadian Pacific CC BY-NC 2.0

Mixing methods to understand changing markets

Our challenge is not to get everyone using exactly the same tools, but to create a global ecosystem in which multiple tools and avenues are accepted. To do this, we need to first understand the market. This means we need to design research that investigates how a variety of factors–cultural, economic, regulatory, technical–shape market practices. This holds even if we are trying to specifically understand consumer behaviour. 

Due to the complexity of markets, relying on one single research method (e.g., a survey or interviews) is unlikely to be sufficient for many research questions. Just as financial markets for consumer services are diversifying, so must our research methods also diversify. Understanding consumer choices requires analysis of both qualitative and quantitative factors that influence behaviour, including price, market structures, personal preferences, social structures, and cultural norms. 

This is not news: product developers, designers, and marketers know well that in order to sell something, the offering must hit the right price point and the right “tone” with the consumer. But the shift to Internet-based and mobile consumer finance services presents a challenge because the transition is incomplete and the market is highly complex. 

While little can be done to predict how regulations will change, it is certainly possible to improve our understanding of changing consumer behaviour and thereby generate more robust market knowledge. As we discuss in the Consumer Finance Research Methods Toolkit (CFRM Toolkit), researchers from both industry and academia are innovating new ways to record and analyze the financial behaviours of individuals and households. 

Ethnography, interview methods, financial diaries, online/offline studies, experiments, and so on, are all being reconfigured and combined with other methods to account for the increasing mobility products and services through accessible digital spaces and technologies. Adapting and combining methods offers substantial potential to generate detailed data on a variety of cultural and economic problems. This is because they either include ways to collect qualitative and quantitative data simultaneously, or because they can be easily incorporated into mixed-methods research. 

Combining interdisciplinary thinking with mixed methods gives us a chance to understand the cultural/economic feedback loops that are shaping the emergence of a new generation of consumer financial practices and markets, not only in Europe, but around the world. Regulators, service providers, and researchers are best placed when they take this range of factors and geographies into account. 

Would you pay more for soap when purchasing with mobile money?

Jean Lee, Jonathan Morduch, Abu Shonchoy

Imagine that someone approached you and asked how much you would be willing to pay for a bar of soap or a bag of potato chips? It seems like a simple question.

You would probably, though, ask which bar of soap and which potato chips? Think a bit longer, and you might be asking “pay for them how?”

Researchers have found that the last question – about the form of payment -- matters. For example, paying by credit card rather than cash changes how consumers spend: Studies suggest that using plastic induces consumers to pay higher tips at restaurants, buy more junk food, and pay more for a chance to see a pro basketball game. These results are not always robust, and studies struggle to separate the liquidity effect of credit cards from the psychological effect of using plastic vs. cold, hard cash. Still, the weight of the evidence suggests that people spend more when using credit cards (or even when thinking about credit cards) for reasons that are at least partly psychological.

The digital financial revolution prompts us to update the question: As mobile money widens in use, will it also influence spending choices in the way that plastic has? Or is digital in fact different?

In 2014, we started a project on the impacts of mobile money in Bangladesh. The study focuses on users of mobile money in northern villages and Dhaka neighborhoods. Mobile money has spread extremely fast in Bangladesh, largely due to the growth of bKash and its competitors. The mobile money sector is one of Bangladesh’s great recent economic success stories, and bKash alone now provides mobile money services to over 20 million customers.

Midway through the study, with financial support from IMTFI, we asked two randomly-chosen groups of people questions about their willingness to pay for household basics and some small luxuries. We asked the study participants how much they would be willing to pay for a quantity of fine rice, a good bar of soap, particular pieces of clothing (a salwar kameez and a lungi), a bag of potato chips, and a packet of biscuits (cookies). We asked the participants to respond (hypothetically) in contexts when using cash or mobile money.

Given the set-up and the fact that the questions were hypothetical, we did not expect to see much difference. But, as the table shows (which is from our urban sample), in 5 of the 6 cases respondents indicated that they would be willing to pay a higher price when using mobile money to facilitate the transaction.

Summary Statistics for Willingness to Pay (WTP) in Taka

Variable	Cash Mean	Mobile Money Mean	Cash Median	Mobile Money Median
WTP for rice	392	403	400	400
WTP for Beauty Soap	89	75	30	30
WTP for Salwar Kameez	701	750	700	700
WTP for Lungi	330	346	300	350
WTP for Potato Chips	39	41	30	30
WTP for Biscuits	74	79	50	60

To dig deeper, we ran a set of regressions to control for the respondent’s age, education, income, work status and other key variables. The regressions again show that in 5 of the 6 cases respondents indicated that they would be willing to pay a higher price when using mobile money to facilitate the transaction. (The negative signs mean that they would not be willing to pay as much when using cash.) The standard errors are fairly wide, however, and only in three of the cases are the differences statistically significant with 95 percent confidence.

Regression Results for Willingness to Pay, With Controls

	(1)	(2)	(3)	(4)	(5)	(6)
	Rice	Beauty Soap	Salwar Kameez	Lungi	Potato Chips	Biscuits
Cash	-11.1**	9.9	-49.0**	-14.9**	-1.8	-3.6
Treatment	(5.4)	(8.0)	(24.4)	(6.8)	(2.8)	(4.0)
Observations	812	813	812	812	811	811

Standard errors in parentheses^* p < 0.10, ^** p < 0.05, ^*** p < 0.01

Since the same participants answered questions about each of the 6 items, there is little concern that a given respondent was considering different qualities of items in the two scenarios. Like much of the earlier literature, however, we cannot distinguish the liquidity effect from psychological effects.

Half a year before, we had introduced mobile money to the first group. Our research team had trained members how to use mobile money, and many had started using it. The second group was an experimental control, and we provided them with no training nor discussion of mobile money. Our initial results suggest that the training and exposure to bKash (and the greater likelihood of its subsequent use) strongly narrowed the difference in willingness to pay between cash and mobile money. The main differences in spending patterns with cash versus mobile money thus come from the control group, and it is possible that their preferences will narrow too with greater exposure to mobile money.

We are now analyzing the rural sample, and the initial results are opposite to the urban sample: In the village, there is greater willingness to pay in cash. Our next step is to investigate why, including whether the result reflects a lack of stores that accept digital payment in the villages (rather than a hypothetical willingness to pay), or whether the result stems from unfamiliarity with mobile banking.

Economists generally assume that money is fungible, a dollar is a dollar, a taka is a taka. However, in both our urban and rural samples, the form of payment clearly makes a difference. There does seem to be something different about holding 20-taka on your mobile phone rather than holding a 20-taka banknote in your hand. In Dhaka, being able to pay by phone appears to raise the price that customers are willing to pay for household goods. If these results stand, retailers may now have another reason to encourage their customers to use mobile money.

Read their Final Report

Jean Lee is an economist at the Millennium Challenge Corporation and was a Post-Doctoral Research Fellow at NYU. Jonathan Morduch is Professor of Public Policy and Economics at the Robert Wagner Graduate School of Public Service at NYU. Abu Shonchoy is Research Fellow at the Institute of Developing Economies (IDE-JETRO) in Chiba, Japan and a Visiting Scholar at NYU (2016-18).