Algorithm aversion
 | This article is written like a personal reflection, personal essay, or argumentative essay that states a Wikipedia editor's personal feelings or presents an original argument about a topic. (October 2023) |
Algorithm aversion is "biased assessment of an algorithm which manifests in negative behaviours and attitudes towards the algorithm compared to a human agent."[1] It describes a phenomenon where humans reject advice from an algorithm in a case where they would accept the same advice if they thought it was coming from another human.
Algorithms, such as those employing machine learning methods or various forms of artificial intelligence, are commonly used to provide recommendations or advice to human decisionmakers. For example, recommender systems are used in E-commerce to identify products a customer might like, and artificial intelligence is used in healthcare to assist with diagnosis and treatment decisions. However, humans sometimes appear to resist or reject these algorithmic recommendations more than if the recommendation had come from a human. Notably, algorithms are often capable of outperforming humans, so rejecting algorithmic advice can result in poor performance or suboptimal outcomes.
This is an emerging topic and it is not completely clear why or under what circumstances people will display algorithm aversion. In some cases, people seem to be more likely to take recommendations from an algorithm than from a human, a phenomenon called algorithm appreciation.[2]
Examples of algorithm aversion
[edit]Algorithm aversion has been studied in a wide variety of contexts. For example, people seem to prefer recommendations for jokes from a human rather than from an algorithm,[3] and would rather rely on a human to predict the number of airline passengers from each US state instead of an algorithm.[4] People also seem to prefer medical recommendations from human doctors instead of an algorithm.[5]
Factors affecting algorithm aversion
[edit]Various frameworks have been proposed to explain the causes for algorithm aversion and techniques or system features that might help reduce aversion.[1][6][7]
Decision control
[edit]Algorithms may either be used in an advisory role (providing advice to a human who will make the final decision) or in a delegatory role (where the algorithm makes a decision without human supervision).[8] A movie recommendation system providing a list of suggestions would be in an advisory role, whereas the human driver delegates the task of steering the car to Tesla's Autopilot. Generally, a lack of decision control tends to increase algorithm aversion.[citation needed]
Perceptions about algorithm capabilities and performance
[edit]Overall, people tend to judge machines more critically than they do humans.[9] Several system characteristics or factors have been shown to influence how people evaluate algorithms.
Algorithm Process and the role of system transparency
[edit]One reason people display resistance to algorithms is a lack of understanding about how the algorithm is arriving at its recommendation.[3] People also seem to have a better intuition for how another human would make recommendations. Whereas people assume that other humans will account for unique differences between situations, they sometimes perceive algorithms as incapable of considering individual differences and resist the algorithms accordingly.[10]
Decision domain
[edit]People are generally skeptical that algorithms can make accurate predictions in certain areas, particularly if task involves a seemingly human characteristic like morals or empathy. Algorithm aversion tends to be higher when the task is more subjective and lower on tasks that are objective or quantifiable.[1]
Human characteristics
[edit]Domain expertise
[edit]Expertise in a particular field has been shown to increase algorithm aversion[2][11] and reduce use of algorithmic decision rules.[12] Overconfidence may partially explain this effect; experts might feel that an algorithm is not capable of the types of judgments they make. Compared to non-experts, experts also have more knowledge of the field and therefore may be more critical of a recommendation. Where a non-expert might accept a recommendation ("The algorithm must know something I don't.") the expert might find specific fault with the algorithm's recommendation ("This recommendation does not account for a particular factor").
Decision-making research has shown that experts in a given field tend to think about decisions differently than a non-expert.[13] Experts chunk and group information; for example, chess grandmasters will see opening positions (e.g., the Queen's Gambit or the Bishop's Opening) instead of individual pieces on the board. Experts may see a situation as a functional representation (e.g., a doctor could see a trajectory and predicted outcome for a patient instead of a list of medications and symptoms). These differences may also partly account for the increased algorithm aversion seen in experts.
Culture
[edit]Different cultural norms and influences may cause people to respond to algorithmic recommendations differently. The way that recommendations are presented (e.g., language, tone, etc.) may cause people to respond differently.[citation needed]
Age
[edit]Digital natives are younger and have known technology their whole lives, while digital immigrants have not. Age is a commonly-cited factor hypothesized to affect whether or not people accept algorithmic recommendations. For example, one study found that trust in an algorithmic financial advisor was lower among older people compared with younger study participants.[14] However, other research has found that algorithm aversion does not vary with age.[2]
Proposed methods to overcome algorithm aversion
[edit]Algorithms are often capable of outperforming humans or performing tasks much more cost-effectively.[4][3]
Human-in-the-loop
[edit]One way to reduce algorithmic aversion is to provide the human decision maker with control over the final decision.
System transparency
[edit]Providing explanations about how algorithms work has been shown to reduce aversion. These explanations can take a variety of forms, including about how the algorithm as a whole works, about why it is making a particular recommendation in a specific case, or how confident it is in its recommendation.[1]
User training
[edit]Algorithmic recommendations represent a new type of information in many fields. For example, a medical AI diagnosis of a bacterial infection is different than a lab test indicating the presence of a bacteria. When decision makers are faced with a task for the first time, they may be especially hesitant to use an algorithm. It has been shown that learning effects achieved through repeated tasks, constant feedback and financial incentives can contribute towards reducing algorithm aversion.[15]
Algorithm appreciation
[edit]Studies do not consistently show people demonstrating bias against algorithms and sometimes show the opposite, preferring advice from an algorithm instead of a human. This effect is called algorithm appreciation.[2][16] Results are mixed, showing that people sometimes seem to prefer advice that comes from an algorithm instead of a human.
For example, customers are more likely to indicate initial interest to human sales agents compared to automated sales agents but less likely to provide contact information to them. This is due to "lower levels of performance expectancy and effort expectancy associated with human sales agents versus automated sales agents".[17]
References
[edit]- ^ a b c d Jussupow, Ekaterina; Benbasat, Izak; Heinzl, Armin (2020). "Why Are We Averse Towards Algorithms ? A Comprehensive Literature Review on Algorithm Aversion". Twenty-Eighth European Conference on Information Systems (ECIS2020): 1–16.
- ^ a b c d Logg, Jennifer M.; Minson, Julia A.; Moore, Don A. (2019-03-01). "Algorithm appreciation: People prefer algorithmic to human judgment". Organizational Behavior and Human Decision Processes. 151: 90–103. doi:10.1016/j.obhdp.2018.12.005. ISSN 0749-5978.
- ^ a b c Yeomans, Michael; Shah, Anuj; Mullainathan, Sendhil; Kleinberg, Jon (2019). "Making sense of recommendations". Journal of Behavioral Decision Making. 32 (4): 403–414. doi:10.1002/bdm.2118. ISSN 1099-0771. S2CID 141066682.
- ^ a b Dietvorst, Berkeley J.; Simmons, Joseph P.; Massey, Cade (2015). "Algorithm aversion: People erroneously avoid algorithms after seeing them err". Journal of Experimental Psychology: General. 144 (1): 114–126. doi:10.1037/xge0000033. ISSN 1939-2222. PMID 25401381.
- ^ Cabitza, Federico (2019). "Biases Affecting Human Decision Making in AI-Supported Second Opinion Settings". MDAI 2019 - International Conference on Modeling Decisions for Artificial Intelligence. Lecture Notes in Computer Science. Vol. 11676. pp. 283–294. doi:10.1007/978-3-030-26773-5_25. ISBN 978-3-030-26773-5. S2CID 201739274.
- ^ Burton, Jason W.; Stein, Mari-Klara; Jensen, Tina Blegind (2020). "A systematic review of algorithm aversion in augmented decision making". Journal of Behavioral Decision Making. 33 (2): 220–239. doi:10.1002/bdm.2155. ISSN 1099-0771. S2CID 210439660.
- ^ Mahmud, H.; Islam, A.N.; Ahmed, S.I.; Smolander, K. (2022). "What influences algorithmic decision-making? A systematic literature review on algorithm aversion". Technological Forecasting and Social Change. 175: 121390. doi:10.1016/j.techfore.2021.121390. ISSN 0040-1625.
- ^ Mahmud, H.; Islam, A.N.; Mitra, R.K. (2023). "What drives managers towards algorithm aversion and how to overcome it? Mitigating the impact of innovation resistance through technology readiness". Technological Forecasting and Social Change. 193: 122641. doi:10.1016/j.techfore.2023.122641. ISSN 0040-1625.
- ^ Hidalgo, Cesar (2021). How Humans Judge Machines. Cambridge, MA: MIT Press. ISBN 978-0-262-04552-0.
- ^ Longoni, Chiara; Bonezzi, Andrea; Morewedge, Carey K (2019-05-03). "Resistance to Medical Artificial Intelligence". Journal of Consumer Research. 46 (4): 629–650. doi:10.1093/jcr/ucz013. ISSN 0093-5301.
- ^ Mahmud, H.; Islam, A.N.; Luo, X.R.; Mikalef, P. (2024). "Decoding algorithm appreciation: Unveiling the impact of familiarity with algorithms, tasks, and algorithm performance". Decision Support Systems. 179: 114168. doi:10.1016/j.dss.2024.114168. ISSN 0167-9236.
- ^ Arkes, Hal R.; Dawes, Robyn M.; Christensen, Caryn (1986-02-01). "Factors influencing the use of a decision rule in a probabilistic task". Organizational Behavior and Human Decision Processes. 37 (1): 93–110. doi:10.1016/0749-5978(86)90046-4. ISSN 0749-5978.
- ^ Feltovich, Paul J.; Prietula, Michael J.; Ericsson, K. Anders (2006), Ericsson, K. Anders; Charness, Neil; Feltovich, Paul J.; Hoffman, Robert R. (eds.), "Studies of Expertise from Psychological Perspectives", The Cambridge Handbook of Expertise and Expert Performance, Cambridge Handbooks in Psychology, Cambridge: Cambridge University Press, pp. 41–68, doi:10.1017/cbo9780511816796.004, ISBN 978-1-107-81097-6, retrieved 2021-09-08
- ^ Lourenço, Carlos J.S.; Dellaert, Benedict G.C.; Donkers, Bas (2020-02-01). "Whose Algorithm Says So: The Relationships Between Type of Firm, Perceptions of Trust and Expertise, and the Acceptance of Financial Robo-Advice". Journal of Interactive Marketing. 49: 107–124. doi:10.1016/j.intmar.2019.10.003. hdl:1765/123799. ISSN 1094-9968. S2CID 211029562.
- ^ Filiz, Ibrahim; Judek, Jan René; Lorenz, Marco; Spiwoks, Markus (2021-09-01). "Reducing algorithm aversion through experience". Journal of Behavioral and Experimental Finance. 31: 100524. doi:10.1016/j.jbef.2021.100524. ISSN 2214-6350.
- ^ Mahmud, H.; Islam, A.N.; Luo, X.R.; Mikalef, P. (2024). "Decoding algorithm appreciation: Unveiling the impact of familiarity with algorithms, tasks, and algorithm performance". Decision Support Systems. 179: 114168. doi:10.1016/j.dss.2024.114168. ISSN 0167-9236.
- ^ Adam, Martin; Roethke, Konstantin; Benlian, Alexander (2022-11-10). "Human Versus Automated Sales Agents: How and Why Customer Responses Shift Across Sales Stages". Information Systems Research. 34 (3): 1148–1168. doi:10.1287/isre.2022.1171. ISSN 1047-7047. S2CID 254271045.