Sep 16, 2021 — In the five years since we released the first AI100 report, much has been written about the state of artificial.

257 KB – 82 Pages

PAGE – 1 ============
1Gathering Strength, Gathering Storms: In the ˜ve years since we released the ˜rst AI100 report, much has been written about the state of arti˜cial intelligence and its in˚uences on society. Nonetheless, AI100 remains unique in its combination of two key features. First, it is written by a Study Panel of core multi-disciplinary researchers in the ˜eldŠexperts who create arti˜cial intelligence algorithms or study their in˚uence on society as their main professional activity, and who have been doing so for many years. The authors are ˜rmly rooted within the ˜eld of AI and provide an fiinsider™sfl perspective. Second, it is a longitudinal study, with reports by such Study Panels planned once every ˜ve years, for at least one hundred years. SEPTEMBER 2021 This report, the second in that planned series of studies, is being released ˜ve years after the ˜rst report. Published on September 1, 2016, the ˜rst report was covered widely in the popular press and is known to have in˚uenced discussions on governmental advisory boards and workshops in multiple countries. It has also been used in a variety of arti˜cial intelligence curricula. In preparation for the second Study Panel, the Standing Committee commissioned two study-workshops held in 2019. These workshops were a response to feedback on the ˜rst AI100 report. Through them, the Standing Committee aimed to engage a broader, multidisciplinary community of scholars and stakeholders The One Hundred Year Study on Arti˜cial Intelligence (AI100) 2021 Study Panel Report PREFACE

PAGE – 2 ============
2in its next study. The goal of the workshops was to draw on the expertise of computer scientists and engineers, scholars in the social sciences and humanities (including anthropologists, economists, historians, media scholars, philosophers, psychologists, and sociologists), law and public policy experts, and representatives from business management as well as the private and public sectors. An expanded Standing Committee, with more expertise in ethics and the social sciences, formulated a call and actively encouraged proposals from the international community of AI researchers and practitioners with a broad representation of ˜elds relevant to AI™s impact in the world. By convening scholars from the full range of disciplines that rigorously explore ethical and societal impacts of technologies, the study-workshops were aimed at expanding and deepening discussions of the ways in which AI shapes the hopes, concerns, and realities of people™s lives, and, relatedly, the ethical and societal-impact challenges that AI raises. After circulating a call for proposals and reviewing more than 100 submissions from around the world, two workshops were selected for funding. One, on fiPrediction in Practice,fl studied the use of AI-driven predictions of human behavior, such as how likely a borrower is to eventually repay a loan, in settings where data and cognitive modeling fail to account for the social dimensions that shape people™s decision-making. The other, on fiCoding Caring,fl studied the challenges and opportunities of incorporating AI technologies into the process of humans caring for one another and the role that gender and labor relationships play in addressing the pressing need for innovation in healthcare. Drawing on the ˜ndings from these study-workshops, as well as the annual AI Index report , a project spun off from AI100, the Standing Committee de˜ned a charge for the Study Panel in the summer of 2019[1] and recruited Professor Michael Littman, Professor of Computer Science at Brown University, to chair the panel. The 17-member Study Panel, composed of a diverse set of experts in AI, from academia and industry research laboratories, representing computer science, engineering, law, political science, policy, sociology, and economics, was launched in mid-fall 2020. In addition to representing a range of scholarly specialties, the panel had diverse representation in terms of home geographic regions, genders, and career stages. As readers may note in the report, convening this diverse, interdisciplinary set of scholarly experts, allowed varying perspectives, rarely brought together, to be reconciled and juxtaposed within the report. The accomplishment of the Study Panel is that much more impressive considering the inability to meet face-to-face during the ongoing COVID-19 global pandemic. Whereas the ˜rst study report focused explicitly on the impact of AI in North American cities, we sought for the 2021 study to explore in greater depth the impact that AI is having on people and societies worldwide. AI is being deployed in applications that touch people™s lives in a critical and personal way (for example, through loan approvals, criminal sentencing, healthcare, emotional care, and in˚uential recommendations in multiple realms, for example). Since these society-facing applications will in˚uence people™s relationship with AI technologies, as well as have far-reaching socioeconomic implications, we entitled the charge, fiPermeating In˚uences of AI in Everyday Life: Hopes, Concerns, and Directions.fl In addition to including topics directly related to these society-facing applications that resulted from the aforementioned workshops (as represented by WQ1 and WQ2 of this report), the Standing Committee carefully considered how to launch the Study Panel for the second report in such a way that it would set a precedent for all subsequent Study Panels, emphasizing the unique longitudinal aspect of the AI100 study. Motivated by the notion that it takes at least two points to de˜ne a line, as noted by AI100 founder Eric Horvitz, the Study Panel charge suggested a set of fistanding questionsfl for the Study Panel to consider that could potentially then be answered by future Study Panels as well (as represented by SQ1-SQ12 of this report) and included a call to re˚ect on the ˜rst report, indicating what has changed and what remains the same (as represented here ). While the scope of this charge was broader than 1

PAGE – 3 ============
3the inaugural panel™s focus on typical North American cities, it still does notŠand cannotŠcover all aspects of AI™s in˚uences on society, leaving some topics to be introduced or explored further in subsequent reports. In particular, military applications were outside the scope of the ˜rst report; and while military AI is used as a key case study in one section of this report ( SQ7), vigorous discussions of the subject are still continuing worldwide and opinions are evolving. Like the ˜rst report, the second report aspires to address four audiences. For the general public, it aims to provide an accessible, scienti˜cally and technologically accurate portrayal of the current state of AI and its potential. For industry, the report describes relevant technologies and legal and ethical challenges, and may help guide resource allocation. The report is also directed to local, national, and international governments to help them better plan for AI in governance. Finally, the report can help AI researchers, as well as their institutions and funders, to set priorities and consider the economic, ethical, and legal issues raised by AI research and its applications. The Standing Committee is grateful to the members of the Study Panel for investing their expertise, perspectives, and signi˜cant time into the creation of this report. We are also appreciative of the contributions of the leaders and participants of the workshops mentioned above, as well as past members of the Standing Committee, whose contributions were invaluable in setting the stage for this report: Yoav Shoham and Deirdre Mulligan (2015- 2017); Tom Mitchell and Alan Mackworth (2015-2018); Milind Tambe (2018); and Eric Horvitz (2015-2019).We especially thank Professor Michael Littman for agreeing to serve as chair of the study and for his wise, skillful, and dedicated leadership of the panel, its discussions, and creation of the report. Organizers of the Preparatory Workshops Thomas Arnold, Tufts University Solon Barocas, Microsoft Research Miranda Bogen, Upturn Morgan Currie, The University of Edinburgh Andrew Elder, The University of Edinburgh Jessica Feldman, American University of Paris Johannes Himmelreich, Syracuse University Jon Kleinberg, Cornell University Karen Levy, Cornell University Fay Niker, Cornell Tech Helen Nissenbaum, Cornell Tech David G. Robinson, Upturn Peter Stone, The University of Texas at Austin and Sony AI, ChairRuss Altman, Stanford University Erik Brynjolfsson, Stanford University Vincent Conitzer, Duke University and University of Oxford Mary L. Gray, Microsoft Research Barbara Grosz, Harvard University Ayanna Howard, The Ohio State University Percy Liang, Stanford University Patrick Lin, California Polytechnic State University James Manyika, McKinsey & Company Sheila McIlraith, University of Toronto Liz Sonenberg, The University of Melbourne Judy Wajcman, London School of Economics and The Alan Turing Institute Standing Committee of the One Hundred Year Study of Arti˜cial Intelligence

PAGE – 4 ============
4fiThe One Hundred Year Study on Arti˜cial Intelligence (AI100), launched in the fall of 2014, is a long-term investigation of the ˜eld of Arti˜cial Intelligence (AI) and its in˚uences on people, their communities, and society. It considers the science, engineering, and deployment of AI-enabled computing systems. As its core activity, the Standing Committee that oversees the One Hundred Year Study forms a Study Panel every ˜ve years to assess the current state of AI. The Study Panel reviews AI™s progress in the years following the immediately prior report, envisions the potential advances that lie ahead, and describes the technical and societal challenges and opportunities these advances raise, including in such arenas as ethics, economics, and the design of systems compatible with human cognition. The overarching purpose of the One Hundred Year Study™s periodic expert review is to provide a collected and connected set of re˚ections about AI and its in˚uences as the ˜eld advances. The studies are expected to develop syntheses and assessments that provide expert- informed guidance for directions in AI research, development, and systems design, as well as programs and policies to help ensure that these systems broadly bene˜t individuals and society. fiThe One Hundred Year Study is modeled on an earlier effort informally known as the fiAAAI Asilomar Study.fl During 2008-2009, the then president of the Association for the Advancement of Arti˜cial Intelligence (AAAI), Eric Horvitz, assembled a group of AI experts from multiple institutions and areas of the ˜eld, along with scholars of cognitive science, philosophy, and law. Working in distributed subgroups, the participants addressed near-term AI developments, long- term possibilities, and legal and ethical concerns, and then came together in a three-day meeting at Asilomar to share and discuss their ˜ndings. A short written report on the intensive meeting discussions, ampli˜ed by the participants™ subsequent discussions with other colleagues, generated widespread interest and debate in the ˜eld and beyond. fiThe impact of the Asilomar meeting, and important advances in AI that included AI algorithms and technologies starting to enter daily life around the globe, spurred the idea of a long- term recurring study of AI and its in˚uence on people and society. The One Hundred Year Study was subsequently endowed at a university to enable extended deep thought and cross-disciplinary scholarly investigations that could inspire innovation and provide intelligent advice to government agencies and industry.fl ABOUT AI100 The following history of AI100 ˜rst appeared in the preface of the 2016 report.

PAGE – 5 ============
5This report is structured as a collection of responses by the 2021 Study Panel to a collection of 12 standing questions (SQs) and two workshop questions (WQs) posed by the AI100 Standing Committee. The report begins with a list of the 14 questions and short summaries of the panel™s responses, which serves as an overview of the report™s ˜ndings. It then dives into the responses themselves and a brief conclusion section. An appendix includes a collection of annotations to the prior report in the AI100 series, published in 2016. Michael L. Littman, Ifeoma Ajunwa, Guy Berger, Craig Boutilier, Morgan Currie, Finale Doshi-Velez, Gillian Had˜eld, Michael C. Horowitz, Charles Isbell, Hiroaki Kitano, Karen Levy, Terah Lyons, Melanie Mitchell, Julie Shah, Steven Sloman, Shannon Vallor, and Toby Walsh. fiGathering Strength, Gathering Storms: The One Hundred Year Study on Arti˜cial Intelligence (AI100) 2021 Study Panel Report.fl Stanford University, Stanford, CA, September 2021. Doc: . Accessed: September 16, 2021. INTRODUCTIONHOW TO CITE THIS REPORT Study Panel Michael L. Littman, Brown University, Chair Ifeoma Ajunwa, University of North Carolina Guy Berger, LinkedIn Craig Boutilier, Google Morgan Currie, The University of Edinburgh Finale Doshi-Velez, Harvard University Gillian Had˜eld, University of Toronto Michael C. Horowitz, University of Pennsylvania Charles Isbell, Georgia Institute of Technology Hiroaki Kitano, Okinawa Institute of Science and Technology Graduate University and Sony AI Karen Levy, Cornell University Terah Lyons Melanie Mitchell, Santa Fe Institute and Portland State University Julie Shah, Massachusetts Institute of Technology Steven Sloman, Brown University Shannon Vallor, The University of Edinburgh Toby Walsh, University of New South Wales AcknowledgmentsThe panel would like to thank the members of the Standing Committee, listed in the preface. In addition to setting the direction and vision for the report, they provided detailed and truly insightful comments on everything from tone to detailed word choices that made the report clearer and, we hope!, more valuable in the long run. Standing Committee chair Peter Stone, in particular, deserves particular credit for his remarkable ability to ˜nd ways to negotiate clever solutions to the not-uncommon differences of opinion that inevitably arise by design of having a diverse set of contributors. We are grateful to Hillary Rosner, who, with the help of Philip Higgs and Stephen Miller, provided exceptionally valuable writing and editorial support. Jacqueline Tran and Russ Altman were deeply and adeptly involved in coordinating the efforts of both the Standing Committee and the Study Panel. We also thank colleagues who have provided pointers or feedback or other insights that helped inform our treatment on technical issues such as the use of AI in healthcare. They include: Nigam Shah, Jenna Wiens, Mark Sendak, Michael Sjoding, Jim Fackler, Mert Sabuncu, Leo Celi, Susan Murphy, Dan Lizotte, Jacqueline Kueper, Ravninder Bahniwal, Leora Horwitz, Russ Greiner, Philip Resnik, Manal Siddiqui, Jennifer Rizk, Martin Wattenberg, Na Li, Weiwei Pan, Carlos Carpi, Yiling Chen, Sarah Rathnam.

PAGE – 6 ============
6TABLE OF CONTENTS Preface 1About AI100 4Introduction 5Standing Questions and Section Summaries 7Workshop Questions and Section Summaries 11SQ1. What are some examples of pictures that re˚ect important progress in AI and its in˚uences? 12SQ2. What are the most important advances in AI? 12SQ3. What are the most inspiring open grand challenge problems? 18SQ4. How much have we progressed in understanding the key mysteries of human intelligence? 24SQ5. What are the prospects for more general arti˜cial intelligence? 29SQ6. How has public sentiment towards AI evolved, and how should we inform/educate the public? 33SQ7. How should governments act to ensure AI is developed and used responsibly? 37SQ8. What should the roles of academia and industry be, respectively, in the development and deployment of AI technologies and the study of the impacts of AI? 43SQ9. What are the most promising opportunities for AI? 48SQ10. What are the most pressing dangers of AI? 53SQ11. How has AI impacted socioeconomic relationships? 56SQ12. Does it appear fibuilding in how we thinkfl works as an engineering strategy in the long run? 60WQ1. How are AI-driven predictions made in high-stakes public contexts, and what social, organizational, and practical considerations must policymakers consider in their implementation and governance?: Lessons from fiPrediction in Practicefl workshop 63WQ2. What are the most pressing challenges and signi˜cant opportunities in the use of arti˜cial intelligence to provide physical and emotional care to people in need?: Lessons from fiCoding Caringfl workshop 69Conclusions 71Panel Member Bios 72Annotations on the 2016 Report 75

PAGE – 8 ============
8neuroscience, studying how the brain™s hardware is involved in implementing psychological and social processes; and computational modeling, which is now full of machine-learning-inspired models of visual recognition, language processing, and other cognitive activities. The nature of consciousness and how people integrate information from multiple modalities, multiple senses, and multiple sources remain largely mysterious. Insights in these areas seem essential in our quest for building machines that we would truly judge as fiintelligent.fl SQ5. What are the prospects for more general arti˜cial intelligence?The ˜eld is still far from producing fully general AI systems. However, in the last few years, important progress has been made in the form of three key capabilities. First is the ability for a system to learn in a self-supervised or self-motivated way. A self-supervised model called transformers has become the go-to approach for natural language processing, and has been used in diverse applications, including machine translation and Google web search. Second is the ability for a single AI system to learn in a continual way to solve problems from many different domains without requiring extensive retraining for each. One in˚uential approach is to train a deep neural network on a variety of tasks, where the objective is for the network to learn general-purpose, transferable representations, as opposed to representations tailored speci˜cally to any particular task. Third is the ability for an AI system to generalize between tasksŠ that is, to adapt the knowledge and skills the system has acquired for one task to new situations. A promising direction is the use of intrinsic motivation, in which an agent is rewarded for exploring new areas of the problem space. AI systems will likely remain very far from human abilities, however, without being more tightly coupled to the physical world.SQ6. How has public sentiment towards AI evolved, and how should we inform/educate the public?Over the last few years, AI and related topics have gained traction in the zeitgeist. In the 2017Œ18 session of the US Congress, for instance, mentions of AI-related words were more than ten times higher than in previous sessions. Media coverage of AI often distorts and exaggerates AI™s potential at both the positive and negative extremes, but it has helped to raise public awareness of legitimate concerns about AI bias, lack of transparency and accountability, and the potential of AI-driven automation to contribute to rising inequality. Governments, universities, and nonpro˜ts are attempting to broaden the reach of AI education, including investing in new AI-related curricula. Nuanced views of AI as a human responsibility are growing, including an increasing effort to engage with ethical considerations. Broad international movements in Europe, the US, China, and the UK have been pushing back against the indiscriminate use of facial-recognition systems on the general public. More public outreach from AI scientists would be bene˜cial as society grapples with the impacts of these technologies. It is important that the AI research community move beyond the goal of educating or talking to the public and toward more participatory engagement and conversation with the public.SQ7. How should governments act to ensure AI is developed and used responsibly?Since the publication of the last AI100 report just ˜ve years ago, over 60 countries have engaged in national AI initiatives, and several signi˜cant new multilateral efforts are aimed at spurring effective international cooperation on related topics. To date, few countries have moved de˜nitively to regulate AI speci˜cally, outside of rules directly related to the use of data. As of 2020, 24 countries had opted for permissive laws to allow autonomous

PAGE – 9 ============
9vehicles to operate in limited settings. As yet, only Belgium has enacted laws on the use of autonomous lethal weapons. The oversight of social media platforms has become a hotly debated issue worldwide. Cooperative efforts among countries have also emerged in the last several years. Appropriately addressing the risks of AI applications will inevitably involve adapting regulatory and policy systems to be more responsive to the rapidly advancing pace of technology development. Researchers, professional organizations, and governments have begun development of AI or algorithm impact assessments (akin to the use of environmental impact assessments before beginning new engineering projects). SQ8. What should the roles of academia and industry be, respectively, in the development and deployment of AI technologies and the study of the impacts of AI?As AI takes on added importance across most of society, there is potential for con˚ict between the private and public sectors regarding the development, deployment, and oversight of AI technologies. The commercial sector continues to lead in AI investment, and many researchers are opting out of academia for full-time roles in industry. The presence and in˚uence of industry-led research at AI conferences has increased dramatically, raising concerns that published research is becoming more applied and that topics that might run counter to commercial interests will be underexplored. As student interest in computer science and AI continues to grow, more universities are developing standalone AI/machine-learning educational programs. Company-led courses are becoming increasingly common and can ˜ll curricular gaps. Studying and assessing the societal impacts of AI, such as concerns about the potential for AI and machine-learning algorithms to shape polarization by in˚uencing content consumption and user interactions, is easiest when academic-industry collaborations facilitate access to data and platforms.SQ9. What are the most promising opportunities for AI? AI approaches that augment human capabilities can be very valuable in situations where humans and AI have complementary strengths. An AI system might be better at synthesizing available data and making decisions in well-characterized parts of a problem, while a human may be better at understanding the implications of the data. It is becoming increasingly clear that all stakeholders need to be involved in the design of AI assistants to produce a human-AI team that outperforms either alone. AI software can also function autonomously, which is helpful when large amounts of data needs to be examined and acted upon. Summarization and interactive chat technologies have great potential. As AI becomes more applicable in lower-data regimes, predictions can increase the economic ef˜ciency of everyday users by helping people and businesses ˜nd relevant opportunities, goods, and services, matching producers and consumers. We expect many mundane and potentially dangerous tasks to be taken over by AI systems in the near future. In most cases, the main factors holding back these applications are not in the algorithms themselves, but in the collection and organization of appropriate data and the effective integration of these algorithms into their broader sociotechnical systems.SQ10. What are the most pressing dangers of AI?As AI systems prove to be increasingly bene˜cial in real-world applications, they have broadened their reach, causing risks of misuse, overuse, and explicit abuse to proliferate. One of the most pressing dangers of AI is techno-solutionism, the view that AI can be seen as a panacea when it is merely a tool. There is an aura of neutrality and impartiality associated with AI decision- making in some corners of the public consciousness, resulting in systems being accepted as objective even though they may be the result of biased historical decisions or even blatant discrimination. Without transparency concerning either the data or the AI

PAGE – 10 ============
10algorithms that interpret it, the public may be left in the dark as to how decisions that materially impact their lives are being made. AI systems are being used in service of disinformation on the internet, giving them the potential to become a threat to democracy and a tool for fascism. Insuf˜cient thought given to the human factors of AI integration has led to oscillation between mistrust of the system and over-reliance on the system. AI algorithms are playing a role in decisions concerning distributing organs, vaccines, and other elements of healthcare, meaning these approaches have literal life-and-death stakes. SQ11. How has AI impacted socioeconomic relationships?Though characterized by some as a key to increasing material prosperity for human society, AI™s potential to replicate human labor at a lower cost has also raised concerns about its impact on the welfare of workers. To date, AI has not been responsible for large aggregate economic effects. But that may be because its impact is still relatively localized to narrow parts of the economy. In the grand scheme of rising inequality, AI has thus far played a very small role. The ˜rst reason, most importantly, is that the bulk of the increase in economic inequality across many countries predates signi˜cant commercial use of AI. Since these technologies might be adopted by ˜rms simply to redistribute surplus/gains to their owners, AI could have a big impact on inequality in the labor market and economy without registering any impact on productivity growth. No evidence of such a trend is yet apparent, but it may become so in the future and is worth watching closely. To date, the economic signi˜cance of AI has been comparatively smallŠparticularly relative to expectations, among both optimists and pessimists. Other forcesŠglobalization, the business cycle, and a pandemicŠhave had a much, much bigger and more intense impact in recent decades. But if policymakers underreact to coming changes, innovations may simply result in a pie that is sliced ever more unequally. SQ12. Does it appear fibuilding in how we thinkfl works as an engineering strategy in the long run?AI has its own fundamental nature-versus-nurture-like question. Should we attack new challenges by applying general-purpose problem-solving methods, or is it better to write specialized algorithms, designed by experts, for each particular problem? Roughly, are speci˜c AI solutions better engineered in advance by people (nature) or learned by the machine from data (nurture)? The pendulum has swung back and forth multiple times in the history of the ˜eld. In the 2010s, the addition of big data and faster processors allowed general-purpose methods like deep learning to outperform specialized hand-tuned methods. But now, in the 2020s, these general methods are running into limitsŠavailable computation, model size, sustainability, availability of data, brittleness, and a lack of semanticsŠthat are starting to drive researchers back into designing specialized components of their systems to try to work around them. Indeed, even machine-learning systems bene˜t from designers using the right architecture for the right job. The recent dominance of deep learning may be coming to an end. To continue making progress, AI researchers will likely need to embrace both general- and special-purpose hand-coded methods, as well as ever faster processors and bigger data.

PAGE – 11 ============
11WORKSHOP QUESTIONS AND SECTION SUMMARIESWQ1. How are AI-driven predictions made in high-stakes public contexts, and what social, organizational, and practical considerations must policymakers consider in their implementation and governance?: Lessons from fiPrediction in Practicefl workshopResearchers are developing predictive systems to respond to contentious and complex public problems across all types of domains, including criminal justice, healthcare, education, and social servicesŠhigh-stakes contexts that can impact quality of life in material ways. Success is greatly in˚uenced by how a system is or is not integrated into existing decision-making processes, policies, and institutions. The ways we de˜ne and formalize prediction problems shape how an algorithmic system looks and functions. Even subtle differences in problem de˜nition can signi˜cantly change resulting policies. The most successful predictive systems are not dropped into place but are thoughtfully integrated into existing social and organizational environments and practices. Matters are further complicated by questions about jurisdiction and the imposition of algorithmic objectives at a state or regional level that are inconsistent with the goals held by local decision-makers. Successfully integrating AI into high-stakes public decision-making contexts requires dif˜cult work, deep and multidisciplinary understanding of the problem and context, cultivation of meaningful relationships with practitioners and affected communities, and a nuanced understanding of the limitations of technical approaches. WQ2. What are the most pressing challenges and signi˜cant opportunities in the use of arti˜cial intelligence to provide physical and emotional care to people in need?: Lessons from fiCoding Caringfl workshop Smart home devices can give Alzheimer’s patients medication reminders, pet avatars and humanoid robots can offer companionship, and chatbots can help veterans living with PTSD treat their mental health. These intimate forms of AI caregiving challenge how we think of core human values, like privacy, compassion, trust, and the very idea of care itself. AI offers extraordinary tools to support caregiving and increase the autonomy and well- being of those in need. Some patients may even express a preference for robotic care in contexts where privacy is an acute concern, as with intimate bodily functions or other activities where a non-judgmental helper may preserve privacy or dignity. However, in elder care, particularly for dementia patients, companion robots will not replace the human decision-makers who increase a patient™s comfort through intimate knowledge of their conditions and needs. The use of AI technologies in caregiving should aim to supplement or augment existing caring relationships, not replace them, and should be integrated in ways that respect and sustain those relationships. Good care demands respect and dignity, things that we simply do not know how to code into procedural algorithms. Innovation and convenience through automation should not come at the expense of authentic care.

257 KB – 82 Pages