[Special Feature: Science, Technology, and Social Issues] Hirotaka Osawa: From Artificial Agent Research to Narrative Application Research — The Expansion of SF Imagination

For many years, I have focused my research on the field of Human-Agent Interaction (HAI). At the same time, I am also conducting research on the application of narratives. I would like to explain the relationship between these two areas.

HAI is an interdisciplinary field that has been established for only a little over 20 years, so I have spent a lot of time explaining what kind of field it is. HAI is a discipline that deals with the interaction between humans and agents, but that alone is not a sufficient explanation. The "agents" referred to here are primarily what are called social agents—artificial objects that behave as if they have intentions, much like humans or animals.

While HAI spans various domains, in a nutshell, it can be described as "a discipline that adjusts people's imagination toward social agents and explores useful applications for humans." HAI research did not emerge out of nowhere; there were many disciplines dealing with similar technologies, such as social robots and virtual agents. However, HAI was born as a field that treats these agents not merely as media, but as entities that can socially influence people in a complex manner. Consequently, researchers from multiple fields—not only artificial intelligence and robotics, but also human-computer interaction, cognitive science, simulation, game theory, artificial life, video game studies, ethics, social sciences, and philosophy—have focused on this area. Their work ranges from the design and implementation of the appearance, communication, and interaction of robots, virtual agents, voice dialogues, and telepresence systems, to the estimation of others' intentions and the creation of cooperative behavior in AI systems, as well as the examination of the social applications of agent systems and their legal, ethical, and social issues. From this perspective, I have personally conducted research on anthropomorphizing household appliances and on artificial intelligence that cooperates with and deceives humans in games like Werewolf.

In HAI research, the emphasis is placed less on the information processing of the agent itself and more on how that agent is treated by humans when they interact. For example, when designing the emotions of a pet robot, the focus is not on how the pet robot feels, but on how those emotions are perceived by humans. To put it bluntly, one might say that HAI research is the study of "deceiving people into thinking an artificial object has intentions." However, in a sense, this is essential; HAI research is, in a way, the study of designing the "fiction within the user."

An interesting example of implementation from an HAI perspective is Primo Puel (released in 1999), a doll-shaped plush toy developed by Bandai. This doll was a simple plush toy that merely uttered certain words in response to how a human touched it via sensors installed in various parts; from the perspective of AI technology, it could hardly be called a sophisticated piece of engineering. However, Primo Puel was a hit "robot," primarily among middle-aged and older women, with over a million units sold and utilized in various settings, including nursing care.

Primo Puel has a backstory that sets the stage before it is purchased by a user. In this story, Primo Puel is an alien from a small star who exists to heal people's hearts, and its various character traits were part of its appeal. While the story of Primo Puel might not be considered a "masterpiece" of narrative, it is noteworthy that, regardless of the story's quality, the focus was on Primo Puel creating a relationship with the purchaser.

In artificial intelligence research, narrative research has a long history, ranging from narrative analysis to narrative generation. Traditionally, such narrative research and HAI research were not treated within the same framework. This is because HAI is primarily a field for designing dynamic computer devices where the "fiction" the user believes in is secondary and the range of control was narrow, whereas narratives are basically static media where users receive a chronological fiction through text or video. However, in interactive media like video games, the designs and objectives of both approach each other. For example, the design of non-player characters (NPCs) in video games and the research on virtual agents appearing on screens use similar technologies and are sometimes presented at similar academic conferences.

In particular, recent HAI research has seen an approach where, rather than refining the technology of a single agent, the goal is to improve human society as a result by intervening with single or multiple agents. For example, in "Learning by Teaching" proposed by Professor Fumihide Tanaka of the University of Tsukuba, an approach is taken where the agent itself becomes an entity taught by a child, resulting in the child learning. Furthermore, in the "Weak Robots" proposed by Professor Michio Okada of Toyohashi University of Technology, the presence of a robot that can do nothing results in prompting human action. Such research can be called HAI research that expands human society, and at the same time, it is close to narrative research in that it stimulates people's imagination and prompts human action as a result. Narrative is a challenge that requires handling multiple social entities over time, and in that sense, the work of designing imagination toward agents can be said to be included as a part of this. In almost all narratives, there is a narrator, and in many narratives, multiple actors and societies exist.

Based on the above points, I believe there are significant benefits to encouraging exchange between HAI research and narrative research in their respective developments.

While I have shown the intersection of HAI research and narrative research, the positioning of narrative itself for humanity is also transforming with the development of information technology. In particular, artificial intelligence technology centered on generative AI and the literary fields related to science fiction (SF) are driving this development.

In human history, the power to handle fiction based on imagination is considered to have been a major factor in the development of human culture. Human imagination has contributed in various aspects, such as social cohesion, the transmission of culture, the formation of individual and collective identities, and even problem-solving and the promotion of creativity. Literary scholar Brian Boyd discusses how narrative became an essential element for human cognitive development, showing that narrative has played an important role in transmitting information, sharing social rules and values, and fostering a sense of solidarity within groups. Since the modern era, this imagination for handling fiction in humanity has been organized in the form of an industrial structure centered on narrative. In particular, science fiction (SF), which deals with science and technology, was born as a genre to explore the relationship between science and technology and society in such times.

SF, while repeating cycles of longing for science and harsh self-criticism, has moved beyond being just a literary genre to exert various influences on social design. In the humanities as well, multiple related influences can be seen. SF is used as a tool for science communication and, like futurology, has become food for connecting multiple disciplines to discuss the future. What was once called science propaganda is now, even in fields such as STEAM (Science, Technology, Engineering, Art, and Mathematics) education, a major educational support through the imagination provided by SF. In management science, attempts at future prediction using scenarios have been conducted in fields such as foresight. Furthermore, in recent years, attempts such as "SF Prototyping" have emerged, where the process of creating a work—rather than the work itself—is shared between authors and experts to gain new visions for corporate management and education. Additionally, art fields like critical design and speculative art have appeared, inheriting the critical attitude and spirit of SF and making problem-posing their primary focus.

SF prototyping, in particular, can be called a trend in the application of new narratives. This is an expansion of the framework of "SF that foresees the future," which was expected of SF in the past, into a much broader range. SF prototyping does not necessarily predict the future. This is the same as how an SF novel is not simply a future-prediction novel. However, SF can connect current knowledge, technology, people's values, and multiple visions for the future to provide people with possible possibilities in the form of a story. People can then appreciate that story, critically examine it in some cases, and by being involved in the creation process in other cases, gain a deeper perspective on the future and build a vision as a result. In this way, SF contributes to the creation of future visions, and as a result, some SF works receive the honor (whether welcome or not) of having "foreseen the future." SF prototyping can be described as intentionally triggering such attempts within a company.

One of the areas where SF is most effective is the introduction of a simulational perspective in an era where humans are expanded by science and technology and human values are shifting. Such a state of humanity in the future is treated under the theme of "Post-human." Due to the science and technology that emerged from the late 20th century, particularly information technology, the narratives created by imagination and the nature of being human are currently being greatly shaken. With internet technology and social media, we have entered an era where anyone can create their own content, and an environment has been established where narratives themselves are born not only through financial feedback but also through the evaluation of others. In a sense, this is the restoration of narrative as communication. Furthermore, with the generative AI technology that appeared in the 2010s, while facing significant backlash from some creators whose works are used for training, it is expected that the nature of creative skills will become more democratized. On the other hand, due to structural changes in the publishing industry caused by population decline and changes in the content environment, authors themselves are bringing narrative methodologies into the sites where science and technology are born as a form of narrative. In the society of the post-humanity era, where human society is fundamentally strengthened by information technology and the nature of our own intelligence is expanded, I believe that research into human imagination needs to be captured in a more transdisciplinary way than ever before.

In such an era as described above, we established a center at Keio University in January 2024 named the "Science Fiction Research and Development / Implementation Center." It is the first university research center in Japan to bear the name "SF." Similar frameworks exist in several locations in the United States. For example, the University of Kansas has a center for studying SF as literature, and in recent years, centers for innovation emergence using SF include the Center for Science and the Imagination at Arizona State University and the Arthur C. Clarke Center for Human Imagination at UC San Diego. In China, SF author and researcher Wu Yan established the Research Center for Science and Human Imagination at the Southern University of Science and Technology. However, until now, there was no center specializing in SF in Japan. Author Sakyo Komatsu, known for novels such as "Japan Sinks," once focused on the potential of SF under the name "SF-ology" and worked hard to establish it as an academic field. Additionally, author Hideaki Sena has discussed the relationship between SF and science and technology, primarily in collaboration with robotics researchers.

Inheriting these trends, the SF Center will carry out research on imagination centered on SF across six fields: literature, cognitive science, artificial intelligence, management science, science communication, and art. While I am a researcher in engineering, the center is a field where not only science and technology but also the humanities and social sciences contribute significantly. If the changes brought about by the evolution of AI and communication technology are not captured from a human-centered perspective rather than just a technical one, it will lead to technological acceleration that leaves human values behind. In particular, examining the influence of narratives on the developmental process of children who will lead future society can be said to be an urgent task. Furthermore, management science, in particular, expands the theoretical frameworks and analytical tools for deeply understanding the impact of new technologies and innovations on society and culture. The transformation of creative activities through AI technology significantly affects social values, cultural identity, and methods of communication; to understand and give meaning to these changes, collaboration with researchers of future-conception methods is essential. Especially in our country, which possesses rich narratives as a hidden resource and has broadcast a unique narrative culture comparable to the US and Europe, yet faces predicted fundamental structural changes including shifts in the publishing industry due to a declining birthrate and aging population, there will be a great contribution in organizing the influence of SF—which has played a pivotal role in narrative culture including not only novels but also manga, animation, video, video games, and derivative works—and broadcasting new forms of imagination to the world. Moreover, in the current situation where uncertain and catastrophic situations occur, such as the COVID-19 pandemic, the Russian invasion of Ukraine, and large-scale language model technology aiming for artificial general intelligence, it becomes important to develop the ability to examine possibilities in an uncertain future rather than predicting the future itself. In this regard, science communication and speculative art play a major role.

Furthermore, the development of AI technology, starting with generative AI, brings about major differences from the traditional ecosystem regarding narratives. Traditionally, narratives have been recognized as creative expressions of human thoughts or emotions, granted rights including moral rights, and have received legal and ethical protection. In contrast, when the creative process is automated as with generative AI, intervention occurs regarding the creativity of the work. This is expected to increase the things that need to be newly examined from the perspectives of literature and aesthetics. Another point is the diversion of narrative imagination itself. In the narrative environment since the modern era, the privileged role of the "author" existed, supported by publishing, and a form where many people received the narrative was established as an industry. However, in today's social credit economy, where network technology and social network platforms have developed and feedback to authors is given through praise for self-expression as well as money, the boundary between author and reader has become blurred. This era is, in a sense, the restoration of narrative as communication, but it is also happening that the imagination of authors who weave narratives is directly involved in the construction of future visions as part of a kind of simulation ability and insight toward the future. This environment, combined with the imagination-expansion environment provided by AI, results in further broadening the role of narratives created by human imagination in society. Current generative AI is causing problems across various frameworks, such as the attribution of creative works, the ethics of creative activities by AI, and the redefinition of the relationship between humans and AI. The center aims to provide the ethical frameworks and philosophical insights to handle these issues and serve as a bridge to deepen discussions on how society should respond to these new challenges.