People often interact with AI language models to form an opinion on a particular issue. Depending on their prior knowledge and personality, they express varying degrees of confidence in their own knowledge through language. This project examines how language models handle queries with different levels of confidence: Do they adapt their language to match the expressed confidence, or do they predominantly answer queries in a confident and persuasive tone?

The confidence expressed in a query provides insight into people’s metacognitive insight. An initial series of studies in this project examines whether language models a) are capable of recognizing queries of varying confidence, b) whether language models adapt to the level of confidence, or whether they consistently appear confident, and c) how a language model’s response affects the users’ confidence. If language models adapt their confidence, or if a language model’s high confidence increases the user’s confidence, this would constitute metacognitive contagion. From a societal perspective, metacognitive contagion is desirable for many topics; however, for controversial and politicized topics, metacognitive contagion could lead people to gain greater confidence in their knowledge than is actually warranted. Therefore, the project will examine metacognitive contagion for politicized and non-politicized topics.