Artificial intelligence is everywhere. It suggests items for online shopping carts, generates massive quantities of text, organizes Instagram feeds and mimics speech.
Now, it can even predict election outcomes.
Research produced by BYU’s political science and computer science departments suggests that large language models are capable of simulating the opinions, values and judgments of a Republican or Democrat, according to BYU computer science professor David Wingate.
“A language model could simulate what a Republican would say, and it sounds plausible,” Wingate said. “These language models do a really good job of predicting things like vote choice.”
In their recently published study, Wingate and his colleagues used GPT-3, one of OpenAI’s older language models, to simulate election outcomes for several previous presidential elections.
GPT-3 is trained on data through 2018. When the research team quizzed it with future hypotheticals — for example, the 2020 presidential election — its output aligned “surprisingly well” with actual election results, Wingate said.
“We could absolutely rerun our experiments with a hypothetical 2024 election and make predictions,” he said. “That sort of thing would be absolutely possible.”
Wingate said they are working hard to figure out the extent to which language models can replace humans in surveys. Removing humans from the equation could help researchers study difficult and taboo topics such as racism, he explained.
Lisa Argyle, a BYU political science professor and co-author on the study, said the research has significant implications for how artificial intelligence can be used in the study of humans.
“It’s a bit of a pie in the sky hope, but we could maybe improve the representativeness of our surveys, or how well we’re representing … minority groups,” she said.
Using artificial intelligence to generate synthetic survey responses is also incredibly cost-effective. According to the published paper, Study 1 only cost $29 on GPT-3. A single traditional person-to-person survey typically costs at least that, Argyle said.
However, Argyle acknowledged the study’s limitations. Though it produces good results in the context of a national-level American partisan election, she said she is unsure of how GPT-3 performs in other domains.
Additionally, a large language model’s finite training limits its effectiveness in real-time polling.
“We probably aren’t going to see major polling firms switch to purely synthetic polling,” she said. “People will try to do that. It is not apparent if that actually works, across the board.”
Data analyst Daryl Acumen said he doubts the reliability of synthetically-generated survey responses.
Acumen has worked part-time with Integrity Matters, a Utah-based data and voter outreach company which performs data analytics for political candidates in the state, for 10 years.
“I’ve been in tech for 30 years and analytics for 20 years,” he said. “I don’t care about the bright and shiny objects. I care about what actually gets results.”
In Acumen’s experience, meaningful human interactions generate better results than synthetic surveys. Beyond data collection, each person-to-person conversation has an unquantifiable ripple effect. Post-survey, individuals often become more politically involved, he said.
Regardless of its actual potential for market research, both Acumen and Wingate predict companies will continue pushing artificial intelligence in the interest of increased profit margins.
“As we think about the future of AI, relentless progress is what we should all remember,” Wingate said. “Also, we have a choice. We get to choose our own future, and we get to choose how AI is going to impact our society.”
