detalle del documento
IDENTIFICACIÓN
oai:arXiv.org:2405.19323
Tema
Computer Science - Computation and... Computer Science - Artificial Inte... Computer Science - Computers and S... Computer Science - Machine Learnin...Autor
Geng, Mingmeng He, Sihong Trotta, RobertoCategoría
Computer Science
Año
2024
fecha de cotización
23/10/2024
Palabras clave
simulate surveys languageResumen
Can large language models (LLMs) simulate social surveys?
To answer this question, we conducted millions of simulations in which LLMs were asked to answer subjective questions.
A comparison of different LLM responses with the European Social Survey (ESS) data suggests that the effect of prompts on bias and variability is fundamental, highlighting major cultural, age, and gender biases.
We further discussed statistical methods for measuring the difference between LLM answers and survey data and proposed a novel measure inspired by Jaccard similarity, as LLM-generated responses are likely to have a smaller variance.
Our experiments also reveal that it is important to analyze the robustness and variability of prompts before using LLMs to simulate social surveys, as their imitation abilities are approximate at best.
;Comment: 17 pages
Geng, Mingmeng,He, Sihong,Trotta, Roberto, 2024, Are Large Language Models Chameleons? An Attempt to Simulate Social Surveys
High-Frequency Repetitive Magnetic Stimulation at the Sacrum Alleviates Chronic Constipation in Parkinson’s Patients
magnetic stimulation parkinson’s significant patients scale sacrum pd hf-rms chronic constipation scores
