Abstract
Existing research primarily evaluates the values of LLMs by examining their
stated inclinations towards specific values. However, the "Value-Action Gap," a
phenomenon rooted in environmental and social psychology, reveals discrepancies
between individuals' stated values and their actions in real-world contexts. To
what extent do LLMs exhibit a similar gap between their stated values and their
actions informed by those values? This study introduces ValueActionLens, an
evaluation framework to assess the alignment between LLMs' stated values and
their value-informed actions. The framework encompasses the generation of a
dataset comprising 14.8k value-informed actions across twelve cultures and
eleven social topics, and two tasks to evaluate how well LLMs' stated value
inclinations and value-informed actions align across three different alignment
measures. Extensive experiments reveal that the alignment between LLMs' stated
values and actions is sub-optimal, varying significantly across scenarios and
models. Analysis of misaligned results identifies potential harms from certain
value-action gaps. To predict the value-action gaps, we also uncover that
leveraging reasoned explanations improves performance. These findings
underscore the risks of relying solely on the LLMs' stated values to predict
their behaviors and emphasize the importance of context-aware evaluations of
LLM values and value-action gaps.