IEEE VIS 2024 Content: Voicing Uncertainty: How Speech, Text, and Visualizations Influence Decisions with Data Uncertainty

Voicing Uncertainty: How Speech, Text, and Visualizations Influence Decisions with Data Uncertainty

Chase Stokes - University of California Berkeley, Berkeley, United States

Chelsea Sanker - Stanford University, Stanford, United States

Bridget Cogley - Versalytix, Columbus, United States

Vidya Setlur - Tableau Research, Palo Alto, United States

Download preprint PDF

Room: Bayshore VI

2024-10-14T12:30:00ZGMT-0600Change your timezone on the schedule page
2024-10-14T12:30:00Z
Exemplar figure, described by caption below
Example stimuli viewed by participants. (a) Visualization-only representation: a density plot showing the distribution of possible nighttime temperatures. (c) Speech-forward representation: contains the same density mark to provide some visual information, accompanied by an mp3 player which describes the distribution, temperature values, and likelihoods. We tested six different variants of these representations, with three masculine voices and three feminine voices. (c) Text-forward representation: contains the density mark and a text paragraph describing the distribution and likelihoods for different values. This is the same content as present in the speech forecast.
Fast forward
Abstract

Understanding and communicating data uncertainty is crucial for informed decision-making across various domains, including finance, healthcare, and public policy. This study investigates the impact of gender and acoustic variables on decision-making, confidence, and trust through a crowdsourced experiment. We compared visualization-only representations of uncertainty to text-forward and speech-forward bimodal representations, including multiple synthetic voices across gender. Speech-forward representations led to an increase in risky decisions, and text-forward representations led to lower confidence. Contrary to prior work, speech-forward forecasts did not receive higher ratings of trust. Higher normalized pitch led to a slight increase in decision confidence, but other voice characteristics had minimal impact on decisions and trust. An exploratory analysis of accented speech showed consistent results with the main experiment and additionally indicated lower trust ratings for information presented in Indian and Kenyan accents. The results underscore the importance of considering acoustic and contextual factors in presentation of data uncertainty.