IEEE VIS 2024 Content: Old Wine in a New Bottle? Analysis of Visual Lineups with Signal Detection Theory

Old Wine in a New Bottle? Analysis of Visual Lineups with Signal Detection Theory

Sheng Long - Northwestern University, Evanston, United States

Matthew Kay - Northwestern University, Chicago, United States

Download preprint PDF

Room: Bayshore I

2024-10-14T12:30:00ZGMT-0600Change your timezone on the schedule page
2024-10-14T12:30:00Z
Exemplar figure, described by caption below
The image connects a visual lineup task with signal detection theory. It shows a lineup of multivariate images where participants identify if one differs or if there's "no discernible difference." Signal detection theory analyzes this data, assuming perceptual evidence for signal presence/absence as overlapping probability distributions. This quantifies observer sensitivity and decision criterion, separating perceptual sensitivity from response bias. The graphs illustrate concepts like false alarm rate, hit rate, and sensitivity (d'), demonstrating how the theory applies to perceptual decision-making in visual discrimination tasks.
Full Video
Abstract

This position paper critically examines the graphical inference framework for evaluating visualizations using the lineup task. We present a re-analysis of lineup task data using signal detection theory, applying four Bayesian non-linear models to investigate whether color ramps with more color name variation increase false discoveries. Our study utilizes data from Reda and Szafir’s previous work [20], corroborating their findings while providing additional insights into sensitivity and bias differences across colormaps and individuals. We suggest improvements to lineup study designs and explore the connections between graphical inference, signal detection theory, and statistical decision theory. Our work contributes a more perceptually grounded approach for assessing visualization effectiveness and offers a path forward for better aligning graphical inference methods with human cognition. The results have implications for the development and evaluation of visualizations, particularly for exploratory data analysis scenarios. Supplementary materials are available at https://osf.io/xd5cj/.