IEEE VIS 2024 Content: Normalized Stress is Not Normalized: How to Interpret Stress Correctly

Normalized Stress is Not Normalized: How to Interpret Stress Correctly

Kiran Smelser - University of Arizona, Tucson, United States

Jacob Miller - University of Arizona, Tucson, United States

Stephen Kobourov - University of Arizona, Tucson, United States

Screen-reader Accessible PDF
Download preprint PDF

Room: Bayshore I

2024-10-14T12:30:00ZGMT-0600Change your timezone on the schedule page
2024-10-14T12:30:00Z
Exemplar figure, described by caption below
MDS, t-SNE, and RND (random) embeddings of the well-known Iris dataset from left to right (bottom). The plot (top) shows the values of the normalized stress metric for these three embeddings and clearly illustrates the sensitivity to scale. As one uniformly scales the embeddings to be larger or smaller, the value of normalized stress changes. Notably, at different scales, different embeddings have lower stress, including the absurd situation where the random embedding has the lowest stress (beyond scale 9). Moreover, the expected order of MDS, t-SNE, RND is only found briefly at a scalar value slightly greater than 0.25 (hardly visible in the plot), and all six different algorithm orders can be found by selecting different scales.
Fast forward
Full Video
Abstract

Stress is among the most commonly employed quality metrics and optimization criteria for dimension reduction projections of high-dimensional data. Complex, high-dimensional data is ubiquitous across many scientific disciplines, including machine learning, biology, and the social sciences. One of the primary methods of visualizing these datasets is with two-dimensional scatter plots that visually capture some properties of the data. Because visually determining the accuracy of these plots is challenging, researchers often use quality metrics to measure the projection’s accuracy or faithfulness to the full data. One of the most commonly employed metrics, normalized stress, is sensitive to uniform scaling (stretching, shrinking) of the projection, despite this act not meaningfully changing anything about the projection. We investigate the effect of scaling on stress and other distance-based quality metrics analytically and empirically by showing just how much the values change and how this affects dimension reduction technique evaluations. We introduce a simple technique to make normalized stress scale-invariant and show that it accurately captures expected behavior on a small benchmark.