Toward AI VIS Co-Scientists: An Agent Harness for End-to-End Design of Scientific Visualization Applications

Abstract

Recent LLM-based visualization systems generate charts from natural language, drive visualization tools, prototype analytics interfaces, and score well on visualization benchmarks, but complex scientific visualization tasks still require repeated human steering before they yield a usable application. Much of the work done by visualization researchers calls for automating a larger task: an interactive application that starts from ambiguous domain materials, explores the data, derives task requirements, designs visual representations and coordinated interactions, implements the result, and checks whether it answers the original scientific questions. We argue that agentic visualization systems should automate this application-level design loop, and we study it in a zero-shot setting: one prompt with no human intervention after launch. The domain scientists who need such applications rarely have a visualization expert to consult, so the agent has to get as far as it can from an underspecified request. Each clarifying turn adds design intent that the system is supposed to supply, making the final result hard to attribute. We support the position with a reusable multi-agent harness that acts as a VIS co-scientist for domain experts. We test the harness on IEEE SciVis Contests spanning multiple science and engineering fields. These contests include the pressures we care about: ambiguous requirements, diverse data modalities, design trade-offs, and task-driven validation. We use the runs to map what the current harness can and cannot do, and to identify where visualization expertise should enter agent design.