LLM - Supported Exploration of 3D Microscopy Imaging
Aarti Darji - The University of Texas at Arlington, Arlington, United States
Eric Moerth - DBMI, Boston, United States
Morgan L Turner - Harvard Medical School, Boston, United States
David Kouřil - Harvard Medical School, Boston, United States
Jacob Luber - The University of Texas at Arlington, Arlington, United States
Nils Gehlenborg - Harvard Medical School, Boston, United States
Room: Bayshore V
2024-10-13T16:00:00ZGMT-0600Change your timezone on the schedule page
2024-10-13T16:00:00Z
Full Video
Abstract
The 3D Cycled Immunofluorescence (CyCIF) technique produces high-resolution multiplexed images, often representing a large number of biomarkers. With current visualization tools, it is hard to identify the important subset of markers and locate notable regions within the tissue. To address this challenge, we propose an LLM-supported agent to navigate 3D CyCIF Imaging that interprets a novice user's natural language queries, identifies relevant markers, and locates significant regions within the tissue. Our results demonstrate the agent's ability to dynamically update views, answering various queries, from general questions to specific region-based requests.