Interactive visualization of densely populated volumes

Lesar, Žiga (author), Marolt, Matija (mentor)

In this thesis, we investigate interactive visibility management in crowded volumes, characterized by an overwhelming amount of mutually occluding and often intricately intertwined structures. In the first part, we introduce the Volume Conductor, a visibility management system for interactive grouping, coloring, and sparsification of structures in crowded volumes. These functionalities rely on a technical contribution we refer to as the visibility mask, which is an intermediate representation encoding the visibility and optical properties of groups of structures within the volume. Its simple and effective design allows the visibility mask and its corresponding transfer function to be automatically generated, requiring minimal effort from the user. The generation process is facilitated by an easy-to-use user interface with support for sequential and hierarchical structuring of instance groups. Furthermore, the visibility mask can be rendered with any volume rendering technique, provided it supports 2D transfer functions. In addition to standalone use, the visibility mask can be optionally blended with a raw data volume. This flexibility enables straightforward and seamless integration of the Volume Conductor into existing volume visualization applications. The most important aspect of the Volume Conductor is instance sparsification, which is controlled per group by the user through visibility ratio sliders. The order of sparsification is determined by a sparsification function, which assigns importance to instances. We generalize the concept of importance by aggregating voxel-based importance to define instance-based importance. Three sparsification functions, uniform, depth-based, and context-preserving, are investigated. The sparsification procedure can combine different sparsification functions while preserving temporal coherency of the rendering. Finally, visibility of each group as observed from the camera is assessed by counting the number of instance IDs in the ID buffer, and this information is fed back into the user interface to form a feedback loop with the user. We have implemented the Volume Conductor in WebGL 2.0 Compute. The resulting system is fast, easy to integrate into existing tools, and applicable to any domain. We assessed the usability of the Volume Conductor by measuring its performance and gathering domain expert feedback. Performance evaluation showed that the Volume Conductor runs at interactive frame rates on consumer-grade hardware, with rendering being the performance bottleneck. Furthermore, the most computationally intensive steps are trivially parallelizable. Experts from the fields of materials science and microbiology provided feedback regarding the usability of the Volume Conductor after incorporating it into their daily workflows. They presented three distinct use cases in which the Volume Conductor was used to analyze fibers and pores in fiber-reinforced polymers, and visualize the intracellular organelles inside a bladder wall cell. The experts from both fields acknowledged the utility and practicality of the Volume Conductor, emphasizing interactive sparsification to reveal the internal structures otherwise occluded by crowdedness, and the intuitiveness of the user interface for group formation and coloring for basic data analysis. Additionally, the microbiology experts highlighted the ability to blend the visibility mask with the raw data volume when visualizing the internal structures inside intracellular organelles. In the second part, we present a crowdsourced user study we have conducted to evaluate depth perception accuracy in crowded volumes under the influence of dynamic perceptual cues. Many visualization techniques advertise enhanced depth perception, a benefit that has been rigorously tested in numerous studies. However, neither crowded volumes nor the degree of crowdedness have yet been considered as variables. The study presented in this thesis fills this knowledge gap by involving crowded volumes from real use cases and considering dynamic perceptual cues, which more accurately reflects real-world scenarios. In contrast to previous studies, our study emphasizes the significance of camera motion as one of the most important monocular depth cues. The experiment consisted of 128 test cases in which four volumes were rendered using four different rendering techniques and lighting setups and shown from eight different camera views. For each test case, the participants were shown a short video of a volume in which the camera performed a subtle circular motion while keeping the volume in focus. When the video stopped playing, three points lying on the structures in the volume were shown, and participants were instructed to estimate the relative depth of the estimation point between the two bounding points using a slider. Their estimates were collected and analyzed using the mean absolute difference from the true values. The results of our study indicate strong connections between the accuracy of depth perception and volume crowdedness, structure shape, and camera positioning. While previous studies identified significant differences in user performance between different rendering techniques when using static images of non-crowded volumes, our results demonstrate that even non-photorealistic rendering techniques without global illumination features can perform as well as photorealistic techniques when camera motion is involved. Moreover, we showed that the volume's content affects depth perception more than the rendering technique or the lighting setup. Furthermore, results revealed that depth estimation accuracy increases with projected area of the instances and partial occlusion between them, whereas it decreases with transparency and crowdedness. The data set composed of test videos and user responses has been made publicly available.

• Language: English
• Keywords: crowded volumes, volume rendering, visibility management, depth perception study
• Work type: Doctoral dissertation
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2024
• Publication date in RUL: 06.11.2024

Secondary language

• Language: Slovenian
• Title: Interaktivna vizualizacija gosto poseljenih volumnov

Abstract

V doktorski disertaciji raziskujemo interaktivno upravljanje vidnosti v gosto poseljenih volumnih, katerih glavna značilnost je vsebnost velikega števila medsebojno prekrivajočih se in pogosto prepletajočih se struktur. V prvem delu predstavimo Volume Conductor, sistem za upravljanje vidnosti v gosto poseljenih volumnih, ki omogoča interaktivno oblikovanje skupin struktur, njihovo barvanje in redčenje. Funkcionalnosti sistema temeljijo na tehničnem prispevku, imenovanem maska vidnosti, ki služi kot vmesna predstavitev, s katero so predstavljeni podatki o vidnosti in optičnih lastnostih struktur, vsebovanih v volumnu. Preprosta in učinkovita struktura maske vidnosti in njene pripadajoče prenosne funkcije omogoča njuno samodejno generiranje z minimalno vpletenostjo uporabnika. Postopek generiranja je poenostavljen s temu prilagojenim uporabniškim vmesnikom, ki podpira zaporedno ali hierarhično organizacijo skupin struktur. Masko vidnosti je moč upodobiti s poljubno tehniko upodabljanja, pod pogojem da omogoča uporabo dvodimenzionalnih prenosnih funkcij. Poleg samostojne rabe je masko vidnosti mogoče po potrebi zliti s surovimi podatki. Tovrstna fleksibilnost omogoča neposredno in enostavno vgradnjo v obstoječa orodja za upodabljanje volumnov. Najpomembnejši vidik sistema Volume Conductor je redčenje struktur, ki ga preko drsnikov za nadzor deleža vidnosti nadzoruje uporabnik za vsako ustvarjeno skupino posebej. Vrstni red redčenja struktur določa redčilna funkcija, ki vsaki strukturi določi pomembnost. Koncept pomembnosti smo preko agregacije posplošili s pomembnosti vokslov na pomembnost strukture. Preizkusili smo tri redčilne funkcije: enakomerno, globinsko in kontekstno-odvisno, ki jih uporabnik lahko v postopku redčenja združuje in pri tem ohranja časovno doslednost upodobitve. V zadnjem koraku sistem ovrednoti vidnost struktur iz vidika kamere, tako da prešteje njihove identifikatorje v medpomnilniku identifikatorjev, nato pa pridobljene informacije vrne uporabniku ob drsnikih za nadzor deleža vidnosti in s tem sklene povratno zanko interakcije. Sistem Volume Conductor smo implementirali s programskim vmesnikom WebGL 2.0 Compute. S tem smo ustvarili sistem, ki je hiter, omogoča enostavno vgradnjo v obstoječa orodja ter ga lahko apliciramo na poljubno domeno. Uporabnost sistema Volume Conductor smo ovrednotili z merjenjem hitrosti izvajanja in zbiranjem povratnih informacij domenskih strokovnjakov. Meritve hitrosti izvajanja so pokazale, da sistem na potrošniških napravah dosega hitrosti, primerne za interakcijo, pri tem pa glavno ozko grlo predstavlja upodabljanje. Najbolj časovno potratne korake je moč trivialno paralelizirati. Strokovnjaki s področij znanosti o materialih in mikrobiologije so po vpeljavi sistema v svoje vsakodnevno delo podali povratne informacije glede uporabnosti sistema. Predstavili so tri primere uporabe, v katerih so s sistemom analizirali vlakna in pore v ojačenih polimerih ter upodobili notranjosti celičnih organelov v celici stene sečnega mehurja. Strokovnjaki z obeh področij so potrdili uporabno vrednost sistema s poudarkom na redčenju za namen razkrivanja zakritih struktur ter intuitivnost uporabniškega vmesnika za oblikovanje skupin in njihovo barvanje za osnovno analizo podatkov. Poleg tega so strokovnjaki s področja mikrobiologije izpostavili zmožnost zlivanja maske vidnosti s surovimi podatki pri upodabljanju struktur znotraj celičnih organelov. V drugem delu predstavimo uporabniško študijo za ovrednotenje natančnosti globinske zaznave v gosto poseljenih volumnih pod vplivom dinamičnih zaznavnih znakov. Študija je bila izvedena v obliki množičenja. Motivacija za izvedbo študije izhaja iz navedb avtorjev mnogih vizualizacijskih tehnik, ki omenjajo izboljšano globinsko zaznavo. Njihove navedbe so bile strogo preverjene v številnih študijah zaznave, toda nobena dosedanja raziskava ni podrobneje obravnavala gostote, niti vsebovala gosto poseljenih volumnov. Študija, predstavljena v pričujoči disertaciji, to vrzel v obsoječih raziskavah zapolni z vključitvijo gosto poseljenih volumnov iz praktičnih primerov uporabe in z upoštevanjem dinamičnih zaznavnih znakov ter s tem natančneje odraža dejansko rabo. V nasprotju s prejšnjimi študijami naša študija poudarja pomen gibanja kamere kot enega najpomembnejših monokularnih globinskih znakov. Študija je obsegala 128 testnih primerov, v katerih so bili upodobljeni štirje volumni s štirimi različnimi tehnikami upodabljanja, vsaka kombinacija pa je bila prikazana iz osmih pogledov kamere. V vsakem testnem primeru smo udeležencem prikazali kratek video, v katerem se je kamera gibala po majhnem krogu okrog začetnega položaja. Po končanem predvajanju so se na strukturah v volumnu prikazale tri točke, udeleženci pa so morali z drsnikom oceniti relativno globino vmesne točke glede na robni točki. Njihove ocene smo zbrali in analizirali povprečno absolutno odstopanje od pravih vrednosti. Rezultati študije razkrivajo močne povezave med natančnostjo zaznavanja globine in gostoto poselitve, oblikami struktur in položajem kamere. Medtem ko so prejšnje študije odkrile signifikantne razlike v natančnosti globinske zaznave med različnimi tehnikami upodabljanja pri uporabi statičnih slik redko poseljenih volumnov, naši rezultati razkrivajo, da lahko celo nerealistične tehnike upodabljanja brez globalne osvetlitve dosegajo primerljive rezultate kot fotorealistične tehnike ob upoštevanju gibanja kamere. Poleg tega smo pokazali, da vsebina volumna na natančnost globinske zaznave vpliva bolj kot tehnika upodabljanja ali osvetlitev volumna. Rezultati so nadalje razkrili, da se natančnost globinske zaznave povečuje s projicirano površino struktur in njihovim delnim medsebojnim prekrivanjem, medtem ko se zmanjšuje s prosojnostjo in gostoto. Zbirko podatkov, uporabljenih in pridobljenih v raziskavi, vključno z videoposnetki in odzivi udeležencev raziskave, smo javno objavili.

• Keywords: gosto poseljeni volumni, upodabljanje volumnov, upravljanje vidnosti, študija globinske zaznave