Details

Warden Ship: interaktivni vmesnik za raziskovanje umetniških arhivov s semantičnim iskanjem in hkratno klasifikacijo sloga, žanra in avtorja
ID Breskvar, Matej (Author), ID Bovcon, Narvika (Mentor) More about this mentor... This link opens in a new window, ID Meden, Blaž (Comentor)

.pdfPDF - Presentation file, Download (9,46 MB)
MD5: 4FF4247098773BEF85794D68676B7BC6

Abstract
Digitalizirane umetniške zbirke so prerasle zmožnosti klasičnega iskanja po metapodatkih, splet pa hkrati preplavljajo strojno generirane podobe. V delu predstavimo Warden Ship, interaktivno aplikacijo za raziskovanje umetnostnozgodovinske zbirke s pomočjo umetne inteligence. Diplomsko delo je dvodelno: prvi del je interaktivni grafični vmesnik, drugi pa zaledje z modeli strojnega učenja. Šest enakovrednih funkcionalnosti – semantično iskanje, klasifikacijo sloga, žanra in avtorja, karti ustvarjalnosti in vplivov, zemljevid zgodovine umetnosti ter filtre po vizualnih lastnostih – aplikacija upodobi kot prizore iz skupnega pomorskega sveta, povezane z animiranimi prehodi. Čelni del je izdelan z ogrodjema React in React Three Fiber, zaledje s Flaskom ter zbirko MongoDB z GridFS. Jedro sestavljajo model CLIP za besedilno-slikovno iskanje, doučeni slikovni transformer za hkratno napovedovanje sloga, žanra in avtorja ter štirje kanali klasičnega računalniškega vida za filtriranje po vizualnih lastnostih. Modele ovrednotimo na izločeni testni množici 12.680 slik: doučena različica ViT-L/336 doseže točnost 77,0 % pri slogu (27 razredov), 86,3 % pri žanru in 97,3 % pri avtorju, CLIP brez doučitve pa na istih razredih le 25,8 %, 57,7 % in 56,0 %. Tri komponente nato prilagodimo umetniški domeni oziroma jih na novo naučimo. Kontrastno doučeni CLIP izboljša točnost pri prepoznavanju sloga na 55,3 % in žanra na 79,8 %, metrična vložitev ArcFace pa poveča natančnost@1 pri slogu z 41,6 % na 61,0 %. Nova regresijska glava nad vložitvami CLIP letnico nastanka dela oceni s povprečno absolutno napako 31 let v primerjavi z naivno osnovo 73 let. Testne točnosti se tesno ujemajo z validacijskimi, kar potrjuje dobro posploševanje, rezultati pa presegajo zastavljene cilje.

Language:Slovenian
Keywords:Semantično iskanje slik, CLIP, hkratna klasifikacija, doučitev modelov, računalniška analiza umetnosti, interaktivna vizualizacija
Work type:Bachelor thesis/paper
Typology:2.11 - Undergraduate Thesis
Organization:FRI - Faculty of Computer and Information Science
Year:2026
PID:20.500.12556/RUL-187592 This link opens in a new window
COBISS.SI-ID:304203267 This link opens in a new window
Publication date in RUL:11.09.2026
Views:187
Downloads:98
Metadata:XML DC-XML DC-RDF
:
Copy citation
Share:Bookmark and Share

Secondary language

Language:English
Title:Warden Ship: an interactive interface for exploring art archives with semantic search and multi-task classification of style, genre and artist
Abstract:
Digitised art collections have outgrown classic metadata search, while the web is being flooded with machine-generated imagery. This work presents Warden Ship, an interactive application for exploring an art-historical collection with the help of artificial intelligence. The thesis has two parts: an interactive graphical interface, and a backend of machine-learning models. The application renders six equal features – semantic search, classification of style, genre and artist, creativity and influence charts, a map of art history, and visual-attribute filters – as scenes of a shared nautical world, connected by animated transitions. The frontend is built with React and React Three Fiber; the backend uses Flask with MongoDB and GridFS. At its core are a CLIP model for text-to-image search, a fine-tuned vision transformer that predicts style, genre and artist jointly, and four classical computer-vision channels for filtering by visual attributes. The models are evaluated on a held-out test set of 12,680 images: the fine-tuned ViT-L/336 variant reaches 77.0 % accuracy for style (27 classes), 86.3 % for genre and 97.3 % for artist, whereas zero-shot CLIP reaches only 25.8 %, 57.7 % and 56.0 % on the same classes. Three components are then adapted to the art domain or trained anew. Contrastively fine-tuned CLIP lifts style accuracy to 55.3 % and genre accuracy to 79.8 %, while an ArcFace metric embedding raises precision@1 for style from 41.6 % to 61.0 %. A new regression head over CLIP embeddings estimates the creation year of a work with a mean absolute error of 31 years, against a naive baseline of 73 years. Test accuracies closely track validation, confirming good generalisation, and the results exceed the targets that were set.

Keywords:Semantic image search, CLIP, Multi-task classification, Fine-tuning, Computational art analysis, Interactive visualisation

Similar documents

Similar works from RUL:
Similar works from other Slovenian collections:

Back