Diffusion Models in Virtual Clothing Try-On

Cvetković, Ana (author), Peer, Peter (mentor), Lampe, Ajda (comentor)

Virtual try-on aims to synthesize new garments on a person image while preserving identity, pose, and scene context for applications such as e-commerce and design exploration. This thesis extends DiCTI, a text-guided diffusion inpainting pipeline, to reduce pose drift, improve mask coverage for loose garments, and enable finer fabric control than text alone. To address the limitations the proposed PMFR-DiCTI integrates DensePose-based ControlNet conditioning for pose preservation, a union mask combining DensePose and SegFormer clothing segmentation for more reliable garment masking, fabric reference image conditioning via IP-Adapter and region-selective editing. Evaluation on a VITON-HD subset (2250 generated images) shows improved realism and pose consistency, including a 57% reduction in KID metric and a 53% improvement in pose distance compared to the baseline. A user study with 30 participants further favors PMFR-DiCTI outputs across pose preservation, fabric accuracy, and garment structure with statistically significant preference. This work effectively addresses DiCTI's limitations while preserving zero-shot generalization, providing a practical framework for controllable garment synthesis.

• Language: English
• Keywords: diffusion models, virtual try-on, stable diffusion, ControlNet, IP-Adapter
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2026
• PID: 20.500.12556/RUL-181104
• COBISS.SI-ID: 276231939
• Publication date in RUL: 25.03.2026

Secondary language

• Language: Slovenian
• Title: Difuzijski modeli v virtualnem pomerjanju oblačil

Abstract

Virtualno pomerjanje je namenjeno sintetiziranju novih oblačil na sliki osebe ob hkratnem ohranjanju identitete, poze in konteksta prizora za uporabo v e-trgovini in raziskovanju oblikovanja. To diplomsko delo nadgradi DiCTI, besedilno vodeno difuzijsko metodo za zapolnjevanje slike (angl. inpainting), z namenom zmanjšanja sprememb poze, izboljšanja pokritosti maske pri ohlapnih oblačilih ter omogočanja natančnejšega nadzora nad videzom tkanine kot ga omogoča zgolj besedilo. Za odpravo teh omejitev predlagani PMFR-DiCTI vključuje pogojevanje s ControlNet na osnovi DensePose za ohranjanje poze, združeno masko DensePose in SegFormer segmentacije oblačil za zanesljivejše maskiranje ter pogojevanje z referenčno sliko tkanine prek IP-Adapterja in regijsko selektivno urejanje. Ovrednotenje na podmnožici VITON-HD (2250 generiranih slik) pokaže izboljšan realizem in skladnost poze, vključno z 57% zmanjšanjem metrike KID ter 53% izboljšanjem razdalje poze v primerjavi z izhodiščno metodo. Uporabniška študija s 30 udeleženci dodatno potrdi statistično značilno preferenco izhodov PMFR-DiCTI pri ohranjanju poze, natančnosti tkanine in strukturi oblačila. Delo učinkovito naslovi omejitve DiCTI, pri tem pa ohrani posploševanje brez dodatnega učenja ter zagotovi praktičen okvir za nadzorovano sintezo oblačil.

• Keywords: difuzijski modeli, virtualno pomerjanje, stabilna difuzija, ControlNet, IP-Adapter