Video object segmentation and tracking are mature fields, yet modern models
such as SAM2 track objects mainly by their appearance. When an object
substantially changes shape during a sequence, for example while being cut,
broken, or kneaded, it falls apart into several pieces and tracking often fails.
In this thesis we adapt models of the Segment Anything family for object
segmentation through such transformations. We fine-tune SAM2 on the
VOST dataset using the model’s existing training pipeline, while for SAM3 we
develop our own video-training procedure for the tracker, in which gradients
flow through the temporal memory so that it adapts to the task during training.
Fine-tuning improves both models: the mean region overlap (J) rises from
0.471 to 0.539 for SAM2 and from 0.536 to 0.614 for SAM3, with the largest
gains precisely on the sequence sections where transformations happen. Our
fine-tuned model also achieves the best result on the VOTSt2025 challenge.
We also evaluate transfer to ordinary objects and show that specialising for
disintegrating objects reduces accuracy on them.
|
