Generative adversarial model based augmentation of angiographic images for improved detection of intracranial aneurysmsKRIŽAJ, DAVID (Avtor) Špiclin, Žiga (Mentor) aneurysm detectioncomputer visionconvolutional neural networksgenerative adversarial networksdataset augmentationThe risks associated with intracranial aneurysms have motivated research in the past and still continue to do so today, this thesis being no exception. With a reported incidence of around 3.2% (in the absence of risk factors) and with the associated risk of rupture rising steeply in correlation with aneurysm size, the value of early detection, evaluation and potential treatments is crucial. This thesis focuses on computer-aided detection of intracranial aneurysms, which is a computer vision task. Computer vision as a field of research has progressed immensely in the past 10 years or so as a result of the availability and development of deep convolutional neural networks and associated hardware technology. It has become the default approach in most detection and segmentation tasks where non-trivial analytical approaches were employed prior to this era. However, as machine learning approaches require adequately sized training data, the same methods may excel at common problems, where learning sets are easily obtainable and plentiful, but show poor performance in niche fields where training data is scarce. Multiple approaches can be taken in order to tackle this issue, such as fine-tuning the existing models and/or augmentation of available data through the deep learning based creation of artificial data. The lack of publicly available medical image datasets is a problem that must be frequently tackled when applying deep learning machine vision principles in medical image analysis. This is due to multiple reasons such as policies on preserving patient privacy, cost of expert's work when annotating/manually segmenting images and a general lack of initiative to produce large public datasets for scientific use just to name a few. We try to address this issue by proposing a problem-specific dataset augmentation technique, which is itself based on generative convolutional neural networks. Our proposal was to take a scarce dataset of 3D magnetic resonance images with corresponding vasculature and aneurysm segmentations, create several 2D projections at locations of interest from multiple view and train a general adversarial network (GAN) on that dataset so that the network would learn to impaint a randomly shaped and textured aneursym into the regions of healthy vasculature. This would allow us to transform a large number of healthy datasets into datasets depicting the pathology of interest, which would as such be suitable for training a standard U-net based detector network. We also opted for a U-net detector variation that employed a combination of the Tversky index and focal loss concepts in order to increase the performance for highly unbalanced datasets, a property inherent to our aneurysm datasets, where we are normally interested in a very small portion of an input image. Our GAN architecture of choice was an of the shelf Cycle-GAN with some customized cost function terms. Learning adversarial networks for a given problem has proven to be very challenging. In the case of the generative network, the impainted aneurysms were meaningful, but their fusion with the vessels was not optimal in all cases. Therefore, we applied additional processing of the input grayscale image of the aneurysm-free vessel and the synthetically generated aneurysm image to ultimately produce useful images depicting aneurysms and vessels for the synthetic training set. Concretely, the problem turned out to be that the images produced in this way varied greatly depending on the extracted patch of the vascular images at the input to the network. With standard morphological operations, Poisson fusion of the input and the synthetic images and manual visual evaluation, we managed to discard all textural and anatomical irregularities and thus obtain the final synthetic set. The process of creating the final set of training images limited the ability to scale the size of the training set due to involved manual inspection. We limited ourselves to just over 2000 synthetic images versus around 6000 real projections and trained our detector network on combinations of both synthetic and real images at various ratios to determine the contribution of synthetic images to the learning convergence and performance of the aneurysm detector model. From the convergence curves of the loss functions during learning and the results of the evaluation of the detector model, we concluded that learning on only synthetic images is insufficient for adequate detector learning. From the results of further experiments, we concluded that the addition of synthetic images to the real ones in the training set did not degrade the performance of the detector and in some cases even improved the convergence and generalization of the detector model.20232023-02-17 13:35:24Magistrsko delo/naloga144387sl