Self-supervised anomaly detection in production log streamsMartinčič, Tomaž (Avtor) Žitnik, Slavko (Mentor) García Faura, Álvaro (Komentor) Natural language processinganomaly detectionproduction logsmachine learningself-supervised learningLog-based anomaly detection solutions are needed to effectively analyze and interpret vast amounts of generated log data, uncover hidden patterns, and predict system anomalies, enhancing operational efficiency, ensuring system security, and reducing potential downtime. In recent times, there has been development in the field of automatic anomaly detection using machine learning methods. In this work, we extended LogBERT, a well-known method in the field, into a hierarchical transformer by including a pre-trained language model to obtain semantic embeddings of log templates. We provide richer information and avoid the out-of-vocabulary problem that is faced with the original LogBERT method. We introduce a novel method called SemLogBERT. We found out that the results presented in most of the SOTA methods severely overestimate models' performance. We evaluated LogBERT and SemLogBERT in a more realistic scenario, where it improved the performance on some of the standard benchmark datasets.20232023-09-07 11:18:33Magistrsko delo/naloga149498sl