The topic of the bachelor's thesis delves into the realm of collecting and analyzing log entries from various systems and their consolidation. The thesis analyzes different ways of tracking application and database systems, aiming to find the most qualitative method of consolidating log entries in a unified centralized system. The system must be robust for various types of applications, from web servers that log their entries into databases, to systems that store their records in log files. If one wishes to introduce a new type of log entry into the system, the integration method should be as easy and quick as possible, regardless of its composition. The system must be scalable to a potentially large number of log entries, so Apache Lucene is used as the basis for indexing and searching. It operates on technology that is similar to NoSQL and is suitable for searching through unstructured data using JSON-formatted documents. The entire system consists of three major parts. The first is data extraction from various locations to a common place. The second part is data analysis, which transforms them from their basic form into a structure equivalent to all other data types. The third and final part is displaying the data in a user-friendly manner to the end user. We also implemented a centralized system using the OpenSearch tool, which handles the search and analysis of records, Logstash, which manages the data flow from their original location to the OpenSearch system and takes care of the appropriate parsing of log entries, and the OpenSearch Dashboards system, which offers a graphical interface for displaying records and visualizing analyses.
|
