Implementacija vmesnika REST za sporočila Kafka z uporabo pridobivanja podatkov ob upoštevanju strežniške razporeditve

Tomplak, Anej (author), Poženel, Marko (mentor)

V diplomski nalogi bom opisal postopek implementacije vmesnika tipa Representational State Transfer (REST, slov. prenos reprezentacij stanja) ter Application Programming Interface (API, slov. programski vmesnik). To je spletni programski vmesnik za izmenjavo podatkov prek standardnih spletnih zahtevkov. Vmesnik, ki sem ga poimenoval kafka-offset-reader, omogoča branje in pisanje preko Apache Kafke, distribuirane platforme za shranjevanje dogodkov in obdelavo podatkovnih tokov, kar omogoča natančen nadzor sporočil v distribuiranem tokovnem procesiranju. Vmesnik temelji na posredniškemu strežniku Kafka REST Proxy, ki omogoča dostop do sistema Kafka prek standardnih spletnih zahtevkov protokola HyperText Transfer Protocol (HTTP, slov. protokol za prenos hiperteksta). Dodane so nekatere pomembne izboljšave, kot je pridobivanje podatkov ob upoštevanju fizične razporeditve strežnikov (angl. rack aware fetching), uporabljanje bazena povezav za učinkovitejše upravljanje omrežnih zahtevkov ter druge optimizacije. Te spremembe omogočajo občutno izboljšanje prepustnosti pri velikem številu sočasnih zahtev, kjer se zmogljivost obstoječega vmesnika za dostop do Kafke preko REST začne izrazito slabšati.

• Language: Slovenian
• Keywords: REST, API, Apache Kafka, Kafka REST Proxy, fizična razporeditev strežnikov, bazen povezav.
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2026
• PID: 20.500.12556/RUL-185704
• COBISS.SI-ID: 288576003
• Publication date in RUL: 18.08.2026

Secondary language

• Language: English
• Title: Implementation of a REST interface for Kafka Message Retrieval with Server-Aware Data Allocation

Abstract

In this thesis, I describe the process of implementing a Representational State Transfer (REST) Application Programming Interface (API) designed as a web-based interface for data exchange using standard web requests. The implemented service, named kafka-offset-reader, enables reading from and writing to Apache Kafka, a distributed platform for event storage and stream processing. The interface is based on the Kafka REST Proxy, which provides access to Kafka through standard HyperText Transfer Protocol (HTTP) requests. The implemented solution introduces several significant enhancements, including rack-aware data fetching, which takes into account the physical distribution of servers, as well as connection pooling for more efficient management of network operations. Additional performance optimizations are also incorporated. These improvements result in substantially increased throughput when handling a large number of concurrent requests, a scenario in which the performance of the existing Kafka REST Proxy begins to degrade noticeably.

• Keywords: REST, API, Apache Kafka, Kafka REST Proxy, physical distribution of servers, connection pool.