Real-time Data Streaming using Kafka, Kinesis, and RabbitMQ

Аннотация

In the present work a comprehensive OPEN ACCESS comparative analysis of the three leading platforms for SUBMITED 17 July 2025 organizing message streaming — Apache Kafka, Amazon ACCEPTED 28 July 2025 Kinesis and RabbitMQ — is performed with the aim of PUBLISHED 12 August 2025 VOLUME Vol.07 Issue 08 2025 identifying their architectural features, operational strengths and limitations under conditions of peak loads CITATION and stringent latency requirements. The study relies on Vladyslav Vodopianov. (2025). Real-time Data Streaming using Kafka, a comprehensive methodological approach, including a Kinesis, and RabbitMQ. The American Journal of Engineering and Technology, 7(8), 71–77. systematic review of current scientific publications, the https://doi.org/10.37547/tajet/Volume07Issue08-08 conduct of comparative performance measurements in COPYRIGHT laboratory settings and the synthesis of practical case © 2025 Original content from this work may be used under the terms studies of integrating the systems under consideration of the creative commons attributes 4.0 License. into real IT landscapes. The obtained results demonstrate that a reasoned choice of platform for stream processing depends on a multitude of interrelated factors: the volume of messages processed, the required throughput metrics and maximum response time, the preferred deployment model (on- premises solution, cloud service or their hybrid), the capabilities for seamless integration with existing services and infrastructure, as well as the project’s budgetary constraints. On the basis of the conducted analysis a unified decision-making methodology is proposed for selecting tools for streaming data processing, adapted to the tasks of data engineers, distributed systems architects and researchers of high- performance information platforms. The material is of practical interest to specialists designing fault-tolerant and scalable distributed message queues, as well as to experts in real-time analytics and cloud solution developers seeking to gain a deeper understanding of the architectural schemes and methods for optimizing throughput applied in Kafka, Kinesis and RabbitMQ. In addition, the research results may be useful to scientists in the field of distributed computing and the Internet of Things, focusing on the theoretical foundations and The American Journal of Engineering and Technology 71 https://www.theamericanjournals.com/index.php/tajet practical aspects of constructing reliable event-data market segmentation by technology—Complex Event pipelines. Processing (CEP), Event Stream Processing (ESP), and data visualization—as well as by application domain, Keywords: streaming data processing, Apache Kafka, including fraud detection, asset management, and risk Amazon Kinesis, RabbitMQ, big data, distributed management. The authors construct long-term systems, low latency, high bandwidth, data architecture, development scenarios through 2029, relying on platform comparison. deployment statistics and growth rates in key sectors.