Big Data Analytics for Telco Using Open Source Data Pipeline Architecture: Results of SLR and Architecture Recommendation

Abstract

This research paper focuses on some of the important big data analytics architectures (BDA) for telecommunication sectors. Telecom companies handle a huge volume of data (terabytes to petabytes) on daily basis and there is a need to filter meaningful data from this bulk data. There are multiple advancements in recent years which are helpful in deriving these meaningful insights. As a part of our research work, we have initially conducted a SLR and have filtered 36 articles which were later categorized based on use cases, frameworks, white papers, and experimental results. We have identified the research gap exists as there are no papers focusing on high-level cloud-native opensource computing platforms like Kubernetes for Telco data analytics. We recommend a full-fledged, real-time, cloud-native, and distributed data pipeline architecture using opensource components like Apache Kafka, Kubernetes computes and so on. Our open source and modular data pipeline architecture is completely based on open source big data analytics components using a public cloud infrastructure like AWS and Kubernetes pods. Though there are resources that separately talks about Kubernetes for resource management and data pipeline architecture using Apache Kafka, integrating both is of great value and need of the hour which our proposed architecture can deliver.