Data & Innovation • Telecommunications • AWS
Cloud Data META Architecture
Implementation of the first cloud data META architecture on AWS with association to a set of use cases.
Background and Requirements
The team of a major telecommunications company, after a rigorous process of technological evaluation, decided to implement the first AWS cloud data META architecture.
This decision was made with the aim of associating it with a set of use cases selected by the internal team, thereby formalizing and making data flows available within the framework of data management, governance, and efficient delivery.
Project Phases
Within the implementation phase of the META architecture, enabling AWS Direct Connect and integrating with various clouds, data lakes, or on-premise data centers was considered.
First Stage
Direct Connect Implementation
Second Stage
Integration with Various Clouds, Data Lake, and On-Premise Data Centers
Implementation and Results Achieved
The initial phase of the project included a "Lake House" architecture designed to cover several critical use cases, including:
-
- Self-service reporting application.
-
- Analytical layer for processing postpaid and fixed services, among others.
The main objective of the first phase was to achieve rapid results, prioritizing the aforementioned use cases. This was accomplished through the implementation of DataOps strategies, which enabled standardized orchestration of tools, code, and infrastructure, ensuring fast delivery of high-quality data.
For the processing of postpaid and fixed data cubes, all countries covered by the client were considered.
The flows previously implemented with prepaid were replicated on AWS technology to create the necessary BI dashboards, thus visualizing the entire client operation. The new flows for postpaid and fixed cubes were accompanied by the optimization of current models once the pipelines were configured on AWS.
The same approach was applied to the processing flows of value-added services, operational Pretups reports, mobility flows, and mediation/interconnection. For postpaid and fixed flows, components such as activations, deactivations, inventory, billing cycle, and traffic were included for the postpaid cube, and activations, deactivations, and revenues for the fixed cube. These flows were made available and configured on AWS services, with model optimization deferred to a later stage. Additionally, the solution was integrated with existing cloud resources, such as the networking layer and compliance.
Regarding the operational reports flows of Pretup, mobility model, mediation/interconnection reports, and the D10 flow, exploration, understanding, flow, and their adaptation to the AWS environment were considered. This involved from the data lake layer to storage within the data warehouse for consumption.
Self-Service Reporting Application
The project also included the development of a self-service web portal hosted on AWS cloud, aimed at automating various requirements of the client's internal customers.
This portal allows for the generation of reports based on a set of filters, returning data sorted by a selected time period.
Portal Features
Authentication: The portal features authentication for selected users.
Transaction Search: Allows searches by various criteria, such as subscriber name, phone number, IMEI number, identification number (DPI), NIT number, and transaction date.
Report Generation: Based on data filtered by selected dates.
Guardado de Reportes: Permite guardar las consultas generadas.
Information Download: Users can download information in compatible format (CSV) or request download via email.
Audit: Includes storing requests made in a persistent database, complying with regulations.
Conclusions
During the execution of this project, Nubiral demonstrated the ability to develop a data architecture that enabled the client to address various challenges presented by different business areas. The primary priority was to ensure data integrity securely, for which AWS Direct Connect allowed establishing a secure communication channel with dedicated connections.
The client has a significant portfolio of interdependent data sources running in their on-premise data centers or across various public clouds.
For a long time, they have sought to migrate these workloads to the public cloud. Testing was crucial because, in moving these workloads to the cloud, it was essential to ensure a seamless experience for end users (business data users).
AWS Local Zones resolved this issue by providing ultra-low latency communication between different systems in the local zone and on-premise facilities. This facilitated the incremental migration of new information sources, drastically simplifying migrations and simultaneously reducing any business risk with continuous hybrid implementations.
