As companies continue to collect large amounts of data, many of them are looking to leverage that data to grow revenue by monetizing their data. A leading financial services and stock exchange company based in Canada that operates equities, fixed income, derivatives, and energy markets exchanges, was looking for a way to organize and manage their data lake to make data available to analysts and business users to serve use cases such as monetizing data for revenue growth and providing 360-degree customer views to improve customer experience and uncover cross-sell and up-sell opportunities.
Challenge: The company had a complex data environment with data sprawled across the organization. Facing challenges connecting and integrating their siloed data sources, the company wanted to centralize, augment and enrich data sets. They also wanted to provide enriched, trusted data to lines of business in a way that was governed and traceable through a self-service data catalog. This would allow the company to provide broader on-demand accessibility, while ensuring data privacy and security to meet IT and industry regulations.
In order to build their new data environment, they needed to build a flexible and scalable on-demand data lake on AWS. The company ran into some technical challenges trying to integrate several 3rd party technologies and validate their AWS stack.
Solution: Instead of purchasing and integrating several point products, the company selected Zaloni’s platform because it provided the capabilities needed to quickly and effectively build their enterprise data lake on AWS through a single unified platform.
The Zaloni Data Platform (ZDP) on AWS provided end-to-end data management, governance and self-service access to data scientists and business users within the company through a data catalog. ZDP connects to any data source whether it’s on-premises or in the cloud, and ingests a variety of data types into the data lake. The data is cataloged and metadata is captured. Data is then profiled and classified to help automate the process of identifying sensitive or potentially not useful data.
At this point in the process, additional business and technical metadata may be added to further enhance the catalog. Data can continue to be enriched or improved via data quality and data transformations as part of the data management phase.
Finally, data is available for consumption in ZDP through the data catalog, making it easy for analysts and end-users to quickly find trusted data and provision that data to an analytics tool, business intelligence application, cloud data warehouse, or other destination.
The following diagram outlines a reference solution architecture for the cloud-based data lake. Note how data flows through the AWS components and the Zaloni Data Platform thereby creating a cloud-based data lake that is governed and scalable.
Results: ZDP has allowed the company to centralize and manage an AWS data lake giving governed access to analysts and data scientists. ZDP ensured regulatory compliance and reduced risk by applying governance throughout the data supply chain from data source to data consumer. This level of governance has accelerated the use of data within the organization by making it easy for data consumers to access the data they need in a quick self-service manner while maintaining the level of control required by IT.
The flexible and responsive architecture successfully made data a key enterprise asset within the company. Their enterprise data lake on AWS serves as the foundation for use cases such as data monetization, advanced analytics and customer 360.
Learn more about how Zaloni can help you.
Ready to start leveraging your data? Contact us today!