Whether you’re planning for a data lake implementation or have a proof-of-concept (POC) in place, one thing is clear: You must hydrate the data lake before it can be ready for production. Depending on the complexity and volume, data ingestion and data preparation can take at least six months.
What if you had a solution to cut that timeline by 75%?
Zaloni Ingestion Factory is a solution that can help you successfully hydrate and organize your data lake within eight weeks.
The solution offers the best of Zaloni services and platforms and frees you from the delay of a POC. Our services experts develop the architecture for a production-ready data lake. You immediately get to use the metadata management, data governance, security and privacy capabilities of the enterprise-grade Zaloni Data Platform.
How it works
By simplifying and automating data lake ingestion and data preparation processes, Zaloni’s Ingestion Factory solution will make production-ready data lakes a reality for more businesses.
The solution is delivered in three stages:
- Discovery and configuration: The Zaloni team works with the customer to define sources, connections, system architecture, and ingest strategies.
- Predefined workflows: A set of workflows are created to automate the ingestion and registration of data sources.
- Data lake hydration: Zaloni operationalizes the ingestion process by running an initial hydration of all sources included in the Ingestion Factory package.
The key features of Zaloni’s Ingestion Factory solution include:
- High-volume, “industrial-strength” ingestion leveraging parallel ingestions; ingestion rate is customized to give clients the best ingestion throughput based on the cluster and network configuration
- Automation of first-time and incremental ingestion of any data type from any data source across an organization into the data lake
- For relational database management system (RDBMS) ingestion to Hadoop, ZDP automatically creates Hive tables for each of the source RDBMS tables
- “Ingestion accelerator” technology that enables repeatable and reliable ingestion processes
- Application of metadata to the data upon ingestion for a managed data lake
- Total visibility into the ingestion process to enable more “hands-off” IT management
- Support for different refresh frequencies, including one-time historical load, full refresh, incremental refresh, and change data capture (CDC); Zaloni's data lake management platform also supports slowly changing dimensions (SCDs) and SQL override options to load data
We’re offering a tiered pricing model, based on the complexity and quantity of data sources to be ingested. For more information, read the Ingestion Factory whitepaper.
About the AuthorMore Content by Kelly Hopkins Schupp