As the name suggests, transient clusters are compute clusters that automatically shut down and stop billing when processing is finished. However, using this cost-effective approach has been an issue because metadata is automatically deleted by the cloud provider when a transient cluster is shut down.
This is noteworthy because metadata is the key to getting value from big data. Therefore, most enterprises have opted to pay for persistent compute across the board in order to maintain the metadata. How can enterprises leverage transient clusters for cost-savings and maintain their metadata?
To solve this challenge, enterprises either choose a persistent cloud presence or deploy an intelligent data lake management platform like Zaloni's. Learn even more about how our platform can help leverage transient clusters or contact us for more information.
About the Author