Why Avocado Datalake?
Avocado Datalake offers comprehensive data ingestion and management solutions designed to streamline the creation of data lakes and data warehouses across major cloud providers like AWS, GCP, and Azure. The platform provides ready-to-use, scalable codebases for ingesting structured, semi-structured, and unstructured data sources into centralized data lakes, supporting formats such as Apache Iceberg, Hudi, Delta Lake, and XTable. It enables organizations to efficiently extract data from RDBMS, NoSQL, and file-based sources, transforming and loading it into cloud storage and data warehouses like Redshift, BigQuery, and Snowflake.
Supported products as of now
- Supports ingestion from RDBMS, NoSQL, and unstructured data sources
- Provides scalable ETL pipelines for data lake and warehouse synchronization
- Integrates with centralized data catalogs and access control systems
- Compatible with multiple cloud providers and open table formats
Full support of reading various sources like structured, semi-structured, and unstructured data and ingestion into a centralized Data Lake in cloud storage
How it works
The platform offers configurable pipelines that extract data from various sources, transform it as needed, and load it into centralized data lakes or warehouses. It includes support for open data formats and catalog interoperability, ensuring easy discovery and secure access control. The solution is designed for data engineers, data scientists, and BI teams seeking efficient data management and analytics infrastructure.
- Pipelines infrastructure is built using Terraform and can be deployed on AWS, GCP, and Azure.
- Pipeline ETL codebase is built using Apache Spark + Scala and fat Jar is created for each pipeline.
- We do support PySpark + Python in Databricks + Snowflake as well.
- Data catalog management using terraform and we do support Unity catalog in Databricks, AWS Glue Data Catalog, Azure Data Catalog, and Google Cloud Data Catalog.
- Permission on the datacatalog is managed by terraform as a code.
Pricing is flexible, with options for free, freemium, or paid plans, depending on organizational needs. The platform is ideal for enterprises looking to accelerate their data lake and warehouse setup, improve data accessibility, and enable advanced analytics and machine learning applications.
Need data lake or data warehouse solution implementation or bootstrapping in your organization?
Visit our product pages for more information for below ingestion pipelines:
