Apache PyIceberg

The athena adapter plugin for dbt (https://getdbt.com)

Example code for running Spark and Hive jobs on EMR Serverless.

Modern serverless lakehouse implementing HOOK methodology, Unified Star Schema (USS), and Analytical Data Storage System (ADSS) principles on Adventure Works. Features programmatic model generation, event-enhanced Puppini bridges, and temporal resolution across DAS/DAB/DAR layers.

Execute SQL queries and manage databases seamlessly with Timeplus. Leverage powerful tools to interact with your data, Kafka topics, and Iceberg tables efficiently. Enhance your data workflows with a user-friendly interface and robust backend capabilities.

Example pipeline to stream the data changes from RDBMS to Apache Iceberg tables

Current Antarctic large iceberg positions derived from ASCAT and OSCAT-2

Tequila: My journey to apache iceberg. You'll definitely need your pinch of salt! opinions are mine!!

Data Warehousing ETL Demo with Apache Iceberg on EMR Local Environment

Python script that will update S3 file paths in Iceberg metadata files (metadata.json + AVRO)

Dagster + DBT + Spark + Iceberg

A tool for tables migration from Apache Iceberg to S3 Tables

Kafka streaming job from iomete. This streaming job copies data from Kafka to Iceberg.

The power of duckdb with the ease of pandas

Implement a big data stack with Apache Kafka for data ingestion, Apache Flink for stream processing, Druid for OLAP querying, Iceberg for data versioning, and Apache Superset for visualization

Creating a modern data stack in Kubernetes with open-source products, both on-premises and cloud-agnostic, is an increasingly popular approach. By leveraging Kubernetes for container orchestration, you can deploy and manage data processing, storage, and analysis tools more efficiently.

This project benchmarks query performance between Databricks and Snowflake on Iceberg and Delta tables, allowing for detailed comparison and analysis. It supports customizable queries, logging of execution metrics, and fetches Snowflake query history to assist with performance optimization and cost analysis.

A project to create a Data Lake House

Engaging, interactive visualizations crafted with Streamlit, seamlessly powered by Apache Flink in batch mode to reveal deep insights from data.

A compact project showcasing local data lake setup using Docker, Mage, Spark, MinIO, Iceberg, and StarRocks. Ideal for learning modern data engineering practices.