Process Common Crawl data with Python and Spark

demo applications that show how to deploy offline feature engineering solutions to online in one minute with fedb and nativespark

A SparkSQL formatter based on https://github.com/zeroturnaround/sql-formatter, with customizations and extra features.

A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning

spark机器学习

Data Mining Census ECON using Apache Spark

Apache Hadoop

Structured Spark Streaming with Apache Kafka and Twitter

Big Data Project - SSML - Spark Streaming for Machine Learning

Performance Optimizations and Benchmarks for Distributed SQL Engines

SCD2 on Databricks using Spark and Delta with Change Data Feed

The project harnessed an ETL multi-hop architecture, ingesting data from the Ergast API into a storage backed by Azure Data Lake. The process involved weekly ingestion of bronze layer data as cutover and delta files. Raw data, in varied formats, was transformed using Azure Databricks PySpark notebooks into enriched Silver and Gold layers.

📊 📑This project provides a step-by-step big data analytics applied in the retail industry through the use of a variety of big data technologies. such as HDFS, Hive and Spark..

Spark application using python API to run analytics using CSV and JSON data

DATABRICKS PROJECT- END TO END SALES ANALYSIS

Repositório para processamento e modelagem dimensional dos dados das eleições utilizando Spark no Databricks Community

Extração de dados do site Fundamentus utilizando a biblioteca Fundamentus

Spark 1.4 and 2.0 tests and exercises.