Streaming
Processing Real-Time Streams in Databricks – Part 2
The first part of this post can be found at Processing Real-Time Streams in Databricks – Part 1. This is the continuation of Part 1 — we won’t repeat the architecture and …
Processing Real-Time Streams in Databricks – Part 1
Databricks is becoming the new normal in data processing technologies in cloud, both Azure and AWS. This is a step-by-step guide to get started on real-time (streaming) analytics …
Introduction to Delta Architecture
In my previous blogs I introduced Kappa and Lambda Architectures. These are big data architectures designed to support massive amounts of data both in real time and at rest. The …
Kappa Architecture – Another Way of Data Processing
Kappa architecture was proposed by Jay Kreps (co-creator of Apache Kafka) as a simplification of the Lambda architecture. The core idea: remove the batch layer entirely and treat …
Introduction to Lambda Architecture
Lambda architecture is a data processing architecture designed to handle massive quantities of data by taking advantage of both batch and stream processing methods. The …