Architectural Background

This project embraces the event-streaming paradigm reflected in the Kappa Architecture:

1. All state changes are recorded as immutable events in an ordered, distributed log
2. Business logic is implemented in distributed processing algorithms that operate on the stream of events
3. Views of the output data are deterministically materialized in appropriate data stores to support querying use cases, e.g. an inverted index for searching, a key-value store for random access, etc.

References: Concepts

• Event Sourcing - Martin Fowler
• Command Query Responsibility Segregation - Martin Fowler
• The Log: What every software engineer should know about real-time data’s unifying abstraction - Jay Kreps
• Turning the Database Inside Out - Martin Kleppmann
• How to Beat the CAP Theorem - Nathan Marz
• Questioning the Lambda Architecture - Jay Kreps
• Why local state is a fundamental primitive in stream processing - Jay Kreps
• Why Avro For Kafka Data? - Jay Kreps

References: Specifics

• Introduction to Kafka
• Event sourcing, CQRS, stream processing and Apache Kafka: What’s the connection?
• Introducing Kafka Streams
• Yes, Virginia, You Really Do Need a Schema Registry
• This entire series from confluent which culminates in a walkthrough of building a streaming system is great.