Event Streams in Action
Real-time event systems with Kafka and Kinesis
Valentin Crettaz, Alexander Dean
- 344 pages
- English
- ePUB (adapté aux mobiles)
- Disponible sur iOS et Android
Event Streams in Action
Real-time event systems with Kafka and Kinesis
Valentin Crettaz, Alexander Dean
Ă propos de ce livre
Summary Event Streams in Action is a foundational book introducing the ULP paradigm and presenting techniques to use it effectively in data-rich environments.Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the Technology Many high-profile applications, like LinkedIn and Netflix, deliver nimble, responsive performance by reacting to user and system events as they occur. In large-scale systems, this requires efficiently monitoring, managing, and reacting to multiple event streams. Tools like Kafka, along with innovative patterns like unified log processing, help create a coherent data processing architecture for event-based applications. About the Book Event Streams in Action teaches you techniques for aggregating, storing, and processing event streams using the unified log processing pattern. In this hands-on guide, you'll discover important application designs like the lambda architecture, stream aggregation, and event reprocessing. You'll also explore scaling, resiliency, advanced stream patterns, and much more! By the time you're finished, you'll be designing large-scale data-driven applications that are easier to build, deploy, and maintain. What's inside
- Validating and monitoring event streams
- Event analytics
- Methods for event modeling
- Examples using Apache Kafka and Amazon Kinesis
About the Reader For readers with experience coding in Java, Scala, or Python. About the Author Alexander Dean developed Snowplow, an open source event processing and analytics platform. Valentin Crettaz is an independent IT consultant with 25 years of experience. Table of Contents
PART 1 - EVENT STREAMS AND UNIFIED LOGS
- Introducing event streams
- The unified log 24
- Event stream processing with Apache Kafka
- Event stream processing with Amazon Kinesis
- Stateful stream processing
PART 2- DATA ENGINEERING WITH STREAMS
- Schemas
- Archiving events
- Railway-oriented processing
- Commands
PART 3 - EVENT ANALYTICS
- Analytics-on-read
- Analytics-on-write
Foire aux questions
Informations
Part 1. Event streams and unified logs
Chapter 1. Introducing event streams
- Defining events and continuous event streams
- Exploring familiar event streams
- Unifying event streams with a unified log
- Introducing use cases for a unified log
- The people or things that they interact with on a daily basisâfor example, customers, the Marketing team, code commits, or new product releases
- The software and hardware that they use to get stuff done
- Their own daily inbox of tasks to accomplish
A company is an organization that generates and responds to a continuous stream of events.
- Fresher insightsâ A continuous stream of events represents the âpulseâ of a business and makes a conventional batch-loaded data warehouse look stale in comparison.
- A single version of the truthâ Ask several coworkers the same question and you may well get different answers, because they are working from different âpotsâ of data. Well-modeled event streams replace this confusion with a single version of the truth.
- Faster reactionsâ Automated near-real-time processing of continuous event streams allows a business to respond to those events within minutes or even seconds.
- Simpler architecturesâ Most businesses have built up a birdâs nest of bespoke point-to-point connections between their various transactional systems. Event streams can help to unravel these messy architectures.
1.1. Defining our terms
1.1.1. Events
Figure 1.1. The precision on the timestamps varies a little, but you can see that all four of these events are discrete, recordable occurrences that take place in the physical or digital worlds (or both).
- A description of the ongoing state of somethingâ The day was warm; the car was black; the API client was broken. But âthe API client broke at noon on Tuesdayâ is an event.
- A recurring occurrenceâ The NASDAQ opened at 09:30 every day in 2018. But each individual opening of the NASDAQ in 2018 is an event.
- A collection of individual eventsâ The Franco-Prussian war involved the Battle of Spicheren, the Siege of Metz, and the Battle of Sedan. But âwar was declared between France and Prussia on 19 July 1870â is an event.
- A happening that spans a time frameâ The 2018 Black Friday sale ran from 00:00:00 to 23:59:59 on November 23, 2018. But the beginning of the sale and the end of the sale are events.
1.1.2. Continuous event streams
Figure 1.2. Anatomy of a continuous event stream: time is progressing left to right, and individual events are ordered within this time frame. Note that the event stream is unterminated; it can extend in both directions beyond our ability to process it.
- The start of the stream may predate our observing of the stream.
- The end of the stream is at some unknown point in the future.
1.2. Exploring familiar event streams
- Transactional systemsâ Many of these respond to external events, such as customers placing orders or suppliers delivering parts.
- Data warehousesâ These collec...