- This event has passed.
ksqlDB and the patterns and anti-patterns of streaming
March 4 @ 6:00 pm - 8:00 pm
Join us on Slack (https://kafka-london-invites.herokuapp.com/)
6.00pm – Doors open, Food + Drinks, Network
6.20pm – Talk – “The many patterns and anti-patterns of Streaming” with Richard Noble and Francesco Nobilia from Babylon (https://babylonhealth.com/)
Following in the veins of the infamous Gang of Four, and hopping on the pattern bandwagon, this talk aims to demystify a number of core streaming patterns.
At Babylon, we have a whole lot of smart engineers, and as we all know, smart engineers question things, so we’ve had to think hard about our approaches.
We’ll start from Messages vs Event Logs and CQRS, how when and where. Schema driven vs Schemaless; Internal vs Boundary, Stateful vs Stateless, Pull vs Push as well as a number of other patterns and anti-patterns (in our opinion), all underpinned with practical examples, with design trade-offs of their use-cases. We’ll also discuss the supporting tech stack (we have an obvious preference for Kafka, and we’ll discuss why), with some tech that plays nicely together.
The key takeaway from this talk should be that while there are no silver bullets, there are a number of considerations that should be taken into account that could help you chose the best approach for where your business currently finds itself.
Richard Noble – Richard has been writing software for over 20 years, and is constantly astounded by what he doesn’t know. Recently running a team of data and software engineers, he’s gotten into the Kafka ecosystem in a big way.
Francesco Nobilia – Francesco is a big Kafka fun and organiser of Kafka Guild at Babylon Health. In the last year, he has been part of the team designing and building a distributed real-time streaming platform based on Kafka and Kafka Streams. Currently, he is focusing on making the above mentioned platform self-service.
7:00 pm – Talk – “Apache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline!” with Robin Moffatt, Senior Developer Advocate, Confluent
Apache Kafka is a distributed, scalable, and fault-tolerant streaming platform, providing low-latency pub-sub messaging coupled with native storage and stream processing capabilities. Integrating Kafka with RDBMS, NoSQL, and object stores is simple with Kafka Connect, which is part of Apache Kafka. ksqlDB is the source-available SQL streaming engine for Apache Kafka, and makes it possible to build stream processing applications at scale, written using a familiar SQL interface.
In this talk, we’ll explain the architectural reasoning for Apache Kafka and the benefits of real-time integration, and we’ll build a streaming data pipeline using nothing but our bare hands, Kafka Connect, and ksqlDB.
Gasp as we filter events in real-time! Be amazed at how we can enrich streams of data with data from RDBMS! Be astonished at the power of streaming aggregates for anomaly detection!
Robin is a Developer Advocate at Confluent, the company founded by the original creators of Apache Kafka®, as well as an Oracle Groundbreaker Ambassador and ACE Director (alumnus). His career has always involved data, from the old worlds of COBOL and DB2, through the worlds of Oracle and Hadoop, and into the current world with Kafka. His particular interests are analytics, systems architecture, performance testing and optimization. He blogs at http://cnfl.io/rmoff and http://rmoff.net/ (and previously http://ritt.md/rmoff) and can be found tweeting grumpy geek thoughts as @rmoff. Outside of work he enjoys drinking good beer and eating fried breakfasts, although generally not at the same time.