Stream Processing Patterns in Apache Flink

Course details

Frameworks such as Apache Flink can help you build fast, scalable stream processing applications, but big data engineers still need to design smart use cases to achieve maximum efficiency. In this course, instructor Kumaran Ponnambalam demonstrates how to use Apache Flink and associated technologies to build stream-processing use cases leveraging popular patterns. Kumaran begins by highlighting the opportunities and challenges that stream processing brings to big data. He then goes over four popular patterns for stream processing: streaming analytics, alerts and thresholds, leaderboards, and real-time predictions. Along the way, he reviews example use cases and explains how to leverage Flink, as well as key technologies like MariaDB and Redis, to implement key examples.

Concepts

0. Introduction

01 - Stream processing with Flink
02 - What you should know

1. Stream Processing with Flink

03 - What is stream processing
04 - Streaming - Opportunities and challenges
05 - Streaming with Flink
06 - Setting up the exercise files
07 - Setting up Kafka
08 - Setting up MariaDB and Redis

2. Streaming Analytics

09 - Streaming analytics - Pattern
10 - Streaming analytics - Use case design
11 - Streaming analytics - Helper classes
12 - Streaming analytics - Pipeline implementation
13 - Streaming analytics - Results review

3. Alerts and Thresholds

14 - Alerts and thresholds - Pattern
15 - Alerts and thresholds - Use case design
16 - Alerts and thresholds - Helper classes
17 - Alerts and thresholds - Pipeline implementation
18 - Alerts and thresholds - Review

4. Leaderboards

19 - Leaderboards - Pattern
20 - Leaderboards - Use case design
21 - Leaderboards - Helper classes
22 - Leaderboards - Pipeline implementation
23 - Leaderboards - Review

5. Real-Time Predictions

24 - Real-time predictions - Pattern
25 - Real-time predictions - Use case design
26 - Real-time predictions - Helper classes
27 - Real-time predictions - Pipeline implementation
28 - Real-time predictions - Review

6. Use Case Project

29 - Use case definition
30 - Design of the project
31 - Code walkthrough
32 - Execute and analyze

Conclusion

33 - Next steps

Stream Processing Patterns in Apache Flink

Authors

Kumaran Ponnambalam