Troubleshooting and Debugging Kafka

Course details

In this course, learn to troubleshoot and debug Kafka, so you can get to the root of any issues you encounter instead of solving individual problems as they arise. Instructor Janani Ravi shows how you can use the Confluent platform—a full-scale data streaming platform that lets you easily access, store, and manage data as continuous, real-time streams—to expand the benefits of Kafka with enterprise-grade features and greatly improve Kafka management and monitoring. Janani first gives you an introduction to Kafka on Confluent, then shows how to observe consumers and brokers. Learn to process Kafka streams, monitor consumer groups, monitor multi-broker clusters, debug brokers and consumers, and configure alerts.

Concepts

0. Introduction

01 - Overview of troubleshooting and debugging Kafka

1. Introducing Cloud-Native Kafka on Confluent

02 - Prerequisites
03 - An overview of Apache Kafka
04 - The Confluent platform
05 - Downloading the Confluent platform binaries
06 - Running the Confluent platform locally
07 - Overview of the control center

2. Observing Consumers and Brokers

08 - Observing message production and consumption
09 - Produce messages using the control center
10 - Using datagen to generate synthetic data
11 - Observing consumers

3. Processing Kafka Streams

12 - Understanding ksqlDB
13 - Creating streams and tables using ksqlDB
14 - Performing join and windowing operations
15 - Viewing streams and tables

4. Monitoring Consumer Groups

16 - Programmatically produce messages using Python
17 - Troubleshooting partition usage
18 - Consumer groups
19 - Monitoring consumption in a consumer group
20 - Debugging consumption lag
21 - Monitoring consumer group rebalancing

5. Monitoring Multi-Broker Clusters

22 - Configuration properties for a multi-broker cluster
23 - Running a multi-broker cluster on the Confluent platform
24 - Monitoring brokers
25 - Observing partitions and replication
26 - Monitoring broker throughput and latency

6. Debugging Brokers and Consumers

27 - Debugging failed brokers, part 1
28 - Debugging failed brokers, part 2
29 - Debugging slow consumers

7. Configuring Alerts

30 - Configuring alerts
31 - Configuring email settings for alerts

Conclusion

32 - Summary and next steps