Administration of Confluent Apache Kafka Training Course

Module 1: Introduction to the architecture and configuration of the Confluent Apache Kafka cluster

• The role of Kafka in modern data pipelines
• Distinguishing between Apache Kafka and Confluent Kafka
• Core components: producers, consumers, brokers, topics, and partitions
• Deployment models and scaling considerations for Kafka clusters

Module 2: Zookeeper Quorum Configuration

• Understanding Zookeeper
• The function of Zookeeper within a Kafka cluster
• Determining Zookeeper Quorum size
• Zookeeper setup and configuration
• Setting up SSH on our servers
• Practical exercise: Configuring Zookeeper as both a team and a service
• Utilizing the Zookeeper Command Line Interface (CLI)
• Practical exercise: Configuring the Zookeeper Quorum
• The Zookeeper internal file system
• Performance factors impacting Zookeeper
• Demonstration of management tools for Zookeeper and Zoonavigator

Module 3: Kafka Cluster Configuration

• Fundamental Kafka concepts
• Configuring Kafka settings
• Practical exercise: Configuring Kafka brokers
• Practical exercise: Running Kafka commands
• Practical exercise: Setting up a Kafka Multi-Broker Cluster
• Practical exercise: Testing the Kafka cluster
• Verifying connectivity to the Kafka cluster
• Advertised.listeners configuration: a critical setting
• Topic settings
• Configuration for downloading and ingesting messages into topics
• Practical exercise: Demonstrating Kafka resilience
• Kafka performance: I/O operations
• Kafka performance: Network (RED)
• Kafka performance: RAM usage
• Kafka performance: CPU load
• Kafka performance: Operating System (OS) interactions
• Kafka performance: Other factors
• Practical exercise: Modifying Kafka broker configurations

Module 4: Advanced Kafka Configuration

• Configuring Landoop Kafka topic UI, Confluent REST Proxy, and Confluent Schema Registry
• Sending and receiving messages via CLI, Java, and the Spring framework
• Monitoring metrics and tools (including Confluent Control Center, Elasticsearch, and others)
• Managing log files and offsets
• High availability and disaster recovery strategies
• Achieving high availability through replication
• Optimizing producer and consumer performance
• Developing disaster recovery plans
• Controlling failover and recovering data
• Setting up connectors
• Implementing Kafka Connect
• Exploring Kafka security features

Summary and Next Steps