Wednesday, September 12, 2018

Kafka - The beginning


Getting started with Apache Kafka 


  • Enterprise Challenges with Data
  • Messaging Limitations and Challenges
  • Linkedin's search for a Better solution
  • Apache Kafka as a viable solution



Getting to know Apache Kafka's Architecture



  • Apache Kafka as a Messaging System
  • The Apache Kafka Cluster
  • Principles of Distributed Systems
  • Reliable Work Distribution
  • Distributed Consensus with Apache Zookeeper



Understanding  Topics, Partitions, and Brokers


  • Apache Kafka Setup Demo
  • Apache Kafka Topics in Detail
  • The Consumer Offset and Message Retention Policy
  • Demo  :  Starting Apache Kafka and Producing and Consuming Messages
  • Apache Kafka as a Distributed Commit Log
  • Apache Kafka Partitions in Detail
  • Distributed Partition Management in Apache Kafka
  • Achieving Reliability with apache Kafka Replication
  • Demo :  Fault-tolerance and Resiliency in Apache Kafka




Introduction and Setting up an Apache Kafka Development Environment


  • Basics of Creating an Apache Kafka Producer
  • Creating and Preparing Apache Kafka Producer Records
  • Apache Kafka Producer Record Properties
  • The Process of Sending Messages, Part One
  • The Process of Sending Messages, Part Two
  • Message Buffering and Micro-batching
  • Message Delivery and Ordering Guarantees
  • Demo: Creating and Running an Apache Kafka Producer Application in Java
  • Advanced Topics



Consuming Messages with Kafka Consumers and  Consumer Groups


  • What's Apache Kafka Consumer Overview
  • Subscribing and Unsubscribing to Topics
  • Comparing Subscribe and Assign APIs
  • Single Consumer Subscriptions and Assignments.
  • The Poll Loop
  • Demo:  Simple Kafka Consumer
  • Walkthrough:  Consumer Polling
  • Walkthrough :  Message Processing
  • The Consumer Offset in Detail
  • Offset Behavior and Management
  • CommitSync and CommitAsync for Manual Offset Management
  • When to Manager Your Own Offsets Altogether
  • Scaling out with Consumer Groups
  • Consumer Group Coordinator
  • Demo :  Consumer Groups
  • Configuration and Advanced Topics


What next  -  Exploring the Kafka Ecosystem -


  • Apache Kafka's success and Challenges
  • Challenges and Solutions for Data Governance
  • Challenges and Solutions for Consistency and Productivity
  • Challenges and Solutions for Fast Data
  • Apache Kafka's Ecosystem and Summary


































No comments:

Post a Comment

Hyderabad Trip - Best Places to visit

 Best Places to Visit  in Hyderabad 1.        1. Golconda Fort Maps Link :   https://www.google.com/maps/dir/Aparna+Serene+Park,+Masj...