Apache Kafka Crash Course

    By
    Hussein Nasser
    -
    22
    25



    Apache Kafka is a distributed stream processing software developed by LinkedIn and written in Scala and Java.

    Chapters
    0:00 Intro
    2:32 Kafka Broker
    5:39 Kafka Producer
    7:11 Kafka Consumer
    8:34 Kafka Partitions
    12:50 Queue vs Pub-Sub
    16:47 Consumer Group
    23:06 ZooKeeper
    29:45 Coding Example
    1:02:48 Kafka Pros-Cons
    1:17:00 Summary

    Download slides here
    https://payhip.com/b/Y0Cq

    In this video I want to focus on the basics of Kafka, talk about how it works give finally spin up a Kafka cluster and write a producer and a consumer.

    Kafka Components
    _ producer
    _consumer
    _topic
    _partition
    How Kafka works
    Example ( Ride sharing )
    Spin Kafka cluster
    Write a producer
    Write a consumer
    Kafka Applications
    Pros and Cons

    Source Code
    https://github.com/hnasr/javascript_playground/tree/master/kafka

    Docker commands spin up kafka
    //if you ever mess up or you want to start fresh just stop and remove the containers and rerun the command

    docker stop zookeeper kafka
    docker rm zookeeper kafka

    docker run –name zookeeper -p 2181:2181 -d zookeeper

    docker run -p 9092:9092 –name kafka -e KAFKA_ZOOKEEPER_CONNECT=husseinmac:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://husseinmac:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 -d confluentinc/cp-kafka

    Resources

    KIP 500 (Remove Zookeeper)
    https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum

    Support my work on PayPal
    https://bit.ly/33ENps4

    Become a Member on YouTube
    https://www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join

    šŸ§‘ā€šŸ« Courses I Teach
    https://husseinnasser.com/courses

    šŸ­ Backend Engineering Videos in Order
    https://backend.husseinnasser.com

    šŸ’¾ Database Engineering Videos

    šŸŽ™ļøListen to the Backend Engineering Podcast
    https://husseinnasser.com/podcast

    Gears and tools used on the Channel (affiliates)

    šŸ–¼ļø Slides and Thumbnail Design
    Canva
    https://partner.canva.com/c/2766475/647168/10068

    šŸŽ™ļø Mic Gear
    Shure SM7B Cardioid Dynamic Microphone
    https://amzn.to/3o1NiBi

    Cloudlifter
    https://amzn.to/2RAeyLo

    XLR cables
    https://amzn.to/3tvMJRu

    Focusrite Audio Interface
    https://amzn.to/3f2vjGY

    šŸ“· Camera Gear
    Canon M50 Mark II
    https://amzn.to/3o2ed0c

    Micro HDMI to HDMIĀ 
    https://amzn.to/3uwCxK3

    Video capture card
    https://amzn.to/3f34pyD

    AC Wall for constant power
    https://amzn.to/3eueoxP

    Stay Awesome,
    Hussein

    source

    Previous articleKubernetes Tutorial – How to Crack šŸ”„CKA ExamšŸ”„ in 21 Hours?
    Next articleApa Itu Big Data?
    Hussein Nasser

    RELATED ARTICLESMORE FROM AUTHOR

    25 COMMENTS

    11. Only worked with docker compose for me. (m1 mac).

      version: '3'

      services:
      zookeeper:
      image: wurstmeister/zookeeper
      container_name: zookeeper
      ports:
      – "2181:2181"
      kafka:
      image: wurstmeister/kafka
      container_name: kafka
      ports:
      – "9092:9092"
      environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181