Apache Kafka® 101: Partitioning

    2
    19



    https://cnfl.io/kafka-101-module-3 | With partitioning, a single topic log is broken into multiple logs. Each of these logs can live on a different node in the Apache Kafka® cluster. The effort behind storing and processing messages as well as writing new messages can be split among many nodes in the cluster. Learn how partitioning works in this video with Tim Berglund (Senior Director of Developer Experience, Confluent).

    Use the promo code KAFKA101 to get $101 of free Confluent Cloud usage: https://www.confluent.io/confluent-cloud/tryfree/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka

    Promo code details: https://www.confluent.io/confluent-cloud-promo-disclaimer/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka

    LEARN MORE
    ► How to Choose the Number of Topics/Partitions in a Kafka Cluster?: https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
    ► Benchmarking Apache Kafka Latency at the 99th Percentile ft. Anna Povzner: https://developer.confluent.io/podcast/benchmarking-apache-kafka-latency-at-the-99th-percentile-ft-anna-povzner?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
    ► How to Change the Number of Partitions and Replicas of a Kafka Topic: https://kafka-tutorials.confluent.io/change-topic-partitions-replicas/ksql.html?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
    ► How to Read from a Specific Offset and Partition with the Kafka Console Consumer: https://kafka-tutorials.confluent.io/kafka-console-consumer-read-specific-offsets-partitions/confluent.html?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka

    CONNECT
    Subscribe: https://youtube.com/c/confluent?sub_confirmation=1
    Site: https://confluent.io
    GitHub: https://github.com/confluentinc
    Facebook: https://facebook.com/confluentinc
    Twitter: https://twitter.com/confluentinc
    LinkedIn: https://www.linkedin.com/company/confluent
    Instagram: https://www.instagram.com/confluent_inc

    ABOUT CONFLUENT
    Confluent, founded by the creators of Apache Kafka®, enables organizations to harness business value of live data. The Confluent Platform manages the barrage of stream data and makes it available throughout an organization. It provides various industries, from retail, logistics and manufacturing, to financial services and online social networking, a scalable, unified, real-time data pipeline that enables applications ranging from large volume data integration to big data analysis with Hadoop to real-time stream processing. To learn more, please visit https://confluent.io

    #kafka #kafkastreams #streamprocessing #apachekafka #confluent

    source

    Previous articleprepare for the Certified Kubernetes Application Developer exam Part 2
    Next articleHerramientas Data Science y Big Data ¿Te lías? Te lo explico fácil

    19 COMMENTS

    1. What is the relation between events and messages?
      Messages have key-value pairs? or Events have key-value pairs?
      what exactly … Events or messages are stored in partitions with key-value pairs?

      this was a helpful video
      Take key ——> HASH Function——-> Output mod (Total no. of partitions) ——> Resulting no. is Partitions number where message going to store.
      I was unaware of this concept.
      Thank You Tim.