https://cnfl.io/kafka-101-module-3 | With partitioning, a single topic log is broken into multiple logs. Each of these logs can live on a different node in the Apache Kafka® cluster. The effort behind storing and processing messages as well as writing new messages can be split among many nodes in the cluster. Learn how partitioning works in this video with Tim Berglund (Senior Director of Developer Experience, Confluent).
Use the promo code KAFKA101 to get $101 of free Confluent Cloud usage: https://www.confluent.io/confluent-cloud/tryfree/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
Promo code details: https://www.confluent.io/confluent-cloud-promo-disclaimer/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
LEARN MORE
► How to Choose the Number of Topics/Partitions in a Kafka Cluster?: https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
► Benchmarking Apache Kafka Latency at the 99th Percentile ft. Anna Povzner: https://developer.confluent.io/podcast/benchmarking-apache-kafka-latency-at-the-99th-percentile-ft-anna-povzner?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
► How to Change the Number of Partitions and Replicas of a Kafka Topic: https://kafka-tutorials.confluent.io/change-topic-partitions-replicas/ksql.html?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
► How to Read from a Specific Offset and Partition with the Kafka Console Consumer: https://kafka-tutorials.confluent.io/kafka-console-consumer-read-specific-offsets-partitions/confluent.html?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-101_content.apache-kafka
CONNECT
Subscribe: https://youtube.com/c/confluent?sub_confirmation=1
Site: https://confluent.io
GitHub: https://github.com/confluentinc
Facebook: https://facebook.com/confluentinc
Twitter: https://twitter.com/confluentinc
LinkedIn: https://www.linkedin.com/company/confluent
Instagram: https://www.instagram.com/confluent_inc
ABOUT CONFLUENT
Confluent, founded by the creators of Apache Kafka®, enables organizations to harness business value of live data. The Confluent Platform manages the barrage of stream data and makes it available throughout an organization. It provides various industries, from retail, logistics and manufacturing, to financial services and online social networking, a scalable, unified, real-time data pipeline that enables applications ranging from large volume data integration to big data analysis with Hadoop to real-time stream processing. To learn more, please visit https://confluent.io
#kafka #kafkastreams #streamprocessing #apachekafka #confluent
source
Thanks Tim.. This is best video over internet for those who just jumped in to Kafka….
Thank you, crystal clear!
Very well done Tim – Thank you!
Amazing explanation!!
good explanation, thanks!
Why don't use consistent hashing but use hash/ num_partition?
Is it possible one message can belongs to multiple partitions ?
It's simply a great explanation. Thanks Man
so it is generally correct , if not always, to say that messages in different partitions within same topic are mutually exclusive?
If the number of partitions changes, does Kafka re-hash and redistribute events? If not then events with the same ID could end up in different partitions?
finally i found the partition logic in this video. Thanks a lot for the crisp video tim.
Great video guys, very helpful!
clear explanation, thanks
Very well explained ..thank you
What's the name of this song? ps: Kafka is amazing
Yet again Tim, rock solid short snappy overview.
What is the relation between events and messages?
Messages have key-value pairs? or Events have key-value pairs?
what exactly … Events or messages are stored in partitions with key-value pairs?
this was a helpful video
Take key ——> HASH Function——-> Output mod (Total no. of partitions) ——> Resulting no. is Partitions number where message going to store.
I was unaware of this concept.
Thank You Tim.
hey tim, can you tell me another advantage using partition
Thank so much, it is so clearly