How Fluentd simplifies collecting and consuming logs | Fluentd simply explained



Fluentd explained | How Fluentd simplifies collecting and consuming logs | Fluentd Tutorial

Fluentd is a Cloud Native Computing Foundation (CNCF) project.
► I’m happy that this video was sponsored by the Cloud Native Computing Foundation https://www.cncf.io/ 🙌🏼
► More infos on Fluentd: https://www.fluentd.org/

Fluentd is an open source log data collector, which lets you unify the data collection and consumption for a better use and understanding of data.

With Fluentd you can send any data from any data source to any destination or storage.

#devops #fluentd #techworldwithnana

▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
0:00 – Intro
0:26 – Why we actually need log data?
1:40 – Challenges of collecting and consuming logs
3:40 – Solution – a unified logging layer
4:17 – How does Fluentd work
6:29 – How to configure Fluentd?
9:03 – Built-In Reliability
10:09 – Other Use Cases

▬▬▬▬▬▬ Want to learn more? 🚀 ▬▬▬▬▬▬
Full K8s course course ► https://youtu.be/X48VuDVv0do
DevOps Tools, like Terraform, Prometheus ► https://bit.ly/2W9UEq6
Full Docker course ► https://youtu.be/3c-iBn73dDE
Jenkins Pipeline Tutorials ► https://bit.ly/2Wunx08

▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
Join private Facebook group ► https://bit.ly/32UVSZP
DEV ► https://bit.ly/3h2fqiO
INSTAGRAM ► https://bit.ly/2F3LXYJ
TWITTER ► https://bit.ly/3i54PUB
LINKEDIN ► https://bit.ly/3hWOLVT

▬▬▬▬▬▬ Courses & Ebooks & Bootcamp 🚀 ▬▬▬▬▬▬
► Become a DevOps Engineer – full educational program 👉🏼 https://bit.ly/3gEwf4V
► Udemy courses – get biggest discounts here 👉🏼 http://bit.ly/2OgvzIO
► Kubernetes 101 – compact and easy-to-read ebook bundle 👉🏼 https://bit.ly/3mPIaiU

source

26 thoughts on “How Fluentd simplifies collecting and consuming logs | Fluentd simply explained”
  1. ► Follow me on IG for behind the scenes content 😊👉🏼 https://bit.ly/2F3LXYJ
    ► This video was sponsored by the Cloud Native Computing Foundation https://www.cncf.io/ 🙌🏼
    ► More infos on Fluentd: https://www.fluentd.org/

    ▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
    0:00 – Intro
    0:26 – Why we actually need log data?
    1:40 – Challenges of collecting and consuming logs
    3:40 – Solution – a unified logging layer
    4:17 – How does Fluentd work
    6:29 – How to configure Fluentd?
    9:03 – Built-In Reliability
    10:09 – Other Use Cases

  2. Great Video Nana! I use FluentD for quite some time already collecting logs and sending them to specific indexes outisde of the cluster so they can easially be observed using query's within observability platforms 🙂

  3. Hey Nana, the content of your video is always very good and easily understandable.
    Except for your audio level, it seems low to me. I have tried many ways to increase your audio and try different devices as well but it didnt increase the sound volume. please make sure your videos are having little high audio quality so that some one can easily listen to it without earphones or headphones.
    I hope it wouldn`t appear offensive to you.

  4. Thank you. The contents are same as your Fluentbit lecture. Could you please give bit precise use case info when to use which one or how to use both.

  5. very well broken down, the config snippets and animations help to visualize it a as well, thanks !

  6. Hello.. thanks for taking the time to put together this video. It’s excellent, it really does help with conceptualizing how to utilize fluentd.. I got it up and running and I am capturing logs from several of my back up processes and syslogs from few servers. Something I’ve not been able to figure out is how to dynamically assign labels to my Loki match blocks. I’ve used the rewrite_tag_filter plugin to push log lines to different match blocks where I have different labels but that feels like I’m hard coding the same thing over and over just to get one label to be different. I’d like to store the program or process name in a variable based on the log line and use it as a label. Is that possible? I’ve tried several different ways but I have not been successful doing that as yet. Any info would be greatly appreciated. Thanks again

    *edit when I say labels I’m referring to labels used in Loki and visible in Grafana

  7. I don't think the explanation is good. it is so abstracted and I wonder if you really have configured it by your self. Is Fluentd flies along the applications like the animation you put on 4:26?

  8. How to create Fluentd index with application name , Like Fluend-containername , deployed with helm

  9. Can u please let me know how to push specific namespace logs only to elasticsearch from fluentd in k8s any specific parameters we need to use ?

  10. Thanks for the video. is it possible fluent-logger-java integrate in android? if possible please provide details?

  11. What is the need to restart/rollout Fluentd deamonset and statefulset?
    in which cases we have to do it normally? any use-cases?
    Ex:- kubectl rollout restart daemonset/fluentd
    Thanks

  12. Dear Nana, when i try to deploy fluentd using helm chart by default it deploys as root user. i changed to non root user by using security context. But the problem is it errors out "[error]: #0 unexpected error error_class=Errno::EACCES error="Permission denied @ rb_sysopen – /var/log/fluentd-containers.log.pos". Can you please advise in case if you had this issue

  13. VERY Great Nana,, BIG Thanks. Can you please make a video on EFK deployment and JSON log parsing and RAW Log parsing using Fluentd regexp? would be really helpful

  14. Thanks for the video…Currently I am using Fluentd but it's can't go beyond 2k/EventPerSec/PerIntance. Where I am looking for 20k/EPS/PerInstance with each average event size of 800 Byte. Do you have any suggestion for the same?

  15. Thanks for this video. is there a way to limit nested parsing in json parser plugin @t ? as current it's parsing json in all levels to lots of field getting created. to wanted to limit josn parsing on first level.

Leave a Reply

Your email address will not be published.

Captcha loading...