Fluent Bit v3: Unified Layer for Logs, Metrics and Traces

The last talk of the conference. Notes may be a bit unstructured due to tired note taker.

Background

  • FluentD is already graduated
  • FluentBit is a daughter-project of FluentD (also graduated)

Basics

  • FluentBit is compatible with
    • Prometheus (It can replace the Prometheus scraper and node exporter)
    • OpenMetrics
    • OpenTelemetry (HTTPS input/output)
  • FluentBit can export to Prometheus, Splunk, InfluxDB or others
  • So pretty much it can be used to collect data from a bunch of sources and pipe it out to different backend destinations
  • Fluent ecosystem: No vendor lock-in to observability

Architectures

  • The fluent agent collects data and can send it to one or multiple locations
  • FluentBit can be used for aggregation from other sources

In the Kubernetes logging ecosystem

  • Pod logs to console -> Streamed stdout/err gets piped to file
  • The logs in the file get encoded as JSON with metadata (date, channel)
  • Labels and annotations only live in the control plane -> You have to collect it additionally -> Expensive

New stuff

Limitations with classic architectures

  • Problem: Multiple filters slow down the main loop
flowchart LR
    subgraph main[Main Thread/Event loop]
        buffer
        schedule
        retry
        fitler1
        filter2
        filter3
    end
    in-->|pipe in data|main
    main-->|filter and pipe out|out

Solution

  • Solution: Processor - a separate thread segmented by telemetry type
  • Plugins can be written in your favorite language (c, rust, go, …)
flowchart LR
    subgraph in
        reader
        streamner1
        processor2
        processor3
    end
    in-->|pipe in data|main(Main Thread/Event loop)
    main-->|filter and pipe out|out

General new features in v3

  • Native HTTP/2 support in core
  • Content modifier with multiple operations (insert, upsert, delete, rename, hash, extract, convert)
  • Metrics selector (include or exclude metrics) with matcher (name, prefix, substring, regex)
  • SQL processor -> Use SQL expression for selections (instead of filters)
  • Better OpenTelemetry output