• Supported in Jaeger since 1.6.0
• Supported Kafka versions: 0.9+

Kafka can be used as an intermediary buffer between collector and an actual storage. jaeger can be configured to both act as the collector for Kafka data before adding it to a Kafka topic and also as a ingester to get data from Kafka and forward it to another storage backend.

Writing to Kafka is particularly useful for building post-processing data pipelines.

Kafka also has the following officially supported resources available from the community:

• Docker container  for getting a single node up quickly
• Helm chart  by Bitnami
• Strimzi Kubernetes Operator 

Configuration

Collector

In order to collect tracing data and add it to a Kafka topic this configuration  is what should be used.

Ingester

In order to write tracing data which is added to a Kafka topic this configuration  is what should be used.

Topic & partitions

Unless your Kafka cluster is configured to automatically create topics, you will need to create it ahead of time. You can refer to the Kafka quickstart documentation  to learn how.

You can find more information about topics and partitions in general in the official documentation  . This article  provide more details about how to choose the number of partitions.