How can Apache Kafka be integrated with Eventuate?

At the moment, there is no Apache Kafka integration with Eventuate. We can imagine the following integration options and plan to provide them in future releases:

  • Event log storage backend: Apache Kafka would serve best as Event log storage backend. A storage plugin would map a Local event log to a Kafka topic partition to preserve total event order. Replication of that topic is Kafka-specific and used to provide stronger durability guarantees within a location. Asynchronous replication across locations is Eventuate-specific and preserves causal event order. Kafka storage backends from different locations do not directly communicate with each other.
  • Stream processing adapter: Another option is to treat Kafka as an external integration hub to other stream processing applications and to implement a stream processing adapter for exchanging events with Kafka. It needs to be investigated if implementation efforts for this and the previous option can be shared.