2

I am trying to find a suitable Python library to do stream processing with streams Kafka topics, Kafka streams. Specifically, I am looking for libraries that support the following operations.

  • KStream-to-Kstream
  • KTable-to-Ktable
  • KTable-to-KTable Foreign-Key Join
  • KStream-to-KTable

as listed here https://kafka.apache.org/24/documentation/streams/developer-guide/dsl-api.html#joining. Can anyone point me in the right direction?

I have looked into the confluent_kafka library and kafka_python library but apart from supporting simple consumption of Kafka topics, they don't look like they have out-of-the-box support for more complex operations.

Improve this question
0

1 Answer 1

Reset to default
0

You could check out Faust or bytewax. I think either of those would allow you to do complex operations in Python like windowing, aggregations and joins in some sense.

Unfortunately, I don’t think Faust isn’t being actively maintained by Robinhood anymore.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Faust was forked to faust/faust-streaming

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.