What is this Python project?
Quix Streams is a library for developing real-time streaming applications focused on time-series data and high-performance. It was developed at McLaren for processing huge sensor data streams from F1 racing cars, and is designed to be used for high-frequency telemetry services when you need to process high volumes of time-series data with up to nanosecond precision. It uses a message broker such as Apache Kafka (instead of a database) so you can process time-series data with high performance and resource savings.
Using Quix Streams, you can:
- Produce time-series and event data to a Kafka Topic.
- Consume time-series and event data from a Kafka Topic.
- Process data by consuming it from one Kafka Topic, process it, and then producing the results back to another Kafka Topic.
- Attach metadata to streams and group streams by metadata attributes.
What's the difference between this Python project and similar ones?
What's distinct about Quix Streams is its processing API—although it’s a client library, it’s incredibly powerful.
- On the feature spectrum, it lies somewhere between Kafka client libraries such as Faust and kafka-python (which are easy to use but have a basic feature set) and full-fat stream processing engines like Flink and Spark (which are powerful but have a steep learning curve).
- Includes many built in functions and data types such as
ParameterData(for time-series data) andEventData, buffering and state management for rolling window calculations, serialization support for video and audio streams, and more. - The main difference is that processing is done in Python rather than Java, and it runs in a client environment rather than a server-side engine. --
Anyone who agrees with this pull request could submit an Approve review to it.