Apache Kafka® running on Kubernetes

Memphis.dev is a highly scalable and effortless data streaming platform

Apache InLong - a one-stop, full-scenario integration framework for massive data

An extensible distributed system for reliable nearline data streaming at scale

Low-code tool for automating actions on real time data | Stream processing for the users.

Http Connector for Apache Flink. Provides sources and sinks for Datastream , Table and SQL APIs.

DagsHub client libraries

Adapter for dbt that executes dbt pipelines on Apache Flink

A Python library for machine-learning and feedback loops on streaming data

Example projects and demos around data streaming , stream processing, change data capture, and more.

Sample code that shows the important aspects of developing custom connectors for Kafka Connect. It provides the resources for building, deploying, and running the code on-premises using Docker, as well as running the code in the cloud.

Sample Applications for Pravega.

Docker Compose environments for developing modern data platform architectures using Kafka, Flink, Spark, Iceberg, OpenLineage, OpenMetadata, Pinot, ClickHouse, StarRocks + Kpow & Flex by Factor House

🙈 The best way to lurk on Reddit

Apache Flink Demo Projects

Strimzi canary

High-performance and efficient Framework and Agent for creating data pipelines. The core of pipeline descriptions is based on the Configuration As Code concept and the Pkl configuration language by Apple.

Apache Kafka and Related Projects

Udacity Data Streaming Nanodegree Program

Kafka Connector for Apache Doris

Developer-friendly MCP server bridging Kafka and Pulsar protocols—built with ❤️ by StreamNative for an agentic, streaming-first future.

A library for data streaming and augmentation

This project implements a real-time data pipeline using Apache Kafka, Python's psutil library for metric collection, and SQL Server for data storage. The pipeline collects metrics data from the local computer, processes it through Kafka brokers, and loads it into a SQL Server database. Additionally, a real-time dashboard is created using Power BI.

Different flavours of CUSUM for change point detection.

Pulse is a tiny real-time data streaming framework (mini Flink/Beam) in Rust. Async (Tokio), pluggable operators, state, and I/O. Fast, modular, local-first.

A Federated Learning Method for Real-time Emotion State Classification from Multi-modal Streaming

Demonstration of PubNub's real-time data streaming capabilities from Twitter, Wikipedia & more

FastFlight is a high-performance data transfer framework using Apache Arrow Flight for efficient, modular, and pluggable data streaming with optional FastAPI integration for HTTP-based access.

A simple, time-tested, family of random hash functions in Java, based on CRC32, affine transformations, and the Mersenne Twister. 🎲

Native Apache Arrow for the BEAM: IPC streaming, Arrow Flight, and ADBC database bindings. Column data lives in Rust buffers; Elixir holds lightweight opaque handles. Precompiled NIFs for Linux, macOS, and Windows — no Rust required to use.