C# and F# language binding and extensions to Apache Spark

A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.

Stream real time Tweets of current affairs like covid-19 using Kafka 2.0.0 high throughput producer & consumer into Elasticsearch using safe, idempotent and compression configurations. Aggregate the data and use it for further analytics.

This project implements an end-to-end pipeline for batch to stream processing using Kafka on Docker images. Besides that, there are exercises for a full course in Kafka.

Build a Kafka streaming app using IBM Streams

Scripts for music feature analysis, model training, and real-time recommendation using Apache Kafka. Extract features, store them in MongoDB, and process the data with Apache Spark. A web interface is also included.

A pipeline that consumes twitter data to extract meaningful insights about a variety of topics using the following technologies: twitter API, Kafka, MongoDB, and Tableau.

Kafka streaming performance tests.

Data streaming project with Apache Druid & Grafana: Real-time data processing, alerts, integration with AWS. It uses a combination of technologies and services, including Confluent-Kafka, Apache Druid, AWS SNS, EC2, Athena, S3, Glue and EventBridge, StepFunctions. Contribute to this powerful solution!

Streaming fraud detection system

Learn kafka real-time stream processing

The purpose of this project is to learn, develop and explore microservices using spring boot, spring cloud, event streaming

Base project for kafka-streams development and testing

The objectives of this project are to get experience of coding with: Spark, Spark SQL, Spark Streaming, Kafka, Scala and functional programming

No description provided.

Tests with Spark local, kafka streaming and socket streaming

This repository contains the code to read the streaming kafka data and perform various streaming operations

Using kafka streaming to implement the streaming ETL job, append data onto GCP Big Query after formatting.

No description provided.

An interactive dashboard simulating live sales and warehouse data for the company.

SENTINEL GOLD 🛡️: multi-modal Cyber-Physical Threat Fusion Engine

This repository is a comprehensive guide and toolkit for anyone looking to understand and implement data streaming solutions using Apache Kafka.

Consumer Producer Components

Example from Designing Event-driven Applications Using Apache Kafka Ecosystem By Bogdan Sucaciu m 5

🚆 Optimize Chicago's public transit with real-time tracking and monitoring of train movements, station activity, and ridership trends using Apache Kafka.

⚡ Process real-time FX rates with SQL and Kafka for accurate, low-latency display of active currency pairs and percentage changes.

Kafka integration in Springboot, including producer, consumer, streaming, and interactive query of Ktable.

End-to-end aircraft risk detection pipeline using Kafka, Spark Structured Streaming, and ML

Real-time FX rates processing with SQL batch and Kafka streaming solutions

Real-time CTA transit simulation using Kafka, Faust, KSQL, and Python consumers. Ingests station, weather, and turnstile events, processes streams, aggregates with KSQL, and powers a live transit dashboard. Demonstrates event-driven design, Avro serialization, and end-to-end streaming architecture.