Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

Algorithms for outlier, adversarial and drift detection

nannyml: post-deployment data science in python

Curated list of open source tooling for data-centric AI on unstructured data.

Frouros: an open-source Python library for drift detection in machine learning systems.

⚓ Eurybia monitors model drift over time and securizes model deployment with data validation

Free Open-source ML observability course for data scientists and ML engineers. Learn how to monitor and debug your ML models in production.

A curated list of awesome open source tools and commercial products for monitoring data quality, monitoring model performance, and profiling data 🚀

A toolkit for evaluating and monitoring AI models in clinical settings

A comprehensive solution for monitoring your AI models in production

Online and batch-based concept and data drift detection algorithms to monitor and maintain ML performance.

PKBoost: Adaptive GBDT for Concept Drift, Built from scratch in Rust, PKBoost manages changing data distributions in fraud detection with a fraud rate of 0.2%. It shows less than 2% degradation under drift. In comparison, XGBoost experiences a 31.8% drop and LightGBM a 42.5% drop

Passively collect images for computer vision datasets on the edge.

A tiny framework to perform adversarial validation of your training and test data.

Sales Conversion Optimization MLOps: Boost revenue with AI-powered insights. Features H2O AutoML, ZenML pipelines, Neptune.ai tracking, data validation, drift analysis, CI/CD, Streamlit app, Docker, and GitHub Actions. Includes e-mail alerts, Discord/Slack integration, and SHAP interpretability. Streamline ML workflow and enhance sales performance.

A long-form article introducing the Twin Test: a practical standard for high-stakes machine learning where models must show nearest “twin” examples, neighborhood tightness, mixed-vs-homogeneous evidence, and “no reliable twins” abstention. Argues similarity and evidence packets beat probability scores for trust and safety.

Drift-Lens: an Unsupervised Drift Detection Framework for Deep Learning Classifiers on Unstructured Data

In this repository, we will present techniques to detect covariate drift, and demonstrate how to incorporate your own custom drift detection algorithms and visualizations with SageMaker model monitor.

A ⚡️ Lightning.ai ⚡️ component for train and test data drift detection

Drift Lens Demo

Fraud detection system using machine learning and deep learning (XGBoost + Autoencoder). Trains on synthetic financial transactions to flag suspicious activity with business-focused evaluation metrics.

Real-time anomaly detection system for GitHub activity using Airflow, MLflow, and Terraform

Data Drift detection using auto encoders

A lightweight, no-install, GUI-based Python toolbox for detecting, measuring, and visualizing domain shift (data shift) across datasets — all running in your browser on Windows, macOS, or Linux.

In this project, we illustrate how the Kolmogorov Smirnov (KS) statistical test works, and why it is commonly used in Machine Learning (ML), Deep Learning (DL) and Artificial Intelligence (AI).

Adversarial labeller is a sklearn compatible instance labelling tool for model selection under data drift.

An end-to-end MLOps pipeline for a production-grade fraud detection model. This project demonstrates best practices including data versioning (DVC), experiment tracking (MLflow), CI/CD (GitHub Actions), containerization (Docker), deployment on GKE, and advanced model analysis (poisoning attacks, drift, fairness, explainability).

Tabular Data Drift Detector & Reporter:
A CLI tool that connects to any database or CSV, computes statistical drift (KS-test, Jensen-Shannon) between “baseline” vs. “current” data, and emits a Markdown/HTML report with charts.

Predicting the number of bicycles at rental stations.