Fast, secure, efficient backup program

Deduplicating archiver with compression and authenticated encryption.

Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

Prometheus Alertmanager

Find duplicate files

A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.

rustic - fast, encrypted, and deduplicated backups powered by Rust

A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows

Extremely fast tool to remove duplicates and other lint from your filesystem

Simple, configuration-driven backup software for servers and workstations

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

Config driven, easy backup cli for restic.

plakar is a backup solution powered by Kloset and ptar

Scalable data pre processing and curation toolkit for LLMs

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Replace complex Borg Backup terminal commands with a beautiful web UI. Create, schedule, and restore backups with just a few clicks.

A powerful and modular toolkit for record linkage and duplicate detection in Python

Open source project for data preparation for GenAI applications

Fast Multimodal Semantic Deduplication & Filtering

Data deduplication engine, supporting optional compression and public key encryption.

Коллекция готовых SQL запросов для PostgreSQL по часто возникающим задачам (получение и модификация данных, ускорение запросов, обслуживание БД)

Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.

Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix

A list of free data matching and record linkage software.

Filter, Sort & Delete Duplicate Files Recursively

A secure and efficient file backup solution that fits both system administrators (CLI) and end users (GUI)

Locality Sensitive Hashing using MinHash in Python/Cython to detect near duplicate text documents

Productivity improvements for Rust ecosystem: warnings are skipped until errors are fixed, LSP-independent Neovim integration, etc.

A high-performance tool for video upscaling, interpolation, depth estimation, and more. Available as a CLI and Adobe Extension.

RocketMQ消息幂等去重消费者，支持使用MySQL或者Redis做幂等表，开箱即用