Automation & Bots

🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

Scrapy, a fast high-level web crawling & scraping framework for Python.

The all-in-one AI productivity accelerator. On device and privacy first with no annoying setup or configration.

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—all for free or enjoy our SaaS plan!

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Python scraper based on AI

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

SeleniumBase simplifies browser automation, scraping, and testing. CDP Mode adds stealth (and works with Playwright). Lots of examples and YouTube tutorials.

Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Lighter web automation with Python

List of libraries, tools and APIs for web scraping and data processing.

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

A Chrome DevTools Protocol driver for web automation and scraping.

Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.

🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.

Self-hosted webscraper.

Snoop — инструмент разведки на основе открытых данных (OSINT world)

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

The Python Code Tutorials

Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.

The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.

Free anti fingerprint browser, 指纹浏览器, 隐私浏览器, 防识别浏览器, 反识别浏览器, 防关联浏览器, 免费的web3空投专用指纹浏览器 https://virtualbrowser.cc/?src=github

Undetected version of the Playwright testing and automation library.