🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

Create agents that monitor and act on your behalf. Your agents are standing by!

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

An autonomous agent that conducts deep research on any data using any LLM providers

Python scraper based on AI

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

A cli tool to browse and play anime

List of libraries, tools and APIs for web scraping and data processing.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

🦊 Anti-detect browser

Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?

Self-hosted webscraper.

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Undetected version of the Playwright testing and automation library.

Web Scraper in Go, similar to BeautifulSoup

Make your job hunt easy by automating your application process with this Auto Applier

Vision utilities for web interaction agents 👀

The web scraping open project repository aims to share knowledge and experiences about web scraping with Python

Persistent HTTP cache for python requests

LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping

👻 Experimental library for scraping websites using OpenAI's GPT API.

Lightweight library for scraping web-sites with LLMs

Undetected Python version of the Playwright testing and automation library.

Creating Scrapy scrapers via the Django admin interface

A blazing fast, async-first, undetectable webscraping/web automation framework based on ultrafunkamsterdam/nodriver. Now with Docker support!

Watch everything from your terminal.

Scalable Python web scraping scripts for +40 popular domains

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Suck the InTernet

Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.