🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Python scraper based on AI

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

A collection of awesome web crawler,spider in different languages

Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

Self-hosted webscraper.

A list of practical knowledge-building projects.

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

Scrape Google AI Mode responses without blocks on a large scale.

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.

Web Scraper in Go, similar to BeautifulSoup

Generate and download e-books from online sources.

A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.

A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.

CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

Faster requests on Python 3

:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy

Bulk download your favourite anime episodes from your favourite anime websites

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

OnlyFans content downloader

A list of scrapers from around the web.

An ergonomic Rust HTTP Client with TLS fingerprint

A universal web-util for PHP.

NBA Stats API via Basketball Reference

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

100% free and full open-source edge Firecrawl alternative with better links extraction for agents - that you can deploy to cloudflare or vercel by yourself.