Workflow automation, bots, and scripting tools
845 repositories found
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio python SDK for intelligent web data gathering.
Web Scraping Framework
Learn Python for the next 30 (or so) Days.
Learn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.
Web crawler and scraper for Rust
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
Ui.Vision Open-Source RPA Software with Computer Vision, OCR, Anthropic Computer Use/LLM. Selenium IDE import/export.
The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.
The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.
General Assembly's 2015 Data Science course in Washington, DC
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
Simple web scraping for R
A code for extracting best-selling items, search results, and currently available deals from Amazon using Python and Oxylabs E-Commerce Scraper API.
The complete web scraping toolkit for PHP.
A guide for extracting titles, authors, and citations from Google Scholar using Python and Oxylabs SERP Scraper API.
A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers, user info, images...
AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.
Collection of patches for puppeteer and playwright to avoid automation detection and leaks. Helps to avoid Cloudflare and DataDome CAPTCHA pages. Easy to patch/unpatch, can be enabled/disabled on demand.
An ergonomic Python HTTP Client with TLS fingerprint
🔥 AI-powered web monitoring platform. Create automated scouts that search the web and send email alerts when they find what you're looking for.
Finviz analysis python library.
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
Undetected Python version of the Playwright testing and automation library.
HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.
Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabs’ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs
A JavaScript library for generating random user agents with data that's updated daily.
Faster requests on Python 3
Write web scrapers in Ruby using a clean, AI-assisted DSL. Kimurai uses AI to figure out where the data lives, then caches the selectors and scrapes with pure Ruby. Get the intelligence of an LLM without the per-request latency or token costs.
In this tutorial, we showcase how to scrape public Google data with Python and Oxylabs API.
AI Map is an AI-powered website mapping tool by Oxylabs AI Studio that uses natural language prompts to intelligently discover and extract relevant URLs from any website.