Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.

Download website to local directory (including all css, images, js, etc.)

Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

Plugin for website-scraper which returns html for dynamic websites using puppeteer

CMS, WebSite, Application and Ecommerce Development Tool Using JavaScript

DPULSE - Tool for complex approach to domain OSINT

🚀从聊天记录创造数字分身的一站式解决方案💡 使用聊天记录微调大语言模型，让大模型有“那味儿”，并绑定到聊天机器人，实现自己的数字分身。 数字克隆/数字分身/数字永生/LLM/聊天机器人/LoRA

🕸 Generates RSS feeds of any website & serves to the web! Automatic scraping. Ready to use configs. Write your own. Rolling Docker releases for speedy updates.

🌱 goClone - clone websites in seconds

Wayback Machine Downloader. 🔥 Download your entire archived websites from the Internet Archive Wayback Machine.

Automatically curates and posts content to LinkedIn. It can optionally use web scraping to gather data, which is then fed to ChatGPT to craft engaging LinkedIn posts.

ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on the scraped content. The bot utilizes Retrieval Augmented Generation and webscraping to return natural language answers to the user's queries.

Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.

A server to collect & archive websites that also supports video downloads

Plugin for website-scraper which returns html for dynamic websites using PhantomJS.

Scraper for https://marvelsnapzone.com to retrieve metadata of Marvel SNAP cards.

Evaluate JavaScript on a URL through headless Chrome browser.

JSON collection of scraped file extensions, along with their description and type, from FileInfo.com

Now you can keep track of your followers from YouTube, Instagram and Twitter accounts - Followers scraper API on AWS serverless

A spider to crawl webpages

Website Penetration Testing Tool With Dos Attack Feature

Use GPTparser with your OpenAI API to scrape & parse files into structured JSON files.

Plugin for website-scraper which allows to save resources to existing directory

sᴇᴀʀᴄʜ ᴇɴɢɪɴᴇ sᴄʀᴀᴘᴇʀ ᴛᴏᴏʟ (ʙᴀsʜ)

Bandwidth efficient scheduled downloads

Scraping websites made easy! A minimalistic yet powerful tool for collecting data from websites.

Download ALL the images (JPEG/GIF/PNG) from any Tumblr website! This project employs Python3 and BeautifulSoup4 to scrape a Tumblr site (with the url provided by the user) to download, page by page, all the images from the Tumblr site's posts. Ideal for archiving other peoples' Tumblrs <3

This script scrapes the verses and references from an openbible.info page into a JSON file - if needed, we use bible-api.com to translate to another bible version.

This is a website url scraper built using python.