A scalable, mature and versatile web crawler based on Apache Storm

Resources for running StormCrawler with Docker services

Process web archives (WARC format) with StormCrawler and index content into Elasticsearch or Solr

Ansible playbook for deploying a Storm cluster

StormCrawler and OpenSearch

No description provided.

StormCrawler topology to evaluate the performance of different backends and configurations