Crawler

A Fast and Powerful Scraping and Web Crawling Framework

x-crawl is a flexible Node.js multifunctional crawler library. Flexible usage and numerous functions can help you quickly, safely, and stably crawl pages, interfaces, and files.

the fastest node.js globber and directory crawler