Scraper

Crawlee covers your crawling and scraping end-to-end and helps you build reliable scrapers. Fast.

Huginn is a system for building agents that perform automated tasks for you online. They can read the web, watch for events, and take actions on your behalf. Huginn's Agents create and consume events, propagating them along a directed graph.

x-crawl is a flexible Node.js multifunctional crawler library. Flexible usage and numerous functions can help you quickly, safely, and stably crawl pages, interfaces, and files.

A fast tool to convert any website into LLM-ready markdown data.