Scraping

Crawlee covers your crawling and scraping end-to-end and helps you build reliable scrapers. Fast.

Automa is an extension for browser automation, from doing a repetitive task, auto-fill forms, taking a screenshot, or scraping website data — the choice is yours.

Huginn is a system for building agents that perform automated tasks for you online. They can read the web, watch for events, and take actions on your behalf. Huginn's Agents create and consume events, propagating them along a directed graph.

Beautiful Soup is a Python library for pulling data out of HTML and XML files. It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree.

A Fast and Powerful Scraping and Web Crawling Framework

x-crawl is a flexible Node.js multifunctional crawler library. Flexible usage and numerous functions can help you quickly, safely, and stably crawl pages, interfaces, and files.