Neura Market
airflow logo

airflow

Free

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

AI AgentsFreeFree tier
Type
Open Source
Visit Website GitHub (45.3k stars)

About airflow

Apache Airflow® is a platform created by the community to programmatically author, schedule, and monitor workflows. It uses Python for dynamic pipeline generation, offers a robust web UI for monitoring and management, and integrates with major cloud providers like GCP, AWS, and Azure. The platform is scalable, extensible via custom operators, and open source with an active community.

Key Features

Pure Python workflow definition
Dynamic pipeline generation using Python
Robust web UI for monitoring and managing workflows
Scalable architecture with message queue orchestration
Extensible via custom operators and libraries
Built-in integrations with GCP, AWS, Azure and other services
Elegant and explicit pipelines with Jinja templating
Open source with active community contribution

Pros & Cons

Pros
  • Scalable modular architecture with arbitrary number of workers
  • Dynamic pipeline generation allows flexibility
  • Extensible with custom operators
  • Elegant and explicit pipeline definitions
  • Pure Python - no command line or XML
  • Useful UI for monitoring and logging
  • Robust integrations with major cloud providers
  • Easy to use for anyone with Python knowledge

Best For

Building machine learning pipelinesData transfer and ETL workflowsInfrastructure management automationScheduling and monitoring complex workflows

Alternatives to airflow

Aiscraper

n8n node to call Parsera API for AI Scraping

Ai Agent Langfuse

n8n community node: AI Agent + Langfuse

Aiagenthub

n8n community node for AI Agent HUB — Publish posts, carousels, videos, Reels, Shorts & Stories to Facebook, Instagram, YouTube, LinkedIn & TikTok. Polling triggers (new comments, new DMs, token expir

agents

Intelligent automation and multi-agent orchestration for Claude Code

Activepieces

Open source automation tool

Advbox

n8n community node for ADVBOX API - manage customers, lawsuits, tasks, movements, transactions and settings