Apache Airflow Reviews & Product Details

Apache Airflow is an open-source platform designed for authoring, scheduling, and monitoring complex workflows. Developed in Python, it enables users to define workflows as code, facilitating dynamic pipeline generation and seamless integration with various technologies. Airflow's modular architecture and message queue system allow it to scale efficiently, managing workflows from single machines to large-scale distributed systems. Its user-friendly web interface provides comprehensive monitoring and management capabilities, offering clear insights into task statuses and execution logs. Key Features: - Pure Python: Workflows are defined using standard Python code, allowing for dynamic pipeline generation and easy integration with existing Python libraries. - User-Friendly Web Interface: A robust web application enables users to monitor, schedule, and manage workflows without the need for command-line interfaces. - Extensibility: Users can define custom operators and extend libraries to fit their specific environment, enhancing the platform's flexibility. - Scalability: Airflow's modular architecture and use of message queues allow it to orchestrate an arbitrary number of workers, making it ready to scale as needed. - Robust Integrations: The platform offers numerous plug-and-play operators for executing tasks across various cloud platforms and third-party services, facilitating easy integration with existing infrastructure. Primary Value and Problem Solving: Apache Airflow addresses the challenges of managing complex data workflows by providing a scalable and dynamic platform for workflow orchestration. By defining workflows as code, it ensures reproducibility, version control, and collaboration among teams. The platform's extensibility and robust integrations allow organizations to adapt it to their specific needs, reducing operational overhead and improving efficiency in data processing tasks. Its user-friendly interface and monitoring capabilities enhance transparency and control over workflows, leading to improved data quality and reliability.

Seller

The Apache Software Foundation

Discussions

Apache Airflow Community

Overview by

VIKASH KUMAR GUPTA (AZURE DATA ENGINEER at Capgemini Technology Services India Limited)

Value at a Glance

Averages based on real user reviews.

Time to Implement

3 months

Return on Investment

6 months

View More Pricing Information

Top-Rated Alternatives

UiPath Agentic Automation

View All Alternatives

Apache Airflow Integrations

(24)

Integration information sourced from real user reviews.

G2 reviews are authentic and verified.

Here's how.

Verified User in Financial Services

Small-Business (50 or fewer emp.)

10/12/2023

"Amazing for daily tasks"

4/5

What do you like best about Apache Airflow?

Airflow is the most intuitive interface for setting up daily workflow jobs that I've come across. The API's are mostly easy to learn/use and it's all I love that it's all in Python. There are a few people on my team who are not trained programmers but they have figured out how to create simple daily jobs. The web interface is can be a bit obtuse but it gets the job done. Using the workflow visualizer makes debugging complex jobs much easier. Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

I wish it were easier to set up jobs that can be manually triggered. It can technically be done but the interface is clunky and lacks some basic quality-of-life features.

The only complaint I have with the actual coding is that Jinja is hard to learn and debugging it can be a nightmare. That being said, if you stay within the straight-forward use cases, you shouldn't have any issues. Review collected by and hosted on G2.com.

Raghwendra S.

SDE 4

Enterprise (> 1000 emp.)

12/6/2023

"best orchestrator in the market , with a huge community and active developments"

5/5

What do you like best about Apache Airflow?

It support , operators for almost every data engineering tool / framework. Highly scalable. Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

There is plenty of options for observability in airflow .

If some more community grafana dashboards or best practices are provided will help even further Review collected by and hosted on G2.com.

Anurag J.

Associate software Architect

Enterprise (> 1000 emp.)

5/31/2023

"Efficient and Reliable Workflow Orchestration with Apache Airflow"

4.5/5

What do you like best about Apache Airflow?

1) Workflow Orchestration: Apache Airflow provides a powerful framework for defining, scheduling, and executing complex workflows.

2) Workflow Orchestration: Apache Airflow provides a powerful framework for defining, scheduling, and executing complex workflows.

3)Monitoring and Alerting: Airflow provides a user-friendly web interface that allows users to monitor the status and progress of their workflows.

4)Active Community and Ecosystem: Apache Airflow has a vibrant and active open-source community.

5)Mature and Production-Ready Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

1) Learning Curve: Apache Airflow has a steep learning curve, especially for users who are new to workflow orchestration concepts or Python programming.

2) Complexity for Simple Use Cases: Airflow's feature-richness and flexibility can sometimes feel overwhelming for simple use cases. Review collected by and hosted on G2.com.

What problems is Apache Airflow solving and how is that benefiting you?

1) Complex Workflow Management: Airflow helps manage and coordinate complex workflows by providing a centralized platform to define, schedule, and monitor tasks and dependencies

2) Dependency Management: Airflow handles dependency management between tasks, ensuring that each task is executed only when its dependencies are met.

3)Scheduling and Retry Mechanisms: Airflow offers robust scheduling capabilities, allowing users to define precise schedules for task execution, including interval-based schedules, cron-based schedules, or specific trigger-based schedules.

4)Monitoring and Alerting

5) Parallel Execution and Scalability Review collected by and hosted on G2.com.

Farhan K.

Big Data Engineer

Small-Business (50 or fewer emp.)

8/10/2023

"Best tool for data flow"

4.5/5

What do you like best about Apache Airflow?

The flexibility and customizability when it comes to creating and scheduling data pipelines.

Uses python, which is the most popular programming language in the world. Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

Not really meant for streaming applications but it can be set up for those.

Has a bit of a learning curve compared to other solutions.

It only supports python for creating dags Review collected by and hosted on G2.com.

Ashita K.

Software Engineer

Enterprise (> 1000 emp.)

10/3/2023

"Best scheduling platform, easy to code in python."

5/5

What do you like best about Apache Airflow?

Use of cron tab expression, importing various modules, importing user defined operators. Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

declaring dag in a fixed pattern or else the scheduler won't pick up ypur dag and show import error Review collected by and hosted on G2.com.

Ishant T.

Mid-Market (51-1000 emp.)

7/18/2023

"Powerful Tool"

5/5

What do you like best about Apache Airflow?

Airflow is very scalable

Dynamic Pipeline integration

We can easily define our own operator by extending pre defined libraries

We can connect Airflow with so many applications and Data Warehouses like Databricks, MySQL and so on Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

User Interface Struggles

It is sometime hectic to manage the metadata database of Airflow

Performance Struggles sometimes when we create numerous tasks

Limited built in features Review collected by and hosted on G2.com.

Vinol D.

Head of Data

Mid-Market (51-1000 emp.)

9/11/2023

"Best for someone starting on data warehouse"

4/5

What do you like best about Apache Airflow?

- Best open source software to get started on

- Great material online to troubleshoot and community Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

- Needs a dedicated data engineer and devops

- Maintianence could take lot of time

- needs another tool for data quality measurement Review collected by and hosted on G2.com.

Anup J.

Enterprise (> 1000 emp.)

3/30/2023

"Airflow: Needs a refresher for the modern data stack"

4/5

What do you like best about Apache Airflow?

The best thing about Airflow is how versatile of a tool it is. Airflow can be used to build workflow on just about every database and tool and the sheer wealth of integrations it has is brilliant and just all-round useful Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

Learning Airflow is a seriously complicated task. And even that is not often enough to become truly good at it. The scheduling system is hardly intuitive. Versioning for the Dags and reverting them a very simple task in a competitor Prefect is not a part of Airflow at all Review collected by and hosted on G2.com.

Verified User in Oil & Energy

Small-Business (50 or fewer emp.)

6/22/2023

"I have used Airflow for developing pipeline in my recent company"

4/5

What do you like best about Apache Airflow?

Python is my favorite coding language, the DAGs in Airflow are written in Python. There are several built-in operators in Airflow to execute the Python function, call Databricks job and execute bash commands. I love to build the pipeline in Airflow Review collected by and hosted on G2.com.

What do you dislike about Apache Airflow?

The Airflow orchestration tool is a bit complicated in developing compared to other pipeline tools. While other tools have drag-and-drop options, coding in Airflow takes more time. Review collected by and hosted on G2.com.