Apache Airflow Pros and Cons

Airflow is great for my python needs on certain projects with my colleagues. Amazing support of cloud storage providers. quick set up and ease of use has saved me time for my data allocations. Review collected by and hosted on G2.com.

Browser UI is very buggy and has issues sometimes during my projects, can make my work lag behind due to this flaw. Review collected by and hosted on G2.com.

This program is hard to begin using, but after learning and using it, I can say myself it’s so useful. It’s easy to track information like job statuses and track failures. The UI is easy and simple to follow Review collected by and hosted on G2.com.

There is no real security to this program, so that could be an issue. The learning curve could also be a downside because it takes a little while to learn and use. Review collected by and hosted on G2.com.

Easy to configure a new pipeline once you create a codebase. I currently use airflow to manage our extraction, transformation, enrichment and quality checks. Because most of our pipelines follow a similar format we can create a standard pipeline swapping out the extraction and transformation code individually. Review collected by and hosted on G2.com.

Because of the flexibility the learning curve for developing an airflow pipeline is a bit long, using an existing pipeline is very easy however development requires knowledge of python, bash. Review collected by and hosted on G2.com.

Perfect data Processing for all the product developments and easy to use once you get the hang of it. We rely on this program to keep track of our progress Review collected by and hosted on G2.com.

Some of the features are hard to figure out and seem less important to have. Wish there was more functions for each tab Review collected by and hosted on G2.com.

Apache Airflow is a great flexible tool that allows the data engineers/scientists to design data-intensive workflows efficiently. Airflow workflows are designed as DAGs (Directed Acyclic Graphs), each node of the graph can be anything (e.g. Python or bash script ...). Airflow manages the workflow DAGs and its scheduling and communication messages between the graph nodes efficiently. The best thing I like in Apache Airflow is that it is a simple Python tool that can design and execute very complicated workflows. In addition, Airflow has the ability to apply user-based strategies for the failed nodes in the graph. For example, the user can restart a task node if it is failed after x seconds, or send email to the administrator with the error message. Also, Apache Airflow has a user-friendly UI that allows the users to list the current running/stopped graphs and to display the logs of each task. Review collected by and hosted on G2.com.

In rare cases, Apache Airflow UI displays a stale information about the current running DAGs. For example, sometimes it shows that a certain graph is running, however, one of its tasks is already failed. Also, sometimes it shows that a workflow it is already finished, but one of its tasks still running in the background. In these cases, the user has to restart the Airflow server and clear the stale logs. Review collected by and hosted on G2.com.

The best part of Airflow is that it is open source and also its power to run on distributed environment. It provides out of the box operators (connectors) to plugin almost any data source. Review collected by and hosted on G2.com.

Language compatibility. Airflow requires one to have a knowledge of python language. Also its UI is not powerful and has glitches sometime. One has to write code to orchestrate workflows as it does not provide drag and drop feature. Review collected by and hosted on G2.com.

Airflow is a very easy to use tool that is very versatile and very well designed to meet user's needs. Setup is quick and easy, which makes it simple for engineers and scientists to implement and create workflows effortlessly. Review collected by and hosted on G2.com.

Many different variables and configuration essential in the function of the tool. One must be familiar with Airflow languages to successfully use. Occasional glitches. Review collected by and hosted on G2.com.

The interface is very clean and simple to use. Pretty universal to use. Review collected by and hosted on G2.com.

Software crashes when multiple tabs are running with high volumes of data. Review collected by and hosted on G2.com.

Easy to tailor work flow programs for the appropriate situation. Review collected by and hosted on G2.com.

Potential pitfalls with data security are a concern. Review collected by and hosted on G2.com.

A great data and workflow management tool that is easy to work with, very flexible Review collected by and hosted on G2.com.

Sometimes there is a delay in displaying the real status (showing that a task is running while it's already completed) Review collected by and hosted on G2.com.