Apache Airflow Pros and Cons

Apache proxy was most useful for me as I could connect different servers easily. Review collected by and hosted on G2.com.

Nothing as such I have found out at this moment. Review collected by and hosted on G2.com.

Apache airflow is a freeware that executes components written in python modules. The written python modules are called DAGs. The DAGs need to configured with appropriate connections for successful execution. The connections and variables are easily configurable. The User Interface aspect of the tool makes the visualization the most attractive aspect of the tool. The triggering of the respective DAGs, status of execution, success and failure notification in green and red colors etc are some of the best aspects of Apache airflow. The configuration of variables in the form of json files are easy and straightforward. The different instances of executions along with the individual DAGs statuses historically enable the users to track the logs successfully. The ability to backtrack the logs for all these instances makes this tool a commendable one. Functions-based categorization of the functionality is flexible. Review collected by and hosted on G2.com.

Apache airflow contains an intensive documentation that needs to be read and reviewed to ensure that the configuration works per your needs. Knowledge of python as a prerequisite is needed. Familiarity with Json file and format is required to some extent. The tool might create an impression of being a bit complex to many users who don't have any background on Python. The formatting of tasks, hooks etc could be a bit complex. Not much of examples are available on open forums and exploring the solution for an end to end functionality could be a bit challenging for new users. Review collected by and hosted on G2.com.

Different types of Operators and Trigger rules. Review collected by and hosted on G2.com.

Nothing as such but Web UI can be improved Review collected by and hosted on G2.com.

Once you know the strengths of Ariflow create data pipelines is easier than ever before, the data lineage now is super clear since Airflow has a lot of visual integrations Review collected by and hosted on G2.com.

UI has lots of bug still, some times the UI will show bad or elements will appear in weird places and it would become impossible to click what you need, It happens often enough that it bothers Review collected by and hosted on G2.com.

You can orchestrate many DAGs at once at different schedules and can easily go back in time to reproduce/replicate runs. Review collected by and hosted on G2.com.

The Airflow CLI is not as friendly as the Airflow UI. The documentation is sometimes not clear/misses details. Review collected by and hosted on G2.com.

Airflow is easy to get started with and can scale and do more things as your tech stack evolves. Airflow comes with a lot of helpful features out-of-the-box, such as the DAG visualizations and task trees. Furthermore, Airflow is very flexible. You can use it for more than just data transformation. We started running batch ML models in Airflow before we onboarded a third-party tool. Review collected by and hosted on G2.com.

There is nothing I dislike about Airflow. We started using it less because it requires more engineering support than other tools (that probably run on Airflow), but we will continue to use it because it is the most flexible tool for ETL. We wished that Airflow had a premium tier so that we could get more support. Review collected by and hosted on G2.com.

It will execute tasks in order , every task will have required resources. Review collected by and hosted on G2.com.

There will be no versioning for data pipelines. Lack of data sharing between tasks Review collected by and hosted on G2.com.

The tree view of your ETLs is a great way to visualize how well the DAG is functioning, if it ran when it was supposed to, and the order and dependency of tasks. Review collected by and hosted on G2.com.

Error messages are difficult to parse, and the navigation isn't intuitive. Review collected by and hosted on G2.com.

Reruns and backfills are super easy, flow chart for jobs is very intuitive, and it's all done in python! Review collected by and hosted on G2.com.

Clunky at times. Spinning up a new instance is difficult if you don't know terraform and can reuse the canned manifest/helm Review collected by and hosted on G2.com.