Answer the question
In order to leave comments, you need to log in
Apache Airflow benchmark for simple frequently DAGs/tasks?
Taurus Hello, I am considering Apache AIrflow as one of the components in my project.
Can't find an approximate benchmark. Therefore, I have a number of questions.
Let's take for example a car with 8 cpu and 32 GB of RAM.
And Doug with such tasks. "curl REST" => "extract content" => "save to DB" => "send email"
- Is Apache AIrflow suitable for frequent tasks that will be executed every second?
- How many parallel tasks can be executed by such a computer per second?
- Approximately how much memory does such a DAG require?
- Will Apache AIrflow do well if for example 1000 Dags are called every 2-3 seconds?
Share your experience. Thank you!
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question