Consulting during deployment
Although the dmp.af library is excellent and easy to use solution, we are ready to provide consulting on its deployment.
Contact our teamFrequently Asked Questions
Everything you wanted to know, all on one page
dmp-af — General Information
What is dmp-af and what is it for?
dmp-af runs dbt models in parallel in Airflow; each model becomes a separate DAG task with dependencies between models and domains preserved. This speeds up pipelines and scales for large projects.
What are the key advantages?
Domain-oriented architecture with separate DAGs, “dbt-first” configuration directly from models, flexible scheduling (@hourly, @daily, @weekly, @monthly, etc.), support for multiple dbt targets, testing strategies, and Kubernetes integration.
What project sizes is it suitable for?
The project is designed for large dbt installations (1000+ models), but works correctly with projects of any size.
Installation and Getting Started
How do I install dmp-af?
Add the package to your Airflow environment using pip install dmp-af or include dmp-af in your Airflow cluster’s requirements.txt.
How do I create my first DAG?
Define the configuration in your dbt models, then generate the DAG in Airflow; dmp-af will convert the models into tasks considering dependencies and schedules.
Is special configuration needed in Airflow?
A standard Airflow installation with access to your dbt project and profiles is required; dmp-af is installed as a regular Python package and uses configuration from dbt.
Architecture and Capabilities
How does dmp-af integrate with dbt?
The configuration is stored within the dbt models themselves; dmp-af reads parameters and builds tasks and DAGs without manual changes to Airflow.
Does dmp-af support multiple environments and targets?
Yes, multiple dbt targets and corresponding run and test policies are available.
What schedule options are there?
You can set different schedules at the model level, including built-in periods like hourly, daily, weekly, and monthly.
Is Kubernetes supported?
Yes, operation in Kubernetes environments is provided for enterprise scenarios.
Performance and Scalability
How is parallelism and scalability ensured?
Splitting by domains and converting models into independent Airflow tasks allows for parallel computation without breaking dependencies.
Is it suitable for data mesh?
Yes, dmp-af is oriented towards domain-driven design and data mesh architectures.
Quality and Testing
How is testing organized?
dbt testing strategies are supported, along with guidance for running local tests and CI in the contribution documentation.
Can different test strategies be set for different models?
Yes, strategies are configured in dbt and read by dmp-af when building DAGs.
Contribution and Community
How can I participate in development?
Open an issue or submit a Pull Request according to the CONTRIBUTING guidelines; the project is supported by IJKOS & PARTNERS LTD and open to the community.
Where can I see the release and testing process?
The Contributing section contains detailed instructions on releases, code style, testing, and the PR workflow.
