Data Modeling and Transformation: Design, build, and maintain robust and scalable data models using dbt and SQL. Create reusable transformations and ensure data is structured for efficient analysis.
Pipeline Development: Develop and optimize ELT pipelines, leveraging dbt for transformations and Databricks for compute and orchestration when needed.
Quality Assurance: Implement and manage data quality tests withindbt to ensure data integrity, accuracy, and consistency.
Documentation and Governance: Maintain comprehensive documentation for dbtmodels, data lineage, and dependencies.
CI/CD and Version Control: Apply software engineering best practices using Git and set up CI/CD pipelines for automated dbtproject deployments.
Collaboration: Partner with data analysts, data scientists, and business stakeholders to deliver clean, reliable datasets for BI and ML use cases.