Data Quality Analyst

Data Quality Analyst

QA Strategy and Leadership:

Define and implement robust QA strategies, methodologies, and processes for data engineering projects.

Lead and mentor a team of QA engineers, ensuring alignment with best practices and project goals.

Collaborate with stakeholders to understand data quality requirements and translate them into test plans and cases.

Testing Frameworks and Automation:

Develop and maintain automated testing frameworks for ETL pipelines, data lakes, and data warehouses on Google Cloud Platform, Databricks, and related services.

Design data validation and verification processes to ensure the accuracy and consistency of data across pipelines.

Automate regression, performance, and integration testing to minimize manual efforts.

Data Quality Assurance:

Conduct root cause analysis for data quality issues and drive corrective actions.

Ensure compliance with data governance policies and data security best practices.

Develop data profiling and monitoring dashboards using tools like Google Monitor, Power BI, or custom solutions.

Technical Skills and Qualifications:

GCP Expertise: Proficient in GCP, Dataflow, Big Query, Cloud Composer, and Google Storage services.

Databricks: Experience with Spark-based data engineering workflows, including Delta Lake.

Testing Tools & Frameworks: Strong experience with tools like PyTest, dbt (data build tool), or similar testing frameworks for data pipelines.

SQL and Scripting: Advanced SQL skills for data validation with Python proficiency for automation.

Big Data Knowledge: Knowledge of Big Data processing and distributed computing. Understand the importance of data modelling concepts.

CI/CD: Knowledge of CI/CD pipelines on or GitHub Actions, focusing on data engineering workflows.

Data Governance: Familiarity with data governance tools and concepts such as metadata management, lineage, and data cataloging.

Communication: Excellent communication skills, with the ability to synthesize, simplify, and explain complex problems to different types of audiences, including executives