QA Automation Engineer - Enterprise Data & AI [Remote]

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a QA Automation Engineer – Enterprise Data & AI based in the United States.

This role sits at the intersection of quality engineering, data validation, and modern cloud data platforms, ensuring the reliability and accuracy of enterprise-scale data pipelines. You will work within a Databricks-powered ecosystem to validate complex data transformations, ingestion flows, and business rules across large datasets.
The position plays a critical role in strengthening data quality frameworks that support analytics, reporting, and downstream business decision-making.
You will collaborate closely with data engineers and quality teams to identify issues, improve test coverage, and enhance automated validation processes.
A strong focus is placed on building scalable, reusable automation within CI/CD pipelines and ensuring consistent data integrity across environments.
This is a hands-on engineering role in a fast-paced, data-driven environment where precision and automation directly impact enterprise data trust.
You will contribute to the evolution of modern data quality practices across a cloud-based data platform.

Accountabilities:

• Execute and extend automated data validation tests within Databricks using Python, PySpark, SQL, and notebook-based frameworks.
• Validate end-to-end data pipelines, including ingestion, batch and incremental loads, transformations, joins, and business rule accuracy.
• Perform data reconciliation between source systems and target datasets to ensure completeness and consistency.
• Enhance and maintain existing data quality frameworks, including rule sets for accuracy, completeness, and reliability.
• Implement and monitor validation checks, thresholds, alerts, and exception handling mechanisms.
• Develop reusable and scalable automated test scripts aligned with enterprise data testing standards.
• Integrate automated tests into CI/CD pipelines (e.g., Azure DevOps) and ensure reliable execution across environments.
• Support testing activities across QA and staging environments, including defect triage and root cause analysis.
• Collaborate with data engineering and analytics teams to ensure data integrity for reporting and visualization tools such as Tableau.

Requirements:

• 5+ years of experience in QA automation, SDET, or data validation engineering roles.
• Strong hands-on experience with Databricks, including notebook development and data pipeline validation.
• Advanced proficiency in Python, PySpark, SQL, and data processing workflows.
• Proven experience in data reconciliation and large-scale data validation across enterprise systems.
• Experience building, extending, or maintaining data quality frameworks in complex environments.
• Familiarity with CI/CD pipelines such as Azure DevOps for test integration and execution.
• Strong analytical, debugging, and problem-solving skills with attention to detail.
• Ability to collaborate effectively with data engineers, QA teams, and cross-functional stakeholders.
• Experience with tools such as Azure Purview or Profisee MDM is a plus.

Benefits:

• Competitive compensation aligned with experience and expertise.
• Fully remote opportunity within the United States.
• Exposure to enterprise-scale data platforms and modern cloud data engineering practices.
• Opportunity to work with Databricks and advanced data quality frameworks.
• Collaborative engineering culture focused on data innovation and automation.
• Career growth in enterprise data engineering, QA automation, and AI-driven data ecosystems.
• Inclusive and flexible work environment supporting work-life balance.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.