Senior Data Engineer

Senior Data Engineer

Location: 100% Remote within the USA

Employment Type: Contract to Hire

Position Overview

We are seeking an experienced Senior Data Engineer to join our growing team and help deliver enterprise data engineering solutions for leading organizations.

This role offers the opportunity to work with modern data technologies, large scale data platforms, and complex data integration environments. The ideal candidate will bring strong hands on experience in ETL testing, data validation, SQL, Databricks, and cloud based data engineering platforms.

You will collaborate with data engineers, developers, business analysts, and technical stakeholders to ensure the accuracy, quality, and reliability of enterprise data solutions.

We offer a collaborative environment with access to global resources, diverse project opportunities, professional development, and the flexibility of working remotely from anywhere in the United States.

Key Responsibilities

• Design, develop, and execute test plans, test scenarios, and test cases for enterprise data platforms, ETL/ELT pipelines, and data integration solutions.

• Perform end to end ETL testing to validate data accuracy, completeness, transformation logic, reconciliation, and data quality across source and target systems.

• Develop and execute complex SQL queries for backend testing, data validation, and source to target reconciliation.

• Test large scale data processing solutions using Databricks, Apache Spark, PySpark, Python, Hive, Hadoop/HDFS, and Azure Data Factory.

• Perform functional, integration, regression, system, and data quality testing across enterprise data pipelines and applications.

• Validate batch processing, job scheduling, dependencies, and workflow execution using Control M and Unix/Linux environments.

• Identify, document, track, and retest defects while collaborating with developers, data engineers, business analysts, and other stakeholders.

• Develop and maintain reusable test scripts and automated testing solutions where applicable.

• Participate in Agile and Scrum activities, including sprint planning, backlog refinement, daily stand ups, sprint reviews, and retrospectives.

• Support continuous improvement of data testing processes, validation frameworks, and quality assurance practices.

Required Qualifications

• 8+ years of experience in software quality engineering or testing, with strong expertise in ETL testing, data validation, integration testing, and regression testing.

• Strong hands on SQL skills, including writing complex queries for source to target validation, reconciliation, and data quality testing.

• Experience testing large scale enterprise data platforms and ETL/ELT pipelines.

• Hands on experience with Databricks, Apache Spark, PySpark, Python, and Azure Data Factory (ADF).

• Experience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive.

• Experience working in Unix/Linux environments and with enterprise job scheduling tools such as Control M.

• Strong understanding of data quality concepts, data transformations, data mappings, and source to target validation.

• Experience with functional, integration, regression, and end to end data testing.

• Experience with defect management and test management processes within Agile/Scrum delivery environments.

• Strong analytical, troubleshooting, problem solving, communication, and collaboration skills.

Preferred Qualifications

• Experience with automated testing frameworks and reusable data validation scripts.

• Familiarity with cloud based data platforms and enterprise data integration architectures.

• Experience supporting production data environments and troubleshooting complex data pipeline issues.

• Ability to work effectively with cross functional technical and business teams.