QA Engineer

QA Engineer – Data Pipeline, Kafka & AWS

Location New York City/Dallas TX/St. Louis MO Hybrid

Employment Type: Long-Term Contract

Job Description

We are seeking an experienced QA Engineer with strong expertise in Data Pipeline testing, Apache Kafka, AWS, API testing, and SQL. The ideal candidate will be responsible for validating data ingestion, transformation, processing, and delivery across distributed data platforms.

Key Responsibilities

• Design and execute QA test plans, test cases, and test scenarios for data pipelines.
• Perform ETL/ELT and data validation testing across source, transformation, and target systems.
• Validate Kafka topics, producers, consumers, messages, offsets, partitions, and data flow.
• Test real-time and batch data pipelines for accuracy, completeness, consistency, and reliability.
• Perform SQL-based backend/database testing and data reconciliation.
• Validate data processing and integrations across AWS cloud services.
• Perform API testing using tools such as Postman, REST Assured, or similar.
• Identify, troubleshoot, and document data discrepancies and defects.
• Work closely with Data Engineers, Developers, Business Analysts, and DevOps teams to resolve issues.
• Develop and maintain automated test scripts where applicable.
• Validate end-to-end data flow from source systems through Kafka and downstream AWS/data platforms.
• Participate in regression, integration, system, performance, and production validation testing.

Required Skills

• QA/testing experience, preferably in data-intensive applications.
• Strong experience in Data Pipeline / ETL Testing.
• Hands-on experience with Apache Kafka.
• Strong SQL skills for data validation and reconciliation.
• Experience testing AWS cloud-based applications and data pipelines.
• Experience with API/REST testing.
• Knowledge of batch and real-time data processing.
• Experience with defect management and test management tools.
• Strong understanding of SDLC, STLC, Agile/Scrum methodologies.