Senior Data Engineer

Senior Data EngineerLocation: Remote Duration: Contract to hireProfile: Data Quality Engineer (AWS Data Platform)Overview: We are seeking a highly skilled Data Engineering - Quality Engineer to define and implement end-to-end testing strategies for a modern data platform built on AWS. This role will be responsible for ensuring data quality, reliability, and performance across the entire pipeline; from ingestion to transformation and reporting.About the Role: The Assistant Director, Strategic Higher Education-Enrolment Leadership (Temporary) provides high-level, confidential administrative and logistical support to ensure smooth daily operations and to facilitate flawless execution of member-focused engagements. This position plays a key role within the SHE unit, with responsibilities closely aligned to existing full-time assistant director roles. This position carries limited responsibility for budget planning, forecasting. Reporting to the Executive Director, this role manages complex calendars, meetings, communications, travel, and materials for internal and external engagements hosted by SHE. Serving as a trusted partner and key point of contact for SHE, the Assistant Director anticipates needs, solves problems proactively, and helps unit leaders stay focused on their most critical work while maintaining alignment across priorities in collaboration with Assistant Directors and the Senior Director for Strategic Higher Education.Team: Team is Client delivery platform and Cumulus – lots of data exchanges: AWS platform using Redshift as data warehouse – Dynamo of meta-data. They are working on evolving code for digital SAT products and need dynamic Senior Data engineer who will also design, build and write test automation.Responsibilities:Define the end-to-end testing scope based on solution architecture and project documentationDesign and implement a comprehensive testing strategy and plan aligned with organizational QA standardsDevelop and maintain test scripts and frameworks for the Redshift serverless platformPerform testing across key technologies, including:AWS RedshiftAWS DMS (Data Migration Service)AWS GluePySpark DeequEvent BridgeData LakesPython-based data pipelinesApache Airflowdbt (data build tool)Build and implement automated testing solutions to ensure:End-to-end data validationData ingestion accuracyTransformation logic integrityData pipeline reliabilityConduct test coverage analysis and ensure adequate validation across all data engineering workflowsPrepare and manage test dataReview and provide feedback on:Solution architectureData modelsDesign and technical documentationCollaborate with cross-functional teams (Data Engineering, BI, DevOps, Product) to:Identify testing impactsMitigate risksEnsure high-quality deliverablesRequired Skills:Proven experience in data engineering testing / data QA / ETL validationStrong hands-on experience with AWS data services (Redshift, Glue, DMS)Proficiency in Python for test automation and validationExperience with Airflow and orchestration testingHands-on experience with dbt and data transformation validationFamiliarity with CDK for infrastructure validationExperience in BI testing in Quicksuite will be highly beneficialExperience with data quality tools such as PySpark Deequ or similarStrong understanding of:Data warehousing conceptsETL/ELT pipelinesData validation techniques (schema, reconciliation, anomaly detection)Preferred Qualifications:Experience designing enterprise-level test strategies for data platformsKnowledge of CI/CD pipelines for data and test automationExperience working in Agile / Scrum environmentsFamiliarity with data observability frameworks.EEO: "Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans."