Senior Data Engineer

Data EngineerResponsibilitiesLead the end-to-end migration (transformation and load) of high-volume, high-complexity data into a new PostgreSQL-based infrastructureDesign, build, and maintain robust ETL/ELT pipelines capable of processing millions of records reliably and efficientlyMap and reconcile complex legacy data structures across multiple business entities into a unified target schemaDefine and implement data validation frameworks to ensure integrity, completeness, and accuracy throughout the migrationOptimize pipeline performance, including query tuning, indexing strategy, and batch/incremental load design in PostgreSQLIdentify, troubleshoot, and resolve data quality issues, schema mismatches, and pipeline failuresDocument data mappings, transformation logic, and migration runbooks for engineering and business stakeholdersPartner with business and technical stakeholders to align migration scope, timelines, and data requirementsEstablish monitoring, logging, and alerting to track pipeline health and data quality post-migrationMentor junior data engineers and contribute to engineering best practices and standardsRequired Qualifications5+ years of experience in data engineering, with demonstrated ownership of large-scale data migration projectsStrong to expert-level proficiency in Python for building and automating data pipelinesDeep hands-on experience with PostgreSQL/Oracle, including schema design, query optimization, and performance tuningProven experience designing and managing ETL/ELT pipelines at scale (millions of records)Experience mapping and transforming complex, legacy data structures across disparate systems or business entitiesStrong understanding of data validation, reconciliation, and quality assurance techniquesSolid grasp of data modeling principles (normalization, indexing, partitioning)Experience with version control (Git) and CI/CD practices for data pipelinesPreferred QualificationsExperience with orchestration tools (e.g., Airflow, Dagster, Prefect)Familiarity with cloud data platforms (AWS, GCP, or Azure)Experience with other relational or NoSQL databases and cross-database migrationsBackground working in regulated or high-stakes data environments (finance, healthcare, etc.)Experience with containerization (Docker) and infrastructure-as-codeExposure to data quality/testing frameworks (e.g., Great Expectations, dbt tests)What Success Looks LikeA fully migrated, validated dataset in PostgreSQL with zero critical data loss or corruptionETL/ELT pipelines that are documented, repeatable, and optimized for ongoing operationA clear audit trail of data lineage and validation results across all business entities involvedDiverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.