Senior Data Engineer

Data EngineerWe are seeking an experienced Data Engineer to design, build, and enhance data ingestion pipelines and metadata frameworks that support data lineage, modernization, and analytics enablement. The ideal candidate has hands-on experience with AWS Glue, Python/PySpark, ETL pipelines, and metadata management tools, with the ability to work in a dynamic, cloud-centric data environment.Key ResponsibilitiesDesign, develop, and maintain data ingestion and ETL pipelines for metadata integration across systems like SQL Server, Informatica, AWS Glue, and Azure Data Factory.Implement end-to-end data lineage tracing, including source-to-target mapping, column-level lineage, and job dependency tracking.Automate metadata ingestion and transformation for lineage reporting and quality validation.Identify static, orphaned, or unused tables/jobs and contribute to data-quality improvement.Collaborate with data architects, analysts, and AI strategist to align metadata ingestion with modernization goals.Participate in the deployment and hardening of solutions in AWS (and Azure as applicable).Contribute to modernization and AI-readiness initiatives through metadata standardization and automation.Required Skills & Experience8–10 years of experience in Data Engineering / ETL Development.Strong experience with: AWS Glue, S3, Redshift, LambdaSQL (SQL Server, Oracle, or Snowflake) – complex queries, stored procedures, performance tuningPython / PySpark for metadata parsing and ETL automationETL Tools: Informatica, AWS Glue, Azure Data FactoryData Lineage / Metadata Management: Collibra, Alation, or custom lineage repositoriesCI/CD & Orchestration: GitHub, Jenkins, or Azure DevOpsNice to HaveExperience with BladeBridge or similar lineage analysis toolsFamiliarity with Client Batch ProcessingExposure to Airflow, dbt, or Databricks for orchestration and transformationBasic understanding of AI/ML data pipeline readinessAzure data platform exposure (ADF, Synapse, Data Lake) Euclid