Senior Data Engineer

Description :

• Optomi, in partnership with a leading financial services organization, is seeking a Senior Data Engineer to support enterprise data modernization and large-scale data pipeline initiatives across a growing cloud-based data ecosystem. This role will focus on building and optimizing ETL/ELT pipelines, improving data quality and usability, and supporting governance and secure data-sharing practices across enterprise platforms. The ideal candidate brings strong experience with SQL, Python, Azure cloud services, and modern data engineering frameworks, along with the ability to work cross-functionally with business stakeholders, product owners, and technical teams.

Responsibilities :

• Design, build, and optimize scalable ETL/ELT pipelines supporting enterprise data products and analytics initiatives
• Modernize and support Azure-based data ecosystems and cloud-native data workflows
• Develop and maintain SQL, Python, and PySpark-based data transformation pipelines
• Support data quality, governance, lineage, and secure data-sharing initiatives across enterprise platforms
• Develop and optimize large-scale batch and streaming data workflows
• Build and maintain data models, ingestion pipelines, and transformation frameworks
• Support metadata management, cataloging, and critical data element tracking processes
• Implement and support CI/CD automation and DevOps workflows for data engineering operations
• Collaborate with business and technical teams to gather requirements and deliver scalable data solutions
• Troubleshoot and optimize pipeline performance, scalability, and operational reliability

Qualifications :

• 7+ years of experience in data engineering, ETL development, or data warehousing
• Strong proficiency with SQL and Python
• Experience building and maintaining ETL/ELT pipelines in enterprise environments
• Hands-on experience with Azure cloud services such as ADF, Synapse, ADLS, or related technologies
• Strong data analysis, troubleshooting, and problem-solving skills
• Experience supporting CI/CD automation and DevOps workflows
• Understanding of data governance, data quality, lineage, and secure data practices
• Excellent communication and cross-functional collaboration skills

Preferred :

• Experience with Apache Spark, PySpark, or Databricks
• Exposure to Delta Lake, Unity Catalog, ADLS, or Azure Data Factory (ADF)
• Experience supporting enterprise data catalogs, governance frameworks, or metadata management initiatives
• Experience working within regulated enterprise environments such as financial services, healthcare, or insurance
• Familiarity with modern cloud data warehouses and lakehouse architectures