Data Engineer

Data Engineer

Employer: Delta Dental Insurance Company

Location: 1130 Sanctuary Pkwy, Alpharetta, GA 30009; Must live within reasonable commuting distance from HQ and able to appear in office as required.

Salary Range: $130,000/yr - $196,500/yr

Responsibilities

• Design, build, and optimize data pipelines and workflows to integrate data from multiple on-prem and cloud-based sources into the Enterprise Data Platform (EDP).
• Hands on experience with Snowflake including data modelling, data lake, Lakehouse concepts, warehouse management, and performance tuning.
• Experience with Matillion and ADF (Azure Data Factory). Experience with BI and reporting tools (e.g., Tableau, Power BI).
• Experience with version control (Git), DevOps, and CI/CD processes.
• Work with EBX for master data management (MDM), ensuring consistent and governed data across systems.
• Collaborate with cross-functional teams to understand data requirements, identify opportunities for data integration, and propose efficient data engineering solutions.
• Implement data governance and quality control measures to ensure data accuracy, consistency, and integrity throughout the data lifecycle.
• Optimize and tune database systems and data processing frameworks for performance and scalability.
• Develop and maintain data ingestion frameworks for both batch and near real-time data processing.
• Mentor and provide guidance to associate data engineers, assisting in their professional development.
• Collaborate with data scientists and analysts to support their data needs, providing efficient access to relevant datasets.
• Develop and maintain data models, schemas, and documentation to facilitate data understanding and accessibility.
• Work closely with infrastructure and DevOps teams to ensure data systems are secure, reliable, and properly integrated with existing infrastructure.
• Perform data profiling, analysis, and data mining tasks to identify patterns, trends, and insights in large datasets.
• Stay up-to-date with emerging data engineering technologies and best practices and proactively recommend improvements to existing systems.
• Troubleshoot and resolve data-related issues, ensuring minimal disruption to data pipelines and systems.
• Collaborate with data privacy and compliance teams to ensure adherence to data privacy regulations and industry standards.
• Conduct performance monitoring and capacity planning to anticipate and address future data engineering needs.
• Participate in code reviews, performance tuning, and data pipeline optimization for efficiency and scalability.
• Participate in data architecture discussions and provide input on data storage, data flow, and data integration strategies.
• Work in an Agile development environment with continuous integration and deployment (CI/CD) best practices.
• Continuously monitor and optimize data engineering processes, striving for efficiency, reliability, and scalability. Must live within reasonable commuting distance from HQ and able to appear in office as required.

Qualifications

• Bachelor's degree or foreign degree equivalent in Computer Science or related field and five (5) years of progressive, post-baccalaureate experience in Data Integration and ETL development, Cloud Data Platforms, Data Engineering, Data Analytical Models, Data Governance, Power BI reporting and EBX for enterprise data management or related roles in the job offered.
• Design and build scalable data pipelines using Matillion for Enterprise Data Platforms.
• Design and implement data models and curated datasets on Snowflake to enable analytics, reporting, and business insights.
• Configure and maintain EBX data models to create and manage business workflows and implement data governance rules to support enterprise-wide master data management.
• Collibra to manage data definitions, lineage, ownership, and data quality rules and ensure data assets are properly documented, governed, and compliant with enterprise standards.
• Power BI to build dashboards, scorecards, and analytical insights that support business decision-making and enable enterprise reporting and self-service analytics.
• Experience with data science to apply scientific methods, statistical techniques, and algorithms to extract insights from structured and unstructured data.
• Python to develop data engineering solutions, automation frameworks, and advanced data processing workflows ensuring enhanced scalability, reliability, and efficiency across Enterprise Data Platforms.