Data Engineer

Role Description

We are looking for a Data Engineer who enjoys solving complex problems and building scalable data solutions. This person will play a key role in designing, developing, and optimizing data platforms that support analytics, reporting, machine learning, AI initiatives, and other data-driven business needs.

The ideal candidate is comfortable working on new projects, contributing to technical architecture decisions, and collaborating with cross-functional teams to deliver end-to-end solutions. Strong communication skills are important, as you'll need to explain technical concepts to both technical and non-technical audiences and work closely with clients, stakeholders, and team members.

Essential Functions:

• Assemble large, complex data sets that meet business requirements through extraction, transformation, and loading of data from a wide variety of data sources.
• Provide operational support and troubleshooting for existing processes and systems.
• Work closely with architects, solution leads, data owners, Data Scientists and key stakeholders to facilitate and coordinate the data platform backlog grooming process, triaging new feature requests in preparation for future project activities.
• Deliver automation & efficient processes to ensure high quality throughput & performance of the entire data & analytics platform.
• Ensure data extraction, transformation and loading data meet data security & compliance requirements.
• Engage with data source platform leads to gain tactical and strategic understanding of data sources required by Agency Data Services AI/ML as well as Data Office standards.
• Create data tools for data scientist team members that assist them in building and optimizing models.

Qualifications

• BS degree in Computer Science, Data Science, Engineering, or equivalent software/services experience required.
• 4+ years working with SQL, Snowflake, Databricks, Spark, and other big data technologies.
• 4+ years using Python, SQL, PySpark, R, or similar languages and manipulating, processing, and extracting value from large, disconnected data sets.
• 4+ years building and optimizing data pipelines, architectures, and data sets to answer business questions and identify opportunities for improvement.
• 2+ years supporting large-scale data processing and storage using Azure Data Factory, Integration Runtime, Data Lake, Databricks, Spark, Azure ML, and Cosmos DB.
• 2+ years addressing privacy, compliance, and security aspects of data storage and processing; and delivering data solutions in Agile environments.
• 2+ years with software development and CI/CD methodologies and tools for automated infrastructure code and MLOps and designing, implementing, and maintaining automation platforms and tools, including Ansible Tower, Azure, ARM, Terraform Enterprise, Azure DevOps, and GitHub Actions.
• 2+ years with Salesforce FSC and Salesforce Data Cloud.

Requirements

• Strong communication and problem-solving skills.
• Troubleshooting expertise.
• Proficiency in Python, SQL.
• Experience with Extract, Transform, Load (ETL) processes.
• Experience building data pipelines.
• Experience with Apache Spark and Apache Hadoop.
• Experience with Amazon Web Services.

Benefits

• Health insurance
• Paid holidays
• Flexible time off
• 401k retirement savings plan and company match with pre-tax and ROTH options
• Dental insurance
• Vision insurance
• Employer paid disability insurance
• Life insurance and AD&D insurance
• Employee assistance program
• Flexible spending accounts
• Health savings account with employer contributions
• Accident, critical illness, hospital indemnity, and legal assistance
• Adoption assistance
• Domestic partner coverage