Senior Data Engineer

Senior Data Engineer, Automation

Location : Chicago, downtown

* Please note this position requires 3 days onsite; local candidates only

Overview:

This position owns the movement, shaping, and quality checking of data that arrives through file-based integrations. The work centers on building and maintaining scripts that automate file extraction, handling both structured and semi-structured files, and supplying the datasets and reports that teams across the agency depend on.

Responsibilities

Data Preparation and Management (20%)

• Clean and standardize high-volume Excel, CSV, and other flat files containing claims and administrative data
• Detect and fix duplicate records, inconsistencies, and other data quality problems
• Reshape files into the layouts and formats that downstream systems expect
• Load, stage, update, and extract data in Snowflake, running jobs both manually and on a schedule
• Move rate data through multiple processing stages, produce outbound files, and upload the resulting CSVs to EXL
• Apply proven patterns for ingestion, transformation, modeling, and orchestration in a modern lakehouse environment

PowerShell Automation (40%)

• Write new PowerShell scripts and improve existing ones
• Automate file transfers, FTP processing, folder management, and email alerts
• Create automated solutions that make day-to-day operations more efficient

SQL Server Development and Python Scripting (40%)

• Write and tune SQL queries, including complex joins and data transformations
• Update backend tables and apply data corrections
• Build reports and datasets, and recommend ways to support analytics and operational needs
• Troubleshoot databases and validate data
• Write Python scripts that automate file movement, copying, editing, and integration work
• Own the end-to-end automation of data and file workflows
• Profile queries ahead of time, tune execution plans, design effective indexes, and set up partitioning to cut latency in compute-heavy workloads

Requirements:

• Bachelor's degree
• 5+ years of experience in ETL/ELT pipeline development, data integration, and process automation
• Solid SQL Server experience, including complex queries, reporting, and data updates
• Hands-on experience loading, staging, and extracting data in Snowflake
• Proficiency in PowerShell for automation and system integrations
• Proficiency in Python for automation, file handling, and system integrations
• Experience working with FTP servers and integrating with external systems
• Expert knowledge of data quality, validation, and transformation