Sr. Data Integration Engineer

Senior Data Integration EngineerThe Senior Data Integration Engineer is responsible for the design, development, and operation of the data integration and ingestion processes that deliver partner and internal data into our analytics environment. The role owns the flow of data from source acquisition through the curated data warehouse tables consumed by reporting platforms, operational systems, and business analysts.This is a senior, hands-on engineering position spanning the full integration lifecycle: acquiring data from a wide range of external partner and internal systems, validating and conditioning that data on arrival, transforming it into the structures that support analysis and operations, and operating those processes reliably in production. The role is concerned equally with building new integrations and with the continued performance, accuracy, and timeliness of those already in service.A central objective of the position is to advance reusable, well-instrumented integration patterns that shorten the time required to onboard new data sources and that improve the reliability and transparency of data delivery to the business. The Senior Data Integration Engineer will also contribute substantially to the planned modernization of our data platform, evaluating and recommending tooling, architecture, and migration approach for leadership consideration.The role sets technical direction and development standards for data integration work and collaborates closely with data architects, business analysts, stakeholders across the organization, and the technical contacts of our external data partners.Essential Duties and Responsibilities:Design, develop, and maintain data integration processes that acquire data from partner and internal sources, including flat file transfers over SFTP, REST API endpoints, and direct database connections.Develop reusable, configuration-driven ingestion patterns that reduce the effort and elapsed time required to onboard new partner data feeds.Develop and maintain the T-SQL transformation logic that carries data from landing and staging layers through to the curated warehouse tables supporting reporting, operational systems, and analyst queries.Design ingestion processes to be idempotent and safely re-runnable, incorporating automated retry and restart behavior for failed executions.Implement automated validation and quarantine processes so that records failing business-defined quality rules are isolated, reported, and prevented from reaching downstream consumers.Implement data quality rules defined by the business, including schema validation, reconciliation, row count and threshold checks, and anomaly detection.Establish monitoring, logging, and alerting for pipeline execution state, data freshness, and load completion, and automate the communication of ingestion status to stakeholders.Diagnose and resolve production data incidents, determine root cause, coordinate remediation, and document preventive measures through runbooks and post-incident review.Contribute to dimensional data model design in collaboration with the data architect and senior team members.Establish and maintain version control, code review, and repeatable deployment practices for database and pipeline code.Define and uphold development standards for data integration work through code review and technical guidance.Evaluate and recommend tooling, architecture, and sequencing for the platform modernization effort for leadership consideration.Migrate established integration workflows to modernized patterns incrementally and without disruption to production operations.Maintain documentation of data feeds, dependencies, lineage, ownership, and escalation paths.Perform all work involving protected health information in accordance with HIPAA requirements, including least-privilege access, secure transmission and storage of partner data, and the exclusion of PHI from logs and non-production environments.Coordinate with partner technical contacts, as needed, to resolve file format, schema, and connectivity questions.Provide occasional off-hours support for critical data load failures or production support rotations as needed to support timely response to critical data issues.Support AI and machine learning initiatives by maintaining reliable, secure, and well-governed data pipelines and datasets used for model development, testing, deployment, monitoring, and ongoing performance evaluation.Maintain confidentiality of information processed & follow company policies and procedures.Qualifications:Requires six (6) or more years of professional data engineering, data operations, data platform operations, or related experience. Bachelor's degree in Computer Science, Engineering, Information Systems, or related field, or equivalent professional experience preferred. Experience leading teams and developing supervisory staff preferred.To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.Other qualifications include:Advanced T-SQL development skills, including set-based rewriting of row-by-row and cursor-based logic.MERGE, upsert, and slowly changing dimension load patterns.Window functions and complex analytic queries.Execution plan analysis, index strategy, statistics management, and resolution of performance issues such as parameter sniffing.Transaction management and structured error handling within stored procedures, including correct rollback behavior on partial failure.Demonstrated proficiency in Python for data engineering applications, including API-based data acquisition, file parsing and format handling, data validation, and the development of packaged, scheduled jobs. Familiarity with common data libraries such as requests and pandas is expected.Demonstrated experience acquiring and integrating data from heterogeneous sources, including delimited, fixed-width, JSON, and XML file formats; REST APIs requiring authentication, pagination, and rate-limit handling; and direct database connectivity.Proficiency with Git and collaborative development workflows, including branching, pull requests, and code review.A code-first development approach, with integration logic authored and maintained in T-SQL and Python under source control.Ability to analyze pipeline and query performance and to improve the reliability, scalability, and cost efficiency of data workloads.Strong written communication skills, with the ability to produce runbooks, technical documentation, and incident reports, and to convey the business impact of technical issues to non-technical stakeholders.Strong problem-solving skills, attention to detail, and demonstrated ownership of production systems.Experience with Microsoft Azure data services such as Azure Data Factory or Microsoft Fabric, or comparable cloud orchestration platforms.Experience migrating on-premises SQL Server integration workloads to a cloud platform.Experience with dimensional modeling and data warehouse design.Experience designing and rationalizing SQL Server Agent job dependencies and scheduling.Experience establishing version control, code review, and repeatable deployment practices for database code.Experience implementing CI/CD pipelines for database projects.Experience handling protected health information under HIPAA, or comparably regulated data under an equivalent framework preferred.Experience with healthcare or pharmacy data, including claims, eligibility, prescription, or delivery data preferred.Familiarity with data governance, metadata management, and data lineage practices.Physical Demands:The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions. (The phrases "occasionally," "regularly," and "frequently" correspond to the following definitions: "Occasionally" means up to 1/3 of working time, "regularly" means between 1/3 and 2/3 of working time, and "frequently" means 2/3 and more working time.)While performing the duties of this job, the employee is frequently required to sit; talk or hear; and use hands to handle, or touch objects or controls. The employee is regularly required to stand and walk. On occasion the incumbent may be required to stoop, bend or reach above the shoulders. The employee would rarely need to lift up to 25 pounds. Specific vision abilities required by this job include close vision, distance vision, color vision, peripheral vision, depth perception, and ability to adjust focus.Work Environment:The position is a hybrid position with 2 – 3 days office presence in Milwaukee, WI required. The colleague may perform work-related travel on rare occasions (less than 10%) with an emphasis on travel for impact.The above statement reflects the general details necessary to describe the principal functions of the occupations described and shall not be construed as a detailed description of all the work that may be inherent in the occupation.EEO Statement:SPS Health, LLC, and all its affiliates, are equal opportunity employers committed to diversity and inclusion in the workplace. We prohibit discrimination and harassment of any kind based on race, color, sex, religion, sexual orientation, national origin, disability, genetic information, pregnancy, or any other protected characteristic as outlined by federal, state