Senior Data Integration Operations Engineer

Northeastern University is seeking an experienced and technically skilled Sr. Data Integration & Operations Engineer to join our team. This role is responsible for the day-to-day management, monitoring, operational support, and optimization of the university’s data integration pipelines and processes. The role will oversee ETL/ELT workflows built on enterprise integration platforms, ensuring reliable data flow from a broad spectrum of university source systems into the data lakehouse and downstream point solutions used across the university. The position requires hands-on expertise in data integration platform administration, pipeline operations, data observability, incident management, and continuous improvement of integration processes in production environments. 24/7 business continuity This role requires availability outside of traditional working hours on a rotating basis to ensure continuous operation of critical AI systems and data pipelines. Responsibilities include monitoring system health, responding to alerts, troubleshooting performance issues, and implementing emergency fixes as needed. The ideal candidate must be able to quickly diagnose and resolve AI system and data pipeline incidents, prioritize issues based on business impact, and coordinate with technical teams to restore service. A strong commitment to system reliability and service continuity is essential for success in this position. Other duties as required This role requires flexibility in performing duties outside of the primary responsibilities to support the evolving AI ecosystem at the university. The ideal candidate must be adaptable and willing to take on additional tasks or projects as required, ensuring consistent and reliable AI and data pipeline operations. This may include assisting with knowledge management, documentation updates, user training, data preparation, or special projects related to AI system improvements. A problem-solving mindset and willingness to tackle emerging challenges are essential for thriving in this dynamic environment. This role is hybrid and in the office a minimum of three days a week to facilitate collaboration with both technical teams and operations staff. In-office presence enables effective coordination with support teams, direct access to infrastructure, and hands‑on troubleshooting of AI systems and data pipelines. Physical presence is particularly important for incident response, change management activities, and cross‑functional problem‑solving sessions that benefit from in‑person collaboration and real‑time communication, Applicants must be authorized to work in the United States. The University is unable to work sponsor for this role, now or in the future Minimum Qualifications Data Integration Platform Expe r ience: Hands‑on experience administering and operating enterprise data integration platforms, with Informatica PowerCenter or IDMC (Intelligent Data Management Cloud) strongly preferred. Experience with SaaS‑based ELT tools such as Fivetran is a plus. Candidates should demonstrate the ability to manage complex integration workflows, configure connectors, and troubleshoot pipeline failures end‑to‑end. Data Pipeline Operations : Extensive experience maintaining, scheduling, and troubleshooting data integration pipelines that extract from enterprise source systems (ERP, SIS, CRM, HR, finance) and load into data lakehouse and downstream operational applications. Strong SQL/Python skills are required for data validation, troubleshooting, and ad hoc investigation of pipeline issues. Familiarity with lakehouse architecture concepts (medallion architecture, incremental loads, schema management) is expected Data Observability and Pipeline Monitoring: Experience with data observability platforms (such as Monte Carlo, Acceldata, Anomalo, or Datafold) or equivalent pipeline monitoring tools that track data freshness, volume, quality, and schema changes strongly preferred. Proficiency in designing alerting frameworks that surface meaningful signals without generating excessive noise. Incident Management : Strong experience in troubleshooting, diagnosing, and resolving AI system and data infrastructure issues, with the ability to prioritize incidents based on business impact. Performance Optimization : Knowledge of techniques to optimize AI system and data pipeline performance, including resource allocation, scaling strategies, and performance tuning Change Management: Experience implementing changes to production AI systems and data pipelines with minimal disruption, including testing, validation, and rollback procedures Data Quality Management : Strong understanding of data quality principles as they apply to integration pipelines, including detection and remediation of issues such as missing records, null rates, duplicate data, schema drift, and late‑arriving data. Ability to identify data quality failures before they affect downstream analytics consumers or operational applications Documentation and Knowledge Management : Excellence in creating and maintaining operational documentation, runbooks, and knowledge articles for AI systems and data pipelines Automation Skills : Ability to create and implement automation scripts and workflows to streamline routine operational tasks for both AI systems and data flows, enhancing overall system reliability DevOps Practices : Familiarity with DevOps and CI/CD principles as applied to AI systems and data pipelines, including containerization, orchestration, and infrastructure as code Security Awareness : Understanding of security best practices for AI operations and data handling, including access control, data protection, and vulnerability management Collaboration Skills: Strong ability to work with cross‑functional teams, communicate technical concepts clearly, and coordinate incident response activities effectively Problem‑solving: Excellent analytical and problem‑solving skills, with the ability to troubleshoot complex issues in AI systems and data infrastructure in a methodical and efficient manner Compliance Knowledge: Understanding of relevant regulations and compliance requirements affecting AI systems and data processing in higher education environments Communication Skills : Clear and concise communication abilities, both written and verbal, to document procedures, report incidents, and coordinate with stakeholders Service Management : Knowledge of IT service management principles and frameworks, with experience applying them to AI and data pipeline operations Bachelor’s degree in Computer Science, Information Technology, Data Management, or a related field; technical certifications in relevant areas (e.g., Informatica, cloud data platforms, data engineering) preferred Minimum of 4–5 years of experience in data integration, data engineering operations, or a closely related IT operations role, with demonstrable hands‑on experience operating enterprise ETL/ELT pipelines in a production environment Experience with cloud platforms (AWS, Azure, or GCP) and familiarity with cloud‑based data lakehouse or data warehouse platforms (e.g., Snowflake, Databricks, Microsoft Fabric, or Delta Lake). Understanding of data lakehouse architecture principles including medallion architecture, incremental load patterns, and schema evolution Key Responsibilities & Accountabilities Pipeline Monitoring, Observability, and Incident Management Monitor data integration pipeline health, data freshness, volume trends, and job completion status using observability tools and dashboards. Proactively detect anomalies — such as late‑arriving data, row count deviations, schema changes, or silent failures — before they cause downstream impact to the lakehouse or operational applications. Detect, triage, and resolve incidents in a timely manner, coordinating with source system owners and technical teams as needed Operational Support and Maintenance Administer and maintain data integration platform environments (Informatica and related tools), including job scheduling, connector configuration, data refreshes, and platform patching. Manage integration jobs that feed both the data lakehouse and downstream point solutions used across the university. Implement scheduled maintenance activities with minimal disruption to dependent systems, and manage user access and permissions according to security policies Performance Analysis and Optimization Analyze integration pipeline performance metrics, identify bottlenecks, long‑running jobs, and resource contention, and implement tuning and optimization measures. Contribute to the evaluation and implementation of the university’s data observability platform, helping define the monitoring strategy, key metrics, SLA thresholds, and alerting rules that will govern pipeline health across the integration landscape Documentation and Knowledge ManagementCreate and maintain comprehensive operational documentation, including runbooks, standard operating procedures, and knowledge base articles. Document system configurations, data pipeline dependencies, and recovery procedures to ensure operational continuity Continuous Improvement and Automation Identify opportunities to automate repetitive operational tasks, improve pipeline reliability, and reduce manual intervention. Develop and implement scripts and workflows to streamline routine integration operations. Contribute to the ongoing evaluation of integration tools (including Fivetran) and the evolution of the university’s data integration practices based on operational experience and emerging best practices. #J-18808-Ljbffr Northeastern University