Senior Data Engineer

Description


Position at The Port Authority of NY & NJ


About the Role

The Aviation Department is seeking a Senior Data Engineer with a proven background in cloud-based enterprise data architecture, excellent troubleshooting and analytical abilities, and strong communication skills who is motivated to strengthen the technical foundation of the Department's data environment and contribute to the continued evolution of the Aviation Data Analytics Warehouse (ADAW). The Data Analytics & Business Intelligence (DABI) team serves as the centralized Data Science Unit for the Department and manages ADAW, a governed data environment that integrates more than fifteen data streams and provides information to more than 500 users across the agency. Reporting to the Program Manager, Data Analytics and Business Intelligence, the Senior Data Engineer will support the development, maintenance, and modernization of the team's data infrastructure; manage a team of data engineer contract staff; help strengthen the Department's technical capabilities; and deliver high quality analytical and engineering solutions that support airport operations, planning, customer experience, and strategic decision- making. This role is the technical backbone of a decision-support system that moves millions through our world-class airport system.


The successful candidate's responsibilities will include, but are not limited to:

• Data Engineering & Infrastructure Development (50% of work portfolio)Design, develop, and maintain ADAW's Azure-based data architecture, including Synapse ETL/ELT Pipelines, SQL pools, Azure Event Hub jobs, Databricks workflows, and blob storage accounts, ensuring data is reliable, well-structured, and aligned with departmental needs.
• Build, maintain and optimize shared Power BI semantic models in close collaboration with Senior Data Scientist and relevant stakeholders, ensuring efficient relationships, stable table structures, and performant DAX measures.
• Monitor and troubleshoot Azure Synapse performance, including CPU utilization, job schedule behavior, failed pipeline runs, and compute scaling needs; configure alerts and automated diagnostics using Azure Monitor, Log Analytics, and shared engineering dashboards.
• Develop and manage end-to-end data ingestion and integration processes in Azure, including high-volume ingestion of third-party message queues, automated API calls, and structured batch transfers from vendor systems, ensuring quality, completeness, and adherence to governance standards.
• Implement durable data validation, schema governance, and automated QA checks and ensure standardized structures, naming conventions, surrogate key patterns, and data-type/format rules.
• Drive innovation in data management practices, stay abreast of industry trends and emerging data warehousing technologies.
• Product Development & Technical Solutioning (35% of work portfolio)Manage team of contract data engineers to establish engineering best practices, manage task priorities, address questions around business logic, help with QAQC activities, and ensure on-time delivery of fixes and enhancements.
• Serve as the primary point of contact with the agency Technology Department to maintain cloud infrastructure, align with cybersecurity and network access requirements, and resolve Azure-related issues (e.g., compute throttling, storage permissions, service interruptions).
• Document dataset structures, key logic, and processing steps to ensure clarity and long-term sustainability of Aviation's data products.
• Help develop enhanced data dissemination methods (API endpoints, progressive web applications) for improved collaboration with other mature internal data teams.
• Support continuous improvement of DABI's engineering standards, internal documentation, business logic, and process transparency.
• Provide technical guidance to colleagues across the Department, including embedded analysts, to strengthen analytical and engineering capacity Stakeholder Engagement & Cross- Department Collaboration (15% of portfolio)
• Coordinate with Senior Data Scientist and Program Manager to provide project updates and align on business rules and technical requirements with stakeholders from other business units, operational teams, and external partners
• Support collaborative efforts with units such as Traffic Engineering, Redevelopment, and airport operations groups, ensuring alignment in data assumptions and analytic practices.
• Participate in recurring technical discussions, coordination meetings, and cross-team engagements to ensure ADAW data and analytical tools remain consistent, reliable, and responsive to departmental needs.

Minimum Qualifications

• Bachelor's degree in Computer Science, Data Science, Data Engineering, Information Systems, Applied Mathematics, or a related quantitative field.
• At least three (3) years of experience supporting data systems, analytics pipelines, technical workflows, or data engineering activities in an enterprise cloud environment (ideally Azure).
• Strong understanding of data architecture, dimensional data modeling, structured data warehousing, integration, ETL processes and analytics.
• A minimum of (3) years of experience building datasets, automating tasks through scripts, writing database queries, and debugging/ maintaining code.
• A minimum of (4) year of experience with Python and SQL programming.Minimum of (3) years of experience with relational databases to obtain and manipulate data.
• Experience designing or supporting structured data solutions or analytical tools for cross-functional business teams, ideally in a large organization or public agency.
• Ability to multitask, prioritize, and manage time effectively.
• Strong written and verbal communication skills and the ability to work collaboratively with technical and nontechnical partners across the Department.

Desired Qualifications

• Master's degree in a relevant technical or analytical field, ideally Data Science or Engineering, Information Systems, Software Engineering or Computer Science
• A minimum of (3) years developing enterprise-level Power BI semantic models and managing Power BI workspaces and resource capacity Experience in FinOps, usage monitoring, billing analysis, or creating automated diagnostic workflows in a Microsoft Azure environment
• Experience working with enterprise data systems or operational datasets in transportation, infrastructure, or public sector environmentsExperience developing or maintaining APIs, Webhooks or other data transfer protocolsKnowledge of MCP-optimized data warehousing best practices for integration with AI AgentsFamiliarity with geospatial data concepts or experience contributing to map-based analyticsCompetency in JavaScript, front end development, or web app development
• Experience improving data quality, documentation, or engineering processes within a collaborative team environment

Selection Process

The application process varies by position, but typically includes an initial phone interview for qualified candidates, followed by a more in-depth interview(s) and/or assessment(s). Selected candidates who are made a conditional job offer will be asked to undergo a background check.


Compensation, Benefits & On-Site Collaboration Requirements

The Port Authority of New York and New Jersey offers a competitive benefits package and a professional environment that supports development and recognizes achievement.


Click here for more information about benefits, our culture, and career development opportunities.


The Port Authority of New York and New Jersey anticipates that the actual salary offered to a successful candidate will depend on aspects such as experience, knowledge, skills, abilities, and internal factors.


The expected compensation range for this role is: Minimum: $100,256 Midpoint: $130,318 Maximum: $160,394


In accordance with Port Authority policy, this position permits employees to work remotely a maximum of one day per week.

In person collaboration is essential to how the Port Authority conducts business, and regular in-office days or team collaboration days may be required on business needs.

How to Apply:Interested candidates should apply to this job by clicking on the "Apply Now" button and submitting a resume. The Port Authority of NY & NJ welcomes veteran and military spouse applications. Only applicants under consideration will be contacted.About The Port Authority:Founded in 1921, the Port Authority of New York and New Jersey builds, operates, and maintains many of the most important transportation and trade infrastructure assets in the country. The agency's network of aviation, ground, rail, and seaport facilities is among the busiest in the country, supports more than 550,000 regional jobs, and generates more than twenty three billion in annual wages and eighty billion in annual economic activity. The Port Authority also owns and manages the 16-acre World Trade Center site, where the 1,776-foot-tall One World Trade Center is now the tallest skyscraper in the Western Hemisphere.Equal Opportunity EmployerThe Port Authority of New York & New Jersey/Port Authority Trans-Hudson (PATH) is an Equal Opportunity Employer.It is the Port Authority's policy, in accordance with the Americans with Disabilities Act (ADA) and other applicable laws, to provide reasonable accommodations to qualified persons with disabilities who are employees or applicants for employment and who request the Port Authority do so, unless providing the requested accommodation would result in an undue hardship to the Port Authority.Job applicants may request a reasonable accommodation (1) if they claim a disability is affecting the interview process, or (2) to ensure that if they are selected for a position, they can perform the essential job functions. To submit a request, please contact the recruiter or email the Human Resources Department's Disability Accommodation Coordinator at [email protected].The Port Authority will treat any request for Reasonable Accommodation by a job applicant confidentially.