Data Engineer

Summary/Objective Reporting to the IT Manager, Data & Analytics, the Data Engineer is responsible for designing, building, and maintaining scalable data pipelines, data platforms, and data warehouse/data lakehouse architectures that support enterprise analytics, reporting, automation, and AI-enabled solutions. This role ensures that data is accurate, reliable, well-governed, secure, and optimized for consumption across Microsoft Fabric, Power BI, generative AI, AI agents, and future technologies. The Data Engineer will enable trusted, AI-ready data foundations and support client- and vendor-facing technology solutions that are critical to business operations, growth, and service delivery. Essential Functions Design, build, and maintain robust ETL/ELT data pipelines to ingest, process, and integrate data from multiple source systems (e.g., CRM platforms, APIs, and external data sources) into Azure and Microsoft Fabric. Develop and manage the enterprise data platform, including lakehouse architecture (bronze, silver, gold layers), ensuring data is structured, scalable, and optimized for analytics. Ensure the quality, integrity, and consistency of data through validation processes, monitoring, and proactive issue resolution. Optimize performance and scalability of data workflows, pipelines, and storage to support efficient data processing and reporting. Implement and enforce data governance, security, and compliance standards, including data lineage, access controls, and regulatory requirements. Enable AI-ready and agentic enterprise capabilities by preparing governed datasets, metadata, lineage, and integration patterns that allow AI agents, automation, and decision‑support tools to operate safely and effectively. Support the design, delivery, and ongoing operation of client- and vendor-facing data integrations, reporting assets, APIs, and platform services that require reliable, secure, and scalable data engineering practices. Responsibilities & Duties Build and maintain data pipelines using Azure Data Factory, Fabric Data Pipelines, and other integration tools. Develop data transformation logic using SQL, PySpark, or similar technologies to standardize and prepare data for analytics use. Design and manage lakehouse structures in Microsoft Fabric, including bronze, silver, and gold data layers. Collaborate with BI developers to ensure data is modeled and structured appropriately for Power BI semantic models and reporting. Monitor data pipelines and platform performance, troubleshooting failures and optimizing workloads for reliability and efficiency. Implement data validation checks and reconciliation processes to ensure data accuracy and completeness. Manage data storage strategies, including partitioning, indexing, and lifecycle management for performance and cost optimization. Ensure secure data handling practices, including role‑based access controls, encryption, and compliance with organizational and regulatory requirements (e.g., PCI considerations). Maintain documentation of data pipelines, data models, and system architecture to support transparency, governance, and knowledge sharing. Partner with business stakeholders and IT teams to understand data requirements and deliver scalable data solutions. Support data governance initiatives, including metadata management, lineage tracking, and data classification (e.g., integration with Purview). Prepare and expose curated, governed, and well‑documented data assets for AI, generative AI, semantic search, retrieval‑augmented generation, and agent‑based enterprise workflows. Partner with business, IT, and analytics stakeholders to identify where data engineering can enable agentic workflows, automation, and AI‑assisted decision‑making while maintaining appropriate human oversight and control. Design, build, and support data integrations, APIs, secure data exchanges, and reporting components used in client- and vendor‑facing technology solutions. Establish monitoring, observability, alerting, and support practices for AI‑enabled and externally facing data solutions to ensure reliability, traceability, performance, and timely issue resolution. Contribute to continuous improvement of the data platform, recommending new tools, practices, and optimizations aligned with industry best practices. This job description is not designed to cover or contain a comprehensive listing of essential functions or responsibilities that are required of the employee for this job. Essential functions and responsibilities and activities may change at any time with or without notice. Education & Experience Required: Bachelor’s degree in Information Systems, Computer Science, Data Engineering, or a related field, or equivalent work experience. 5+ years of experience in data engineering, data integration, or data platform development. Strong experience with data pipeline development and ETL/ELT processes. Proficiency in SQL and data modeling concepts (warehouse/lakehouse). Experience working with cloud-based data platforms (e.g., Azure Data Factory, Azure SQL, Microsoft Fabric, or similar). Understanding of data governance, data quality, and security principles. Experience designing or supporting data assets, integrations, or platform services used by enterprise applications, automation, analytics, or AI‑enabled workflows. Preferred: Experience with Microsoft Fabric, Lakehouse architecture, and Delta Lake concepts. Experience with PySpark or distributed data processing frameworks. Familiarity with Power BI, semantic models, and analytics workflows. Experience with data cataloging and governance tools (e.g., Microsoft Purview). Experience working in regulated environments with compliance requirements (e.g., PCI, financial services). Familiarity with generative AI, retrieval‑augmented generation, semantic search, AI agent concepts, responsible AI practices, or AI‑ready data architecture. Experience supporting client- or vendor‑facing technology solutions, secure data exchanges, APIs, integrations, or externally consumed reporting platforms. DP‑600, DP‑700, AZ‑305 Travel Minimal travel expected; occasional travel may be required based on project needs. Work Environment This job operates in a clerical, office setting. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines. Physical Demands This is a largely sedentary role; however, some filing may be required. This would require the ability to lift files, open filing cabinets and bend or stand as necessary. Position Type & Expected Hours of Work This is a full‑time, exempt position. Days and hours of work are Monday through Friday 8:30 a.m. to 5 p.m. Some flexibility in hours is allowed based on management’s approval. Job Compensation & Benefits Through our competitive compensation and benefits program, we seek to attract, retain and motivate a high‑performing and increasingly diverse Associate population. Our compensation and benefit structure is comprised of many factors, including but not limited the size of our Firm, our financial position, the industry, business objectives, market salary information, benchmark pay levels and practices against the respective peer group (to ensure equal pay for equal work, with allowable pay differences based on factors not prohibited by law), and unique circumstances of the Firm that are subject to change. Compensation Minimum: $116,000.00 - Maximum: $135,000.00 USD Benefits Heavily subsidized medical insurance options Generous employer HSA contributions 100% employer‑paid Short‑Term Disability, Long‑Term Disability, Accident, Basic Life Insurance, and Telehealth/Virtual Urgent Care/Whole Mental Health Dental and Vision plans Unlimited PTO Organization‑paid holidays, including an extended Winter Break Safe Harbor 401k Plan Employer‑paid professional dues, licenses, and renewals Continuing Education resources Family Planning Assistance Program and Paid Parental Leave Subsidized fitness programs Equal Opportunity Employer Statement Cain Watters & Associates is committed to upholding the principles of the Equal Pay and Opportunity Act (EPOA). If you have any concerns regarding compliance with this policy, we encourage you to report them immediately by contacting HR at View email address on click.appcast.io. Your concerns will be handled with the utmost confidentiality and care. #J-18808-Ljbffr