Data Engineer

The Data Engineer builds and maintains the data infrastructure that supports operational and digital analytics across the organization. This position designs, develops, and operates scalable data pipelines and ingestion workflows on a Microsoft Azure-based platform to ensure data is reliable, governed, and available for analytics and reporting. This role partners closely with IT, Finance, Operations, and Digital stakeholders to support the organization’s continued migration to a centralized, cloud-native data architecture. Work Location and Onsite Requirement: This position is Hybrid and requires a few days in the Alexandria VA office. Essential Duties and Responsibilities Design, develop, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of structured and semi-structured data. Ingest data from diverse sources, including relational databases, REST APIs, flat files, and third-party platforms. Build and manage Azure Data Factory pipelines to orchestrate, schedule, and monitor data workflows. Develop Azure Functions and Logic Apps to support event-driven ingestion and lightweight system integrations. Administer and configure Databricks workspaces, including cluster management, access controls, and workspace governance. Develop data processing and transformation logic using Python and PySpark in Databricks notebooks and jobs. Apply data architecture standards and evaluate technical trade-offs related to medallion architecture, data modeling, and governance best practices. Implement and maintain data governance controls, including access controls, data classification, and lineage tracking. Manage Azure Storage resources, including Azure Data Lake Storage Gen2, to support proper organization, partitioning, and lifecycle management. Validate data quality, completeness, and reliability throughout the pipeline and support discrepancy resolution with analysts, developers, and downstream consumers. Collaborate with analysts and developers to deliver curated, conformed datasets for reporting and analytics. Troubleshoot pipeline failures, performance bottlenecks, and data discrepancies. Document architecture decisions, data flows, pipeline logic, and operational runbooks. Qualifications Guidelines To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skills, and/or abilities required. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions. Education/Experience Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field required 3+ years of experience in data engineering roles required Hands-on experience with Microsoft Azure data services required Experience supporting Finance, Operations, or Digital reporting environments preferred Experience in multi-unit retail or restaurant environments preferred Language Ability Excellent verbal and written communication skills are required. This position requires the ability to translate business requirements into technical solutions, collaborate effectively with IT, Finance, Operations, and Digital stakeholders, and document technical work clearly for both technical and non-technical audiences. Math Ability This position requires strong analytical skills, including the ability to interpret data, evaluate metrics, identify trends, and assess risks across data pipelines and reporting environments. Reasoning Ability This position requires strong analytical and problem-solving skills, including the ability to assess complex data issues, evaluate technical trade-offs, and make sound decisions related to data architecture, pipeline design, data quality, and system integrations. The candidate must be able to troubleshoot discrepancies, identify root causes, prioritize risks, and develop reliable solutions in collaboration with technical and business stakeholders. Computer Skills Proficiency in Microsoft Office Suite is required. Strong proficiency in Python, PySpark, advanced SQL, Microsoft Azure data services, Databricks, Azure Data Factory, Azure Functions, Logic Apps, and Azure Data Lake Storage Gen2 is required. Experience with Databricks workspace administration, cluster configuration, Unity Catalog, access management, SQL sources, REST APIs, CSV/XML formats, data quality frameworks, governance controls, validation strategies, and cloud-based access controls is preferred. Supervisory Responsibilities This position has no supervisory responsibilities. Physical Demands This position is primarily office-based and requires the ability to remain in a stationary position for prolonged periods while using a computer, telephone, and standard office equipment. The position may occasionally require standing, walking, reaching, grasping, handling objects, and lifting or moving up to 20 pounds. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions. Special Requirements/Certification Power BI experience, including data modeling, DAX measures, and report publishing in Power BI Service, is preferred. Databricks and Microsoft Azure certifications, such as Databricks Certified Data Engineer Associate/Professional, Databricks Certified Associate Developer for Apache Spark, Azure Data Engineer Associate (DP-203), or Azure Fundamentals (AZ-900), are preferred but not required. #J-18808-Ljbffr Socket.dev