Senior Artificial Intelligence Data Engineer

Summary In this role, you will help build and enhance a modern, AI-ready data enablement platform that supports cross-domain analytics, governed data products, and reusable engineering patterns across the enterprise. You will enable data producers and analytics teams through guided pathways, reusable accelerators, metadata-driven frameworks, CI/CD patterns, Databricks Asset Bundles, dbt transformation standards, Unity Catalog governance, and Starburst/Trino analytical access. You will focus on helping teams create trusted, governed, reusable, and AI-ready derivative data products that support analytics, reporting, GenAI, semantic search, and emerging agentic platform use cases. As a hands‑on senior individual contributor, you combine platform engineering, data transformation, governance, and enablement to drive scalable and repeatable data product delivery across the organization. Responsibilities Build and support scalable data engineering solutions using Azure Databricks, PySpark, SQL, Delta Lake, Azure Data Factory, dbt, and Unity Catalog. Improve metadata‑driven Azure Data Factory and Databricks patterns for orchestration, configuration, monitoring, restartability, and operational support. Develop reusable accelerators including CI/CD templates, Databricks Asset Bundle patterns, deployment automation, environment configuration, and data product onboarding templates. Design, develop, and support dbt models, macros, tests, documentation, and transformation standards for governed analytical data products. Provide guidance on appropriate technology selection and implementation patterns across dbt, Databricks notebooks and workflows, Delta Live Tables, Spark, and Starburst/Trino. Support cross‑domain analytics initiatives by transforming source‑refined data into trusted, reusable, business‑aligned derivative data products. Leverage Unity Catalog to establish and support governed catalogs, schemas, tables, lineage, access controls, naming standards, and certification practices. Support Starburst/Trino as an analytical and federated query layer for governed enterprise data consumption. Apply Azure DevOps, Git, CI/CD, and Infrastructure as Code (IaC) practices to create repeatable, testable, and environment‑aware platform delivery processes. Troubleshoot and resolve production issues related to orchestration, transformations, data quality, access management, query performance, deployments, and operational workflows. Collaborate with data engineering, analytics, platform, governance, and business teams to establish reusable, scalable, and supportable data engineering patterns. Contribute to the evolution of enterprise data engineering standards, governance practices, observability capabilities, and AI‑ready data product frameworks. Qualifications Relevant degree preferred. 5 or more years of hands‑on data engineering experience building production‑grade data platforms, pipelines, or analytical data products required. Strong experience with Azure Databricks, PySpark, Spark SQL, Delta Lake, Azure Data Factory, SQL, and dbt required. Experience with Azure DevOps, Git, pull request workflows, CI/CD pipelines, and release management practices required. Working knowledge of lakehouse architecture, metadata management, data governance, lineage, access control, and operational support required. Demonstrated ability to function as a senior individual contributor with strong ownership, technical judgment, and cross‑functional collaboration skills required. Experience supporting enterprise‑scale analytical platforms and governed data product delivery preferred. Experience with Unity Catalog, Starburst/Trino, Pulumi or other Infrastructure as Code tools, Databricks Asset Bundles, Apache Iceberg concepts, and AKS/Kubernetes‑based platform operations preferred. Experience building reusable frameworks, accelerators, templates, or platform capabilities for engineering teams preferred. Experience preparing governed structured data for AI/ML, GenAI, Retrieval‑Augmented Generation (RAG), semantic search, copilots, or agentic workflows preferred. Experience within healthcare, analytics, supply chain, finance, or other regulated enterprise environments preferred. Strong problem‑solving, communication, and collaboration skills with the ability to influence technical direction and establish best practices preferred. You must be authorized to work in the United States without sponsorship. Compensation & Benefits Estimated Hiring Range: $102,400.00 to $179,000.00. This position is also incentive eligible. Vizient has a comprehensive benefits plan. Please view our benefits here: #J-18808-Ljbffr