Senior Data Architect

We are seeking a visionary Azure Databricks & Agentic AI Architect to design and implement next-generation AI-powered data platforms. This role combines deep expertise in Azure Databricks, Lakehouse Architecture, Data Engineering, and Generative AI to build intelligent, autonomous, and self-optimizing data ecosystems.

The ideal candidate will lead the adoption of Agentic AI within Data Engineering and AI-DLC, enabling autonomous data ingestion, transformation, quality management, lineage discovery, observability, optimization, testing, and governance.

Key Responsibilities

Agentic Data Engineering Leadership

• Design and implement AI-powered Data Engineering platforms leveraging Azure Databricks and Lakehouse architecture.
• Define autonomous workflows using AI Agents for:
• Data quality remediation
• Root cause analysis
• Establish frameworks for Human-in-the-Loop (HITL) decision-making and governance.

AI-Driven Data Lifecycle (AI-DLC)

• Lead architecture for AI-enabled Data Development Lifecycle across:
• Requirement analysis
• Automated testing
• Implement AI copilots to accelerate developer productivity.
• Enable automated lineage creation and intelligent impact analysis.

Lakehouse & Data Platform Architecture

• Design scalable Lakehouse platforms using:
• Unity Catalog
• Architect RAG-based solutions using enterprise data assets.
• Design agent orchestration frameworks using:
• Semantic Kernel
• MCP-enabled architectures
• Build domain-specific AI agents supporting Data Engineering and Analytics teams.

AI Governance & Responsible AI

• Define guardrails for enterprise GenAI adoption.
• Prompt governance
• Observability
• Auditability
• Explainability
• Security controls
• Establish governance models for autonomous AI agents.

AI-Powered Platform Optimization

• Incident triage
• Capacity planning
• Enable intelligent workload placement and model routing.

Technical Skills

• Unity Catalog

AI & Agentic Frameworks

• RAG Architecture
• DevOps & AI-DLC
• GitHub Actions
• Drive AI-First Data Engineering transformation.
• Define enterprise patterns, accelerators, and reusable AI agents.
• Mentor architects, data engineers, and AI engineers.
• Lead executive conversations on AI adoption, ROI, and transformation roadmaps.

Preferred Certifications

• Databricks Certified Data Engineer Professional
• Azure Data Engineer (DP-203)
• Generative AI / Agentic AI Certifications