Data & AI Engineer - Microsoft Fabric, Experimentation Data & Agent Development

Overview:

Job Title: Data & AI Engineer - Microsoft Fabric, Experimentation Data & Agent Development

Location: Bellevue, WA/Remote

Experience: 6-8 Years

Job Summary

We are seeking a highly skilled Data & AI Engineer with strong expertise in Microsoft Fabric, Azure AI Services, data engineering, and AI-powered analytics solutions. The ideal candidate will design and manage scalable experimentation data platforms, build curated Silver/Gold data layers, develop AI-powered data agents, and support enterprise analytics and experimentation workflows.

This role requires hands-on expertise in Microsoft Fabric, PySpark, clickstream analytics, Adobe Customer Journey Analytics (CJA), Azure AI Services, and modern data engineering best practices.

Required Skills

• Strong expertise in:
  • Microsoft Fabric
  • Data Engineering
  • Data Modeling
  • Big Data Platforms
  • Data Pipeline Design
• Hands-on experience with:
  • PySpark
  • Azure AI Services
  • AI/ML Integration
  • Microsoft Fabric Data Agents
  • Azure Data Factory (ADF)
• Strong understanding of:
  • Clickstream data
  • Customer Analytics
  • Adobe Analytics
  • Adobe Customer Journey Analytics (CJA)
  • Experimentation data platforms
• Experience with:
  • Data validation and reconciliation
  • Performance tuning and monitoring
  • Telemetry pipelines
  • Data governance and semantic modeling
• Strong analytical and problem-solving skills
• Ecommerce domain experience preferred

Key Responsibilities

Experimentation Data Enablement

• Design, build, and maintain curated Silver-layer datasets in Microsoft Fabric for experimentation analytics and reporting
• Develop reusable data products including standardized tables and views for dashboards, scorecards, and ad hoc analysis
• Ensure datasets are analysis-ready, cleaned, deduplicated, and semantically aligned

Data Gap Analysis & Modeling

• Conduct gap assessments across experimentation KPIs, telemetry systems, and Silver-layer datasets
• Identify data quality, latency, schema, and join-key issues and recommend remediation strategies
• Improve data models using:
  • Facts and dimensions
  • Conformance rules
  • Surrogate keys
  • Grain optimization

Gold Layer & Stakeholder Management

• Conduct workshops with BI, reporting, experimentation, and engineering teams
• Define KPI calculations, attribution logic, governance standards, and refresh SLAs
• Create source-to-target mappings, data dictionaries, validation rules, and technical specifications

Pipeline Engineering

• Build and maintain robust data pipelines using:
  • Microsoft Fabric Pipelines
  • Azure Data Factory (ADF)
  • Telemetry/1DS pipelines
• Implement orchestration, incremental loads, monitoring, and error handling mechanisms

Data Validation & Reconciliation

• Validate and reconcile datasets with Adobe Customer Journey Analytics (CJA)
• Build automated validation routines for:
  • Missing data
  • Duplicate detection
  • Schema drift
  • Metric anomaly detection
• Coordinate issue resolution with telemetry and engineering teams

AI Agent Development

• Design and build AI-powered data agents using:
  • Fabric Data Agents
  • Copilot
  • Azure OpenAI
• Enable use cases such as:
  • Automated scorecards
  • Narrative summaries
  • Self-service analytics Q&A
  • Metric definition assistants
  • Data quality monitoring assistants
• Define AI agent architecture, RBAC security, grounding sources, and evaluation metrics

Governance & Operational Excellence

• Maintain documentation for:
  • Data transformations
  • Pipeline dependencies
  • Metric definitions
  • Validation rules
• Implement best practices for:
  • Naming conventions
  • Semantic consistency
  • Versioning
  • Cost optimization
• Support operational monitoring, troubleshooting, and continuous improvement initiatives

Preferred Qualifications

• Experience supporting experimentation and customer analytics platforms
• Strong knowledge of ecommerce and digital analytics ecosystems
• Exposure to AI-driven analytics and self-service data platforms
• Strong collaboration and stakeholder communication skills