Data Engineer

Role: Data Engineer (Azure | SQL | Spark | Snowflake | Databricks)

Frisco - TX

Data Engineer needed for Azure-native third party data enrichment platform using Databricks/Spark + Snowflake; focus on reliable governed pipelines, strong Spark troubleshooting, privacy/governance, and cost-aware engineering.
You will join a data engineering team responsible for third party data enrichment augmenting first party datasets with external identity/attribute data to support analytics, activation, and research.
The enriched datasets are consumed by multiple downstream systems and teams, including the Customer Data Platform (CDP) and other analytics/research stakeholders. The platform is Azure-native and built primarily on Databricks (processing + some ML workloads) and Snowflake (analytics/warehouse).
A major focus is building reliable, governed, vendor agnostic datasets while ensuring privacy/compliance, data governance, and cost efficiency.
Key Responsibilities
As a Data Engineer, you will: Data Ingestion & Pipeline Development Build and enhance ingestion pipelines for large batch and event-driven paths (streaming may evolve over time). Integrate data from: Third party enrichment vendors (identity + attributes, very large volumes) Digital platforms via Conversion API (CAPI) integrations (through intermediary/middleware) Rewards/Promotions systems (e.g., TMT) for offer issuance/redemption/consumption data Data Quality, Reliability & Operations Implement strong data validation, idempotency, replay/backfill strategies, and deduplication to prevent quality drift. Own monitoring, alerting, dashboarding, and operational readiness (wrappers around core pipelines).
Troubleshoot failures with root cause analysis not just reruns: Interpret Spark logs Diagnose performance issues (shuffle, skew, partitioning) Improve stability and SLA adherence Governance & Compliance (First-class NFR) Apply privacy, compliance, and governance requirements across pipelines and datasets.