Data Engineer (Azure I Databricks I Snowflake)

Overview:

TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients world-wide. Our client provider of digital technology and transformation, information technology and services

Position: Data Engineer (Azure | Databricks | Snowflake)

Location: Overland Park, KS, Frisco, TX

Duration: 6 Months

Job Type: Temporary Assignment

Work Type: Onsite

Job Description:

• Data Engineer for Azure-native third-party data enrichment platform using Databricks/Spark + Snowflake; focus on reliable governed pipelines, strong Spark troubleshooting, privacy/governance, and cost-aware engineering.
• You will join a data engineering team responsible for third party data enrichment augmenting first party datasets with external identity/attribute data to support analytics, activation, and research.
• The enriched datasets are consumed by multiple downstream systems and teams, including the CDP and other analytics/research stakeholders.
• The platform is Azure-native and built primarily on Databricks (processing + some ML workloads) and Snowflake (analytics/warehouse).
• A major focus is building reliable, governed, vendor agnostic datasets while ensuring privacy/compliance, data governance, and cost efficiency.

Key Responsibilities:

• As a Data Engineer, you will Data Ingestion & Pipeline Development Build and enhance ingestion pipelines for large batch and event-driven paths (streaming may evolve over time).
• Integrate data from: Third party enrichment vendors (identity + attributes, very large volumes) Digital platforms via Conversion API (CAPI) integrations (through intermediary/middleware) Rewards/Promotions systems for offer issuance/redemption/consumption data.
• Data Quality, Reliability & Operations Implement strong data validation, idempotency, replay/backfill strategies, and deduplication to prevent quality drift.
• Own monitoring, alerting, dashboarding, and operational readiness (wrappers around core pipelines).
• Troubleshoot failures with root cause analysis not just reruns: Interpret Spark logs Diagnose performance issues (shuffle, skew, partitioning) Improve stability and SLA adherence Governance & Compliance (First-class NFR)
• Apply privacy, compliance, and governance requirements across pipelines and datasets.
• Support governance standards such as: Unity Catalog, lineage, access controls Managing PII vs non PII access Documentation of tables, schemas, catalogs, and cluster usage Cost Governance & Performance Optimization Design pipelines with cost awareness from day one: Cluster sizing, workload tuning, efficient compute/storage usage Trade-off decisions balancing cost vs quality vs SLA

Must-Have Skills:

• Strong coding: PySpark + SQL (hands-on, not only orchestration) Databricks: notebooks/jobs, performance tuning fundamentals, medallion patterns Spark fundamentals: partitioning, skew/shuffle optimization, understanding failures via logs Snowflake: data modeling/usage for analytics/warehousing workloads
• Azure ecosystem: Azure Data Factory (ADF) (orchestration) Azure-native integrations and services exposure Data engineering reliability patterns: validation, idempotency, replay/backfills, dedup, auditability
• Data governance: Unity Catalog (preferred), lineage, access control patterns, PII handling Ownership mindset: can execute independently without constant approvals/check-ins

Nice-to-Have Skills

• Event-driven/streaming ingestion exposure (even if primary is batch today) Delta/Databricks patterns such as Delta Live Tables (DLT) (some workflows exist) Experience building configurations.

TekWissen Group is an equal opportunity employer supporting workforce diversity.