Technical Lead/Architect (Temporal & Apache Kafka required)

Job Description

Job Description

About Infinitive
Infinitive is a data and AI consultancy that helps clients modernize, monetize, and operationalize their data to generate lasting value. They pride themselves on their deep industry and technology expertise, ensuring that they drive and sustain the adoption of new capabilities. Infinitive is committed to aligning their team with their clients' culture, ensuring a successful partnership by bringing the right mix of talent and skills for high return on investment.

Infinitive has earned recognition as one of the "Best Small Firms to Work For" by Consulting Magazine, receiving this accolade nine times, most recently in 2026. They have also been honored as a “Top Workplace” by the Washington Post, “Best Places to Work” by the Washington Business Journal, and “Best Places to Work” by Virginia Business.

About the Role

We are seeking a Staff / Principal Data Engineer & Platform Architect to define the technical vision, architectural strategy, and technical execution for our enterprise-wide, next-generation event-driven data platforms. In this role, you will act as the principal authority bridging ultra-high-throughput streaming topologies, stateful distributed workflow orchestration, and federated data governance.

You will spearhead our platform architecture centered around Apache Kafka for real-time event distribution and Temporal for durable execution and distributed state orchestration. A core mandate of this position is establishing enterprise data contracts, driving schema evolution frameworks, and designing resilient Lakehouse/warehouse architectures capable of serving high-concurrency analytical and operational workloads. As a technical leader, you will set the engineering standards, mentor senior talent, and align multi-team distributed systems initiatives with executive business priorities.

Key Responsibilities

• Architecture & Strategy: Define the multi-year technical roadmap and reference architecture for our real-time streaming infrastructure, Lakehouse ingestion engines, and distributed orchestration layers.

• Distributed Stream Topologies: Architect and govern multi-tenant, enterprise Apache Kafka ecosystems (producers, consumers, Kafka Connect, Schema Registry), ensuring sub-second latencies, exactly-once processing guarantees, and disaster recovery.

• Data Contracts & Governance: Formulate and enforce company-wide data governance, schema design patterns, and automated contract testing (Avro, Protobuf, JSON Schema) across polyglot microservices, streaming pipelines, and storage boundaries.

• Durable Orchestration & Transactional Integrity: Architect long-running, fault-tolerant workflow topologies using Temporal, standardizing complex distributed transaction patterns (e.g., Saga patterns, compensation logic, distributed state machines, human-in-the-loop flows).

• Lakehouse & Analytical Engine Architecture: Direct the design and operational scaling of large-scale distributed computing workloads (Apache Spark / PySpark) and modern cloud Lakehouse platforms (Snowflake, Databricks, BigQuery) using medallion architectures and dimensional/semantic models.

• Operational Excellence & FinOps: Implement comprehensive observability platforms (tracing, telemetry, automated drift detection, data quality gates) while establishing cloud infrastructure cost optimization and capacity planning frameworks.

• Technical Leadership & Mentorship: Act as the technical authority across platform and product teams; establish engineering excellence standards, drive RFC and Architecture Review Board (ARB) processes, and elevate senior engineering talent across the organization.

Required Experience

• 8+ years of professional experience in data engineering, distributed systems, or backend platform architecture, with 2+ years acting in a Lead, Staff, or Principal/Architect capacity .

• Durable Workflow & State Machine Architecture: Deep architectural experience with Temporal (or Cadence), including deterministic execution modeling, activity orchestration, workflow versioning strategies, signal/query handling, dynamic task routing, and cross-boundary error recovery.

• Mastery of Streaming Systems: Deep foundational expertise with Apache Kafka internals (broker sizing, partition strategies, consensus mechanisms, custom connector development, consumer group rebalancing, and stateful stream processing via Kafka Streams or Flink).

• Enterprise Data Contracts & Schema Engineering:

  • Authority in schema specification frameworks ( Apache Avro, Protocol Buffers/gRPC, JSON Schema ).

  • Hands-on leadership implementing schema registries (Confluent, AWS Glue) with automated CI/CD schema linting, breaking-change prevention, and backward/forward compatibility strategies.

  • Experience standardizing automated data contract verification tooling (e.g., Great Expectations, Pandera, Pydantic, dbt tests, contract-testing frameworks).

• Distributed Computing & Core Engineering: Advanced engineering capabilities in Python, Go, or Java/Scala , emphasizing distributed design patterns, concurrency models, and resilient microservice design. Extensive scale experience with Apache Spark / PySpark internals and query plan optimization.

• Modern Lakehouse & Data Modeling: Expert-level knowledge of dimensional data modeling, Data Vault, or semantic layer design, with practical experience scaling multi-terabyte/petabyte systems on Snowflake, Databricks, or BigQuery.

• Cross-Functional Influence: Demonstrated track record of aligning distributed engineering teams around unified architectural patterns, authoring architectural decision records (ADRs), and influencing executive stakeholders.

Infinitive is required by law in some jurisdictions to include a reasonable estimate of the compensation range for this role. The determination of this range includes various factors not limited to skill set, level, experience, relevant training, and licensure and certifications. Compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range for this role in the U.S. is $90,000 - $154,00.00.

Infinitive is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.

Powered by JazzHR

29b7NGjmI2