Senior Software Engineer - PA086

Responsibilities

• Design, build, and operate large-scale data acquisition pipelines for ingesting, validating, transforming, enriching, deduplicating, and storing high-volume raw data.
• Architect ETL/ELT workflows supporting batch, streaming, scheduled, and event-driven processing.
• Develop production Java services and data processing applications for ingestion, orchestration, enrichment, deduplication, and delivery.
• Build and improve systems using Apache Airflow, Apache Beam, Spark, Google Dataflow, DataProc, Kafka, and Pub/Sub.
• Define approaches for schema evolution, data contracts, validation, backfills, replayability, and idempotent processing.
• Improve pipeline reliability, performance, scalability, resource utilization, and cost efficiency.
• Implement monitoring, alerting, logging, and observability for pipeline health, throughput, latency, failures, and data quality.
• Collaborate with product, data science, platform, and data quality teams to translate requirements into production systems.
• Contribute to technical designs, implementation plans, and modernization efforts across Data Acquisition.

Requirements

• 5+ years of professional software engineering experience focused on backend systems, data engineering, or distributed processing.
• Proven experience building and operating production data pipelines at scale.
• Deep proficiency with Java and object-oriented design.
• Hands-on experience with Apache Beam, Apache Airflow, Spark, Google Dataflow, or DataProc.
• Strong experience with streaming systems such as Apache Kafka or Google Pub/Sub.
• Understanding of batch and streaming processing, data modeling, schema evolution, and data quality management.
• Experience designing ETL/ELT workflows for large volumes of structured and semi-structured data.
• Experience designing high-throughput, fault-tolerant backend services and distributed systems.
• Understanding of APIs, integration patterns, retries, backpressure, idempotency, and operational failure modes.
• Experience with large-scale storage and query technologies such as BigQuery, Snowflake, or Trino.
• Experience with at least one cloud provider, preferably GCP.
• Hands-on experience with BigQuery, GCS, GKE, Dataflow, DataProc, and Pub/Sub.
• Experience operating production services with monitoring, logging, alerting, SLIs, and incident investigation.
• Bachelor’s degree in Computer Science, Software Engineering, or a related field.
• Ability to translate business and data requirements into technical designs and implementation plans.
• Experience with Kubernetes, especially GKE or EKS, is preferred.
• Experience with Terraform or other infrastructure-as-code tools is preferred.
• Experience with Snowflake, BigQuery, Starburst/Trino, or similar query engines is preferred.
• Knowledge of CRM systems, email/calendar APIs, third-party feeds, change data capture, or external data providers is preferred.
• Experience in a B2B data company, data marketplace, or data-as-a-product environment is preferred.
• Expert-level experience with Apache Spark or another distributed processing framework is preferred.

Benefits

• Hybrid work arrangement is indicated by the #LI-Hybrid designation.
• Comprehensive employee benefits are offered.
• Holistic mind, body, and lifestyle programs support employee well-being.
• Additional compensation such as bonus, commission, and equity may apply, separate from base salary.