SENIOR DATA ENGINEER

Data Pipeline Engineer

We are seeking a Data Pipeline Engineer to design, build, and maintain scalable data pipelines in support of mission-critical environments. This role focuses on working within the Palantir ecosystem to integrate, transform, and operationalize complex datasets while ensuring reliability, performance, and data integrity. This is a highly technical, hands-on position ideal for engineers with experience in distributed data processing, backend systems, and troubleshooting complex data workflows.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines supporting structured and unstructured data
• Build and optimize ETL/ELT workflows within Palantir Foundry or similar data platforms
• Develop data processing solutions using Python, PySpark, Spark, and/or Polars
• Debug and troubleshoot pipeline failures, data inconsistencies, and performance bottlenecks
• Build and integrate APIs and backend services to support data ingestion and transformation
• Analyze and trace network traffic and data flows to support system-level troubleshooting
• Reverse engineer backend processes and data services to resolve complex technical issues
• Implement monitoring, logging, and data pipeline reliability improvements
• Collaborate with cross-functional teams to enhance data workflows and platform usability
• Ensure data quality, governance, and consistency across data systems

Required Qualifications:

• Active TS/SCI clearance
• Strong proficiency in Python development
• Hands-on experience with PySpark, Apache Spark, and/or Polars for distributed data processing
• Experience building and maintaining data pipelines and ETL/ELT workflows
• Experience working with Palantir Foundry or similar data platforms
• Experience developing and debugging APIs, endpoints, and backend services
• Ability to trace and analyze network traffic and system interactions
• Experience troubleshooting complex, production-level data issues
• Strong understanding of data structures, data modeling, and pipeline architecture
• Experience with Git/version control and CI/CD practices

Preferred Qualifications:

• Experience working in high-reliability or production-critical environments
• Knowledge of distributed systems and large-scale data processing frameworks
• Experience with streaming pipelines (Kafka, Spark Streaming, etc.)
• Familiarity with cloud platforms (AWS or similar)
• Experience with data visualization or analytics tools
• Background in software engineering or backend systems development

Education:

• Bachelor's degree in Computer Science, Engineering, or related field
• Equivalent hands-on experience may be considered

Why This Role:

Work directly with cutting-edge data platforms and large-scale pipelines

Solve complex, real-world data engineering challenges

Highly technical environment with strong engineering ownership

Stable, long-term onsite work in the DC market