Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Python backend engineer with PySpark/Spark

$57 per hour

PeopleNTech LLC

Job Description

Job Description

Indent : PSL307991-15-1

Role : Python backend engineer with PySpark/Spark

Location : Charlotte, NC (Hybrid)

Rate : $57/hr

Role Summary

- Senior backend engineer building the core integrations between Unity and the enterprise Spark engine and SparkFlow framework

- Designs distributed, high-throughput systems that power data processing and integration workflows at enterprise scale

- Operates with significant autonomy, setting technical direction within a regulated, large-scale engineering environment

Key Responsibilities

Platform & Integration Engineering

- Design, build, and operate integrations between Unity and the Spark engine and SparkFlow framework

- Develop robust APIs, SDKs, and connectors enabling reliable data movement and workflow orchestration

- Implement abstractions that simplify SparkFlow adoption for downstream platform consumers

Distributed Systems Design

- Architect scalable, fault-tolerant systems handling large data volumes and concurrent workloads

- Apply best practices for performance tuning, caching, partitioning, backpressure, and resilience

- Drive design reviews, RFCs, and architectural decisions for core platform components

Reliability & Operations

- Build systems with first-class observability (metrics, logs, traces) and SLOs

- Establish operational excellence: runbooks, alerting, capacity planning, and incident response

- Champion automated testing, and progressive delivery practices

Required Qualifications

- 6 years of backend engineering experience, with deep expertise in distributed systems

- Expert-level Python; strong API design (REST, gRPC) and asynchronous programming

- Proven track record delivering large-scale, production-grade systems

- Strong fundamentals in concurrency, data structures, and systems performance experience

Preferred Qualifications

- Hands-on experience with Apache Spark (PySpark, Spark SQL, Structured Streaming)

- Experience building or integrating with workflow/data orchestration frameworks (Airflow, Autosys)

- Exposure to greenfield platform builds in regulated financial enterprises

- Experience with event-driven architectures (Kafka) and lakehouse formats (Iceberg, Delta, Hudi)

Technical Skills

- Languages: Python (expert), Scala or Java (plus), SQL

- Distributed Compute: Apache Spark

- Storage & Data: S3/ADLS/GCS, Iceberg, Delta Lake, Parquet, Postgres

- Streaming & Messaging: Kafka, Pub/Sub

- APIs & Services: REST, gRPC, GraphQL, OpenAPI

- Containers & Cloud: Docker, Kubernetes, Azure/GCP, Terraform

- CI/CD: GitHub Actions

Soft Skills & Leadership Expectations

- Strong technical judgment and ability to operate
Vacancy posted 22 days ago
Similar jobs that could be interesting for youBased on the Python backend engineer with PySpark/Spark in Santa Clara, CA vacancy
  • $75k - $300k

     ...headquartered in San Jose, California, with offices in Barcelona,...  .... We’re seeking motivated backend engineers to join our infra team. In...  ...Qualifications Proficiency in Python programming, data structures,...  ...processing engines such as Spark or Flink. Experience with... 
    Suggested

    Tensor Auto

    San Jose, CA
    4 days ago
  •  ...Python Backend EngineerWe are looking for a Python Backend Engineer to build the backbone of our platform. You will be responsible for creating high-performance APIs...  ...Python (3.10+ preferred).Frameworks: Deep experience with FastAPI, DjangoAI Tooling: Familiarity with... 
    Suggested

    Yochana

    San Jose, CA
    2 days ago
  •  ...Sr Backend EngineerSunnyvale, California, United StatesAbout The Job Sr Backend EngineerJob...  ...+ years of strong experience interfacing with web services platformsProficient in...  ...pipelines in big data technologies (i.e. Spark/Kafka/Cassandra/Hadoop/Hive/Elasticsearch... 
    Suggested

    Interaction24 LLC

    Sunnyvale, CA
    2 days ago
  • $224k - $356.5k

     ...is looking for a Machine Learning Engineer to join the GPU accelerated Apache Spark team.Apache Spark is the most popular...  ...Spark applications. You will work with the open source community to...  ...models.Excellent programming skills in Python and Python data science related... 
    Suggested
    Full time

    Nvidia

    Santa Clara, CA
    2 days ago
  • $164k - $295k

     ...breaches, and we’ve redefined modern security with the world’s most advanced AI-native...  ...seeking an experienced Software Development Engineer to join our AI Detection and Response (...  ...one cloud native programming language: GO, Python, Java Deep understanding of:... 
    Suggested
    Permanent employment
    Work experience placement
    Work at office
    Local area
    Worldwide
    2 days per week
    3 days per week
    Sunnyvale, CA
    a month ago
  • $184.7k - $324.8k

     ...across product, design, engineering, marketing, editorial,...  ...build, and operate the backend services that form the...  ...servicesExpert proficiency in Python or Node.js/TypeScript —...  ...distributed systems with strict reliability and...  ...— Snowflake, Trino, Spark, PostgreSQL, or equivalentProficiency... 
    Relocation

    Apple

    Cupertino, CA
    4 days ago
  •  ...DescriptionPrimary Skills:Big Data experience8+ years exp in Java, Python and ScalaWith Spark and Machine Learning (3+)Data mining, Data...  ...scraping, calling APIs, write SQL queries, etc.).Work closely with our engineering team to integrate and build algorithmsProcess... 

    Comtech

    San Jose, CA
    3 days ago
  •  ...integrations layer that keeps it in sync with Okta, AWS, and the rest of the...  ...— and prove it laterA policy engine is only as good as the world...  ...Have built and operated backend systems that real customers depended on Strong in Go or Python or Java Comfortable... 

    MetAntz

    San Jose, CA
    4 days ago
  •  ...Senior Backend Engineer (Infrastructure & AI Platform)FlexAI is looking for a Senior Backend Engineer (Infrastructure & AI Platform) with deep Golang expertise to architect and build the core backend...  ...):Languages: Golang (Primary), Python (Secondary)Infrastructure: Kubernetes... 
    Work at office

    FlexAI

    San Jose, CA
    2 days ago
  •  ...Senior Backend EngineerLocation: Sunnyvale, CAResponsibilities:5+ years of professional software engineering experienceExpertise with standard server-side software technologies, including: Java, MySQL, spring, Spring Boot, Hibernate, etc.Experience with REST and API design... 

    Ubertal Inc

    Sunnyvale, CA
    5 days ago
  •  ...#WeArePGD Overview As a Senior Software Engineer – AI & Platform Systems, you will design,...  ...project level while collaborating closely with architects, product leaders, and cross-capability...  ...Strong experience with Java, .NET/C#, or Python including: o Web services (RESTful APIs)... 

    Publicis Media

    San Jose, CA
    2 days ago
  • $150k - $172k

     ...Senior Backend Software Development EngineerHackerRank helps companies...  ...companies to build strong engineering teams ready for what's next....  ...customers can be wildly successful with products they genuinely love...  ...programming language (e.g., Python, Ruby, Go, Java, or Node.js)... 
    Immediate start
    Shift work

    HackerRank

    Santa Clara, CA
    2 days ago
  •  ...com is building a 100× better job search engine — fast, comprehensive, honest, and...  ...responsibility from day one. Technically strong with solid fundamentals in programming and...  ...correctness Experience with Node.js and Python Preferred: an interest in security and... 
    Full time
    Flexible hours

    hiring-cafe

    Cupertino, CA
    5 days ago
  • $150k - $250k

     ...up. Led by industry veterans with a proven track record of success...  ..., secret management, and engines that enable these capabilities...  ...~5+ years of experience as a backend or full-stack engineer ~ Strong...  ...backend programming languages (Go, Python, Node.js, Rust, etc.) ~ Deep... 
    Full time

    Cylake

    Sunnyvale, CA
    19 days ago
  • $140k - $210k

     ...Sr Backend Engineer at CloverCalling all innovators - find your future at Fiserv. We're Fiserv, a global leader in Fintech and payments, and...  ...seamless payment processing to inventory and staff management. With over 15 billion transactions processed annually, Clover empowers... 
    Worldwide

    BentoBox

    Sunnyvale, CA
    2 days ago
  • $150.4k - $277.6k

     ...Tools & AI team is a tight-knit engineering team building the internal...  ...well-designed systems. You'll own backend services and data infrastructure end to end, from Python REST APIs to the model-serving...  ...AI-native tooling, partnering with engineering, science, and quality... 
    Relocation
    Shift work

    Apple

    Cupertino, CA
    4 days ago
  • $152k - $241.5k

    NVIDIA's invention of the GPU 1999 sparked the growth of the PC gaming market, redefined...  ...AI — the next era of computing — with the GPU acting as the brain of computers...  ...”.We are searching for a Senior Backend Compiler Engineer with experience in LLVM code generation... 
    Full time
    Remote work
    Worldwide

    Nvidia

    Santa Clara, CA
    1 day ago
  • $45 per hour

     ...The TikTok LIVE Foundation Governance Engineering team is a global team responsible...  ...the responsibility of building the backend system, collaborating with data scientists, ecosystem governance...  ...including but not limited to Go, Java, Python, C++. Curiosity towards new... 
    Hourly pay
    Full time
    Summer work
    Internship
    Local area
    Worldwide

    TikTok

    San Jose, CA
    13 days ago
  • $150.4k - $277.6k

     ...design, implement, and productize advanced backend compiler optimizations for Apple Silicon...  ...shipping. You will partner directly with GPU architects to explore and evaluate instruction...  ...are seeking a skilled Compiler Backend Engineer to join our team. This role will involve... 
    Relocation

    Apple

    Cupertino, CA
    4 days ago
  • $174k - $255k

     ...and advanced AI. Born at X (the innovation engine behind Waymo, Verily, and Google Brain),...  ...at scale. Tapestry currently collaborates with key partners across the U.S., U.K., Chile,...  ...large-scale data platforms (e.g., BigQuery, Spark, Kafka, Ray) or complex distributed... 
    Full time
    Flexible hours

    X Company

    Mountain View, CA
    2 days ago
  • $184.7k - $324.8k

     ...08SummaryThe growing Apple Services group is seeking a Senior Backend Engineer to join the InfoApps org to help shape what comes next for one...  ...to design and build high-scale, distributed backend services with event-driven architectures and pipelines (e.g., Kafka)Solid experience... 
    Relocation

    Apple

    Cupertino, CA
    4 days ago
  • $196.5k - $291.5k

     ...to exchange funds more safely with merchants using a variety of...  ...accountability for the domain's backend platform features,...  ...experience12+ years of software engineering experience, with significant...  ...systems at scale.Deep expertise in Python and API management — including... 
    Full time
    Work at office
    Local area
    Immediate start
    Flexible hours

    PayPal

    San Jose, CA
    2 days ago
  • $180k - $240k

     ...action. Built on the open-source Conductor engine, our platform empowers developers to...  ...value. The Opportunity As a Senior Backend Engineer at Orkes, you will work at the frontier...  ...architect — has designed systems with real consistency/availability tradeoffs at... 
    Full time
    Work at office
    Flexible hours

    Orkes

    Santa Clara, CA
    2 days ago
  • $128k - $316.8k

     ...Responsibilities The Trust and Safety(TnS) engineering team is responsible for...  ...and abusive behaviors. With the continuous efforts of our...  ...following programming languages: C++, Python, Java; Be proficient in the...  ...and related technologies (Spark, Flink, ClickHouse, HBase etc.... 
    Full time
    Temporary work
    Local area

    TikTok

    San Jose, CA
    5 days ago
  • $150.4k - $277.6k

     ...private, and secure experience for users to locate their devices, personal items, and share location with friends and family. We're a group of highly motivated engineers, designers, and product managers who are passionate about creating a simple and reliable experience... 
    Work experience placement
    Work at office
    Worldwide
    Relocation

    Apple

    Cupertino, CA
    4 days ago
  • $150k - $189k

     ...Seattle, WA / San Francisco, CAEngineering - Engineering - Backend /Full-time /Hybrid*This role supports...  ...quality-driven software development. With high quality software as our ultimate...  ...not required, experience working with Spark pipelines is a plus along with knowledge... 
    Full time
    Work at office

    Wealthfront

    Palo Alto, CA
    1 day ago
  • $129.4k - $198.4k

     ...DescriptionThe Role:As a Senior Software Engineer focused on backend platform and API ownership, you will...  ...role is expected to partner closely with product, frontend, platform, QA, DevOps...  ...programming languages such as Java, Kotlin, Python, Go, or similarExperience designing... 
    Full time
    Local area
    Work from home
    Relocation package
    Flexible hours

    General Motors

    Sunnyvale, CA
    3 days ago
  • $153.2k - $269.1k

     ...a global scale.The RoleIn this Senior Backend Software Engineer role, you will lead and develop high performance...  ...and external customers Collaborate with stakeholders across the organization...  ...experience in Java / Kotlin or Python Experience working with SQL and NoSQL... 
    Full time
    Local area
    Work from home
    Relocation package
    Flexible hours

    General Motors

    Sunnyvale, CA
    2 days ago
  • $117k - $234k

     ...and act on this data. Partner with engineering, AI/ML, and product teams to ensure...  ...clean, testable code in Python, Java, or Scala; experience with PySpark/Spark for distributed data processing...  ...engineering experience across backend and frontend development. Advanced... 

    Walmart

    Sunnyvale, CA
    4 days ago
  •  ...passion and customer focus to the business.If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!Staff Backend and Test Engineering P60:We are looking for a self-motivated, data-oriented Backend and Test Engineering... 
    Full time
    Contract work

    Lumentum Operations

    San Jose, CA
    11 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Python backend engineer with PySpark/Spark. Be the first to apply!