Senior Serverless Spark Engineer

Leading the migration of production Apache Spark workloads from on-premise environments to cloud-native architectures, the fully remote Senior Serverless Spark Migration Engineer will assess applications, design modern solutions, and optimize performance across AWS and GCP. Key responsibilities Lead migrations of enterprise Spark workloads to AWS and GCP, determining the appropriate migration approach Modernize traditional cluster-based workloads for serverless Spark and refactor legacy applications for cloud portability Build reusable migration tooling and implement CI/CD and Infrastructure as Code practices Required qualifications 8+ years of experience in data engineering, distributed systems, cloud engineering, or platform engineering 5+ years of hands-on experience with Apache Spark in enterprise environments Strong development experience in PySpark and/or Scala Proven experience migrating large-scale Spark workloads between infrastructure platforms Hands-on experience with both AWS and GCP cloud services Virtual Vocations Inc