EMR/Spark SQL & Job Query-Tuning Engineer

a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; } EMR/Spark SQL & Job Query-Tuning Engineer

Location: Seattle, WA (Onsite)

About Us

STAFFXPERT LLC is a leading talent solutions and technology staffing firm connecting skilled professionals with innovative organizations across diverse industries. We are committed to delivering exceptional opportunities and helping top talent advance their careers through impactful projects and long-term partnerships.

Job Summary

STAFFXPERT LLC is seeking an EMR/Spark SQL & Job Query-Tuning Engineer on behalf of our client in Seattle, WA. This role is ideal for a Spark performance expert who specializes in optimizing PySpark, Spark SQL, and Hive workloads within large-scale data environments. The successful candidate will focus on improving application-level performance by tuning queries, optimizing execution plans, reducing memory consumption, minimizing shuffle operations, and enhancing overall job efficiency on Amazon EMR.

Key Responsibilities

• Optimize and tune PySpark, Spark SQL, and Hive SQL jobs to improve performance, scalability, and resource utilization.
• Analyze Spark execution plans and DAGs to identify and resolve performance bottlenecks.
• Design and implement efficient partitioning, caching, and data processing strategies.
• Reduce shuffle overhead, spill events, executor memory pressure, and job execution times.
• Optimize join strategies, including broadcast joins, sort-merge joins, and other Spark execution techniques.
• Troubleshoot data skew, serialization issues, and distributed processing inefficiencies.
• Collaborate with data engineering and platform teams to improve workload performance and reliability.
• Monitor, benchmark, and continuously enhance large-scale data processing jobs.
• Recommend and implement best practices for Spark and EMR performance optimization.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent professional experience.
• Strong hands-on experience with PySpark, Spark SQL, Hive SQL, and Amazon EMR.
• Proven expertise in Spark performance tuning, query optimization, and execution plan analysis.
• Deep understanding of Spark internals, distributed computing concepts, and data processing frameworks.
• Experience optimizing large-scale data workloads and addressing memory, partitioning, and shuffle-related challenges.
• Proficiency in analyzing Spark UI metrics and troubleshooting performance issues.
• Strong problem-solving skills and ability to work in a fast-paced environment.

Preferred Qualifications

• Knowledge of Adaptive Query Execution (AQE), Catalyst Optimizer, and Tungsten Engine.
• Experience working with Parquet, ORC, and other columnar data formats.
• Familiarity with AWS data services and cloud-based big data platforms.
• Experience supporting enterprise-scale analytics or data engineering environments.