Lead Data Engineer

Description Senior Data Engineer (Contract) Location: Glendale, CA | Hybrid Employment Type: Long Term Contract Pay: Available on W2 Basis Position Overview We are seeking a Senior Data Engineer to support enterprise data engineering and platform modernization initiatives. This role will focus on designing, building, and optimizing scalable data solutions while partnering with cross-functional teams to enhance cloud-based data platforms and data-driven decision-making. The ideal candidate will bring strong expertise in Databricks, Python, Apache Spark, and Terraform , along with experience designing and operating enterprise-scale data pipelines and platforms. This position offers the opportunity to work on complex data engineering challenges involving modern cloud technologies, streaming and batch processing, automation, governance, and platform architecture. Key Responsibilities:

• Design, develop, test, deploy, and maintain scalable batch and streaming data pipelines.
• Build and optimize data solutions using Databricks, Apache Spark, Python, SQL, and related technologies.
• Collaborate with stakeholders to gather requirements and translate business needs into scalable technical solutions.
• Support and maintain data platform governance, including access controls, data lineage, cataloging, and data discovery capabilities.
• Diagnose platform and pipeline issues, identify root causes, and recommend effective solutions.
• Contribute to solution architecture across cloud, data, and orchestration platforms.
• Develop and maintain containerized services and utilities using Docker and Kubernetes.
• Implement infrastructure automation and deployment standards using Infrastructure as Code practices.
• Monitor platform health, resource utilization, system performance, and operational efficiency.
• Build and enhance CI/CD processes and DevOps workflows.
• Implement data quality, monitoring, logging, and reliability standards across pipelines and platforms.
• Partner with engineers, architects, product teams, and business stakeholders in an Agile environment.
• Maintain technical documentation, standards, and platform configurations.

Required Qualifications 5+ years of experience in Data Engineering or a related field. Strong hands-on experience with Databricks and Apache Spark for both batch and streaming data processing. Advanced Python development experience. Experience developing data pipelines using PySpark and/or Scala. Strong understanding of Spark architecture, including partitioning, shuffling, stages, executors, and performance optimization. Experience with Terraform and Infrastructure as Code methodologies. Advanced SQL development and query optimization skills. Experience building and optimizing ETL/ELT pipelines. Production experience with Airflow (including MWAA or similar orchestration platforms). Experience with data governance, access management, and platform administration. Experience with Docker and Kubernetes. Knowledge of cloud technologies, including compute, storage, networking, and identity/access management concepts. Experience implementing CI/CD practices and Git-based development workflows. Strong troubleshooting, analytical, and problem-solving skills. Excellent communication and collaboration skills.

Preferred Qualifications Experience with AWS cloud services. Experience supporting enterprise-scale cloud data platforms. Experience with Snowflake or comparable modern data technologies. Experience with Scala, Kotlin, or additional programming languages. Familiarity with networking concepts related to distributed systems and cloud integrations. Experience implementing data quality, observability, and monitoring frameworks. Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience. Advanced degree is a plus.

Technology Doesn't Change the World, People Do. OpenKyber is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles. OpenKyber works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the OpenKyber app and get 1-tap apply, notifications of AI-matched jobs, and much more. OpenKyber will consider for employment qualified applicants with arrest or conviction records in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information. 2025 OpenKyber. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to OpenKyber's Terms of Use and Privacy Notice.

For applications and inquiries, contact:View email address on us.fitly.work