Big Data Engineer

Purple Drive

Sunnyvale, CA

Overview:

Key Responsibilities

Design, develop, and maintain big data pipelines leveraging Hadoop, Spark, and Kafka .
Develop and optimize J2EE backend services for large-scale data processing and integration.
Program in Python, Scala, and Java to build robust data and backend solutions.
Architect and implement cloud-native solutions on GCP and Azure platforms.
Design and maintain data models, data warehousing solutions, and BI tools to enable data-driven decision making.
Implement and manage workflow orchestration frameworks such as Automic and Airflow for data pipeline automation.
Collaborate with cross-functional teams including data scientists, analysts, and DevOps engineers to deliver end-to-end data solutions.
Ensure data quality, security, and scalability across all platforms.
Troubleshoot and optimize performance of large-scale distributed systems.

Required Skills & Qualifications

Bachelor's/Master's degree in Computer Science, Engineering, or related field (or equivalent experience).
6-8years of experience in big data engineering and backend development .
Expertise in Hadoop ecosystem (HDFS, MapReduce, Hive, Spark) and Kafka .
Strong programming skills in Python, Scala, and Java .
Hands-on experience with J2EE backend development .
Proven experience with cloud services (GCP, Azure) for data engineering and application deployment.
Proficiency in data modeling, warehousing concepts, and BI tools .
Experience with workflow orchestration tools such as Airflow, Automic .
Strong problem-solving and debugging skills in large-scale distributed environments.

Preferred Skills (Nice to Have)

Familiarity with containerization and orchestration (Docker, Kubernetes).
Knowledge of streaming frameworks (Spark Streaming, Flink, Storm).
Exposure to CI/CD pipelines and DevOps practices.
Experience in Agile/Scrum development environments .

Apply

Vacancy posted 4 days ago

Similar jobs that could be interesting for youBased on the Big Data Engineer in Sunnyvale, CA vacancy

Big Data Engineer
...Big Data Engineer Location: Mountain view CA (3 days onsite) Skill Set: ~ Minimum of 5+ years working with Apache Flink or Spark Structured Streaming (preferably Flink) ~5+ years experience with Java or Scala ~ Strong experience working with micro services...
Big data
Kasmo Global
Mountain View, CA
2 days ago
Big Data Engineer
$50 - $60 per hour
...Bigdata Engineer Location: Phoenix, AZ Duration: 12+ Months Client: Client / Impetus Pay Rate: $50 to $60 Max depends on experience... ...year Bachelors Year and university Total Exp US Exp. Big Data Exp. Current Location Open for Travel Open for...
Big data
Relocation
Omega Solutions Inc
Santa Clara, CA
3 days ago
Big Data Engineer
$80 - $95 per hour
...This range is provided by OSI Engineering. Your actual pay will be based on your skills and experience... ...Mountain View, CA is seeking a talented Data Engineer to help build secure, scalable... ...engineers with strong expertise in Big Data infrastructure and data warehousing...
Big data
Full time
Contract work
OSI Engineering
Mountain View, CA
2 days ago
Data Engineer
...train AI agents to automate tasks, analyze data, and enhance team productivity. Our... ...overview We are looking for an experienced Data Engineer to join our Engineering team. The ideal candidate... ...cloud technologies, especially AWS, and big data systems, your role will be pivotal in...
Big data
Flexible hours
BREVIAN
Sunnyvale, CA
2 days ago
Senior Data Engineer
$150.4k - $277.6k
...accomplish. We are seeking an exceptional engineer with a background in large scale software... ...using cutting‑edge technologies to build data intensive systems, designing and optimizing... ...languages: Java or Scala 5+ years experience with big data platforms Familiarity with ML/AI...
Big data
Apple
Sunnyvale, CA
4 days ago
Data Engineer
...Data Engineer Location: Mountain View, CA & Dallas, TX (Onsite) You Must Have BS in Computer Science/Engineering, relevant technical field,... ...experience, with specialization in Data Engineering Exp with (Big Data Eng, Hadoop, Hive, Python , PySpark, ETL & SQL, Any Cloud,...
Big data
Permanent employment
Contract work
Local area
Robotics Technologies LLC
Mountain View, CA
2 days ago
Data Engineer
$135k - $180k
...Responsibilities Create and maintain optimal data processing pipeline architecture Build... ..., and loading of data using SQL and big data technologies. Identify, design, and... ...with stakeholders including the Executive, Engineering, and Operation teams to assist with data-...
Big data
PlusAI, Inc.
Santa Clara, CA
2 days ago
Senior Data Engineer
Job Summary Seeking a Data Engineer with Spark & Streaming skills to build real-time, scalable data pipelines using tools like Spark, Kafka,... ...Strong proficiency in Python, SQL, and potentially Scala/Java. Big Data: Expertise in Apache Spark (Spark SQL, DataFrames,...
Big data
Compunnel
Sunnyvale, CA
17 hours ago
Data Engineer
$65 per hour
...onsite and offshore delivery models. We are currently looking for a Data Engineer to support our client in Sunnyvale, CA for a long-term contract... ...designing, developing, testing, and deploying enterprise-grade big data solutions that drive critical business initiatives. The...
Big data
Long term contract
Local area
DRISHTICON
Sunnyvale, CA
1 day ago
Data Engineer
$129.3k - $225.3k
...States Software and Services Apple's Media, Graphics, and Compute Technologies Group (MGC) is looking for a talented and dedicated big data engineer to join our Data Engineering team. The Data Engineering team within the MGC organization plays a critical role in supporting...
Big data
Relocation
Apple
Cupertino, CA
2 days ago
Remote Data Engineer
Remote Data Engineer needs 10 years experience Must be local top Santa Clara Cloud Platform Expertise (GCP Focus): Extensive hands-on experience... ...intelligence, advanced analytics Implement and support Big Data solutions for both batch (scheduled) and real-time/streaming...
Big data
Remote job
Local area
Global Channel Management
Santa Clara, CA
17 hours ago
Lead Data Engineer
Job Description RadiumOne is seeking a Lead Data Engineer to join our Data Technology team with the responsibility of designing and implementing... ...desired Qualifications Knowledge and expertise of No SQL and Big Data technologies Strong knowledge of Linux Demonstrated...
Big data
Stepping Up Solutions
Sunnyvale, CA
1 day ago
Senior Data Engineer
$181.1k - $272.1k
...started. Do you have a strong passion for using data to drive business decisions, generate... ...? We are seeking for a Senior Data Engineer to join our App Store Data team and play an... ...solving skills Hands-on experience in modern big data technologies such as: Spark/Flink, Kafka...
Big data
Relocation
Apple
Cupertino, CA
2 days ago
Cloud Data Engineer
...Cloud Data Engineer This role has been designed as 'Onsite' with an expectation that you will primarily work from an HPE office. Job... ...~1+ years of programming experience in Java ~ Expertise in big data technologies such as Apache Spark or Kafka with at least 1...
Big data
Work experience placement
Work at office
Hewlett Packard Enterprise
Alviso, CA
3 days ago
Big Data Engineer
...Big Data Engineer San Jose, CA (Remote) Experian Need 8 years of experience and hands on programming experience in any one or more programming languages like Java, Scala or Python Good experience with distributed systems like Hadoop, HDFs and No SQL databases...
Big data
Remote work
Omega Solutions Inc
San Jose, CA
17 hours ago
Big Data Engineer
...Java, Spark and Hadoop A minimum of 2 years of experience working with distributed systems Knowledge in distributed system design, data pipelining, and implementation Knowledge in machine learning algorithms Knowledge and experience in building large scale applications...
Big data
Contract work
Immediate start
Cygnus Professionals
Palo Alto, CA
1 day ago
Big Data Engineer
Job Title: Big Data Engineer Location: San Jose, CA Duration: Long Term Contract Job Description Client is looking for Big Data engineers with expert level experience in developing enterprise software applications on Spark/Hadoop, NoSql, InMemory Databases. You will...
Big data
Long term contract
Immediate start
Rapinno Tech
San Jose, CA
3 days ago
Big Data Engineer
$100k - $150k
...Big Data Engineer – Remote Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This is a fantastic opportunity to join an established and well-respected...
Big data
Full time
H1b
Local area
Immediate start
Remote work
Visa sponsorship
Bright Vision Technologies
Sunnyvale, CA
1 day ago
Data Engineer Intern (E-commerce) - 2026 Summer (BS/MS)
$45 per hour
...Responsibilities Design and build data transformations efficiently and reliably for different... ..., scalable, robust and extensible big data systems that support core products and... .... Establish solid design and best engineering practice for engineers as well as non‑technical...
Big data
Hourly pay
Summer work
Internship
Ellis Technologies, Inc.
San Jose, CA
4 days ago
Data Engineer
...and utilize structured and unstructured data from diverse sources. We build the essential... ...the cleansing and organization of big data to ensure its integrity. You will work... ...visualize large datasets. Improve feature engineering and build innovative models to solve a variety...
Big data
Local area
Cisco
San Jose, CA
11 hours ago
Data Engineer
$69.01 per hour
Job Details Team: US Digital Global Digital Commerce Data Engineer 1 (Contract) Location: San Jose, CA 95110 (Hybrid) Duration: 6/08/202... ...complex queries against large datasets Hands‑on experience with Big Data technologies such as Hadoop, Hive, Spark, or similar...
Big data
Hourly pay
Contract work
Cypress HCM
San Jose, CA
1 day ago
Senior Data Engineer
$175k - $200k
...Overview East West Bank is looking for a motivated Senior Data Engineer to contribute towards the success of our Data and Analytics Technology... ...Scala with Databricks/Spark Experience in NoSQL Databases and Big data technologies including Azure CosmosDB or other Document...
Big data
Work experience placement
East West Bank
Palo Alto, CA
17 hours ago
Data Engineer Intern (E-commerce) - 2026 Summer (BS/MS)
$45 per hour
...th, 2026 June 8th, 2026 June 22nd, 2026 Team Introduction As a data engineer in the Data Platform E‑Commerce team, you will have the opportunity... ...Design and implement reliable, scalable, robust and extensible big data systems that support core products and business; Establish...
Big data
Hourly pay
Summer work
Internship
Local area
Tik Tok
San Jose, CA
11 hours ago
Data Analytics Developer
We are seeking a developer to join the data analytics team with at least 5 years of software... ...be a self-learner as it relates to new big data analytics technologies. Job... ...Qualifications B.Sc. degree in Computer Science, Engineering, Mathematics, Statistics or related field...
Big data
Samsung
San Jose, CA
4 days ago
Senior Manager, Data Engineering
$83.98 - $111.27 per hour
...Health Care is seeking a dynamic and experienced Senior Manager of Data Engineering to lead our data engineering teams. This role is critical to... ..., Snowflake, BigQuery, Redshift). Proficiency with big data technologies (e.g., Spark, Kafka) and workflow orchestration...
Big data
Hourly pay
Stanford Health Care
Palo Alto, CA
2 days ago
Data Engineer
$70 - $95 per hour
...Job Description Job Description Data Engineer The Cloud services group is looking for world class engineers to join our technology innovation... .... Looking for world class server software engineers with Big Data Infrastructure and Data warehousing experience to join our...
Big data
Hourly pay
Cypress HCM
Mountain View, CA
16 days ago
Staff Data Engineer
...accelerate next-generation computing—from AI and data centers to PCs, gaming, and embedded systems. Join... ...and beyond. The Role We are seeking an expert Data Engineer to design, develop, and maintain an Enterprise-wide Big Data and Data Analytics Platform. Key Responsibilities...
Big data
AMD
San Jose, CA
3 days ago
Senior Data Engineer
$148.75k - $361k
...while gaining meaningful experience across a variety of disciplines. About the team The mission of Roku's Data Engineering team is to develop a world-class big data platform that enables internal and external customers to leverage data to grow their businesses. Data Engineering...
Big data
Work at office
Local area
Remote work
Worldwide
Monday to Thursday
Flexible hours
TryApplyNow
San Jose, CA
2 days ago
Senior Data Infra Engineer
$175k - $221k
...content intelligence. If you're inspired to dream big, innovate fast, and make a difference, we'd love... ...A/B test, and every ML model we ship runs on the data infrastructure you'd own. We're looking for a senior engineer to build and scale the batch and streaming...
Big data
Full time
Local area
Work from home
NewsBreak
Mountain View, CA
28 days ago
Senior Data Engineer, Fleet Monitoring & Analysis
$153k - $204k
...and high-reliability operations. About the role: As a Senior Data Engineer you will own and evolve the data lake and analytics stack that... ...data pipeline orchestration tools (e.g., Apache Airflow) and big data technologies (e.g., Apache Spark). ~ Experience designing...
Big data
Permanent employment
Temporary work
Casual work
Work at office
Flexible hours
CoreWeave
Sunnyvale, CA
15 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!