Big Data Engineer
Purple Drive
Overview: Key Responsibilities
- Design, develop, and maintain big data pipelines leveraging Hadoop, Spark, and Kafka .
- Develop and optimize J2EE backend services for large-scale data processing and integration.
- Program in Python, Scala, and Java to build robust data and backend solutions.
- Architect and implement cloud-native solutions on GCP and Azure platforms.
- Design and maintain data models, data warehousing solutions, and BI tools to enable data-driven decision making.
- Implement and manage workflow orchestration frameworks such as Automic and Airflow for data pipeline automation.
- Collaborate with cross-functional teams including data scientists, analysts, and DevOps engineers to deliver end-to-end data solutions.
- Ensure data quality, security, and scalability across all platforms.
- Troubleshoot and optimize performance of large-scale distributed systems.
- Bachelor's/Master's degree in Computer Science, Engineering, or related field (or equivalent experience).
- 6-8years of experience in big data engineering and backend development .
- Expertise in Hadoop ecosystem (HDFS, MapReduce, Hive, Spark) and Kafka .
- Strong programming skills in Python, Scala, and Java .
- Hands-on experience with J2EE backend development .
- Proven experience with cloud services (GCP, Azure) for data engineering and application deployment.
- Proficiency in data modeling, warehousing concepts, and BI tools .
- Experience with workflow orchestration tools such as Airflow, Automic .
- Strong problem-solving and debugging skills in large-scale distributed environments.
- Familiarity with containerization and orchestration (Docker, Kubernetes).
- Knowledge of streaming frameworks (Spark Streaming, Flink, Storm).
- Exposure to CI/CD pipelines and DevOps practices.
- Experience in Agile/Scrum development environments .
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Big Data Engineer in Sunnyvale, CA vacancy
- ...Big Data Engineer Location: Mountain View, CA (3 days onsite) Skill Set: ~ Minimum of 5+ years working with Apache Flink or Spark Structured Streaming (preferably Flink) ~5+ years experience with Java or Scala ~ Strong experience working with microservices...Big data
Kasmo Global
Mountain View, CA4 days ago $50 - $60 per hour
...Bigdata Engineer Location: Phoenix, AZ Duration: 12+ Months Client: Client / Impetus Pay Rate: $50 to $60 Max depends on experience... ...year Bachelors Year and university Total Exp US Exp. Big Data Exp. Current Location Open for Travel Open for...Big dataRelocationOmega Solutions
Santa Clara, CA2 days ago- ...Job Summary Seeking a Data Engineer with Spark & Streaming skills to build real-time, scalable data pipelines using tools like Spark, Kafka... ...Strong proficiency in Python, SQL, and potentially Scala/Java. Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming...Big data
Compunnel
Sunnyvale, CA2 days ago - ...Remote Data Engineer needs 10 years experience Must be local top Santa Clara Cloud Platform Expertise (GCP Focus): Extensive hands-on... ...business intelligence, advanced analytics Implement and support Big Data solutions for both batch (scheduled) and real-time/...Big dataLocal areaRemote work
Global Channel Management
Santa Clara, CA1 day ago - ...Sunnyvale, CA although will accept candidates from Bentonville, AR. Job Description: We are seeking a skilled Data Engineer with strong expertise in Big Data, Cloud platforms, and distributed data systems . The ideal candidate will have hands-on experience in...Big data
Purple Drive
Sunnyvale, CA1 day ago - ...Title: Senior Data Engineer Location: Sunnyvale, CA Position: 2 Month (Contract) Summary Provide business insights,... ...fast-paced environment. Strong background in transforming big data into business insights Technical Requirements...Big dataContract work
TWO95 International
Sunnyvale, CA2 days ago - ...train AI agents to automate tasks, analyze data, and enhance team productivity. Our... ...overview We are looking for an experienced Data Engineer to join our Engineering team. The ideal candidate... ...cloud technologies, especially AWS, and big data systems, your role will be pivotal in...Big dataFlexible hours
BREVIAN
Sunnyvale, CA3 days ago - ...Data Engineer Rootshell Enterprise Technologies Inc. is a recognized provider of professional IT Consulting services in the US. We are actively... ...building and optimizing data pipelines and integrating with big data platforms like Apache Spark and Hadoop. Develop efficient,...Big data
Rootshell Inc
Santa Clara, CA4 days ago - ...impacting millions of customers worldwide drive value from data at Walmart Scale Interact with Walmart engineering teams across geographies to leverage expertise and... .... Your Qualifications Min 6-10 years of Big data development experience Demonstrates up-to-...Big dataWorldwide
Software Technology Inc
Sunnyvale, CA4 days ago - ...for your reference. Recent update from client for this data engineer role / Request-ID: 13145-1 _ Bentonville, AR position is no... ...As a Senior Data Engineer, you will • Design and develop big data applications using the latest open source technologies....Big data
Texas State Library and Archives Commision
Sunnyvale, CA1 day ago - DRISHTICON Inc is seeking a Data Engineer to support our client in Sunnyvale, CA, for a hybrid long-term contract position. The Data Engineer will design, develop, and deploy enterprise-scale big data solutions, primarily using Hadoop ecosystem technologies. The ideal candidate...Big dataLong term contract
- A pioneering AI technology firm is seeking an experienced Data Engineer to enhance its scalable data systems. The position requires expertise... ..., particularly AWS, and solid experience in developing big data systems. With responsibilities that include designing high...Big data
- ...seeking a skilled professional with strong data pipeline development skills and hands-on... .... This role combines traditional data engineering responsibilities with infrastructure-focused... ...of data modeling, warehousing, and big data ecosystems ~ Prior work experience...Big dataWork experience placementImmediate startFlexible hours
Artech
Cupertino, CA1 day ago - ...Data Engineer (SW) Cupertino, CA 12 months Hybrid (3 days in office/week) Hours: 40 per week Role Overview We are seeking a Data Engineer... ...• Solid understanding of data modeling, warehousing, and big data ecosystems Responsibilities • Build and maintain scalable...Big dataWork at office3 days per week
TalentBurst
Cupertino, CA1 day ago $147.4k - $272.1k
...accomplish. We are seeking an exceptional engineer with a background in large scale software... ...using cutting-edge technologies to build data intensive systems, designing and optimizing... ...Java or Scala * 5+ years experience with big data platforms Familiarity with ML/AI workflows...Big dataRelocationApple
Sunnyvale, CA3 days ago- ...Job Title: Data Engineer . Location: Sunnyvale, CA. Direct Client: Req. Key Qualifications: ~ Data Engineer... ...software. ~ Experience in Dimensional Modeling (DM) is a big plus. ~ Ability to learn and adapt to new tools and...Big data
Numeric
Sunnyvale, CA5 days ago - ...Data Engineer We are seeking a highly skilled Data Engineer to join our team, focusing on building, maintaining, and optimizing our data... ...and reporting. This role requires expertise in ETL processes, big data technologies, and cloud data platforms. Key Responsibilities...Big data
Omni Inclusive
Santa Clara, CA5 days ago - A major technology company based in Mountain View is looking for experienced Data Engineers to optimize data infrastructures for machine learning and analytics. Candidates should have a strong foundation in algorithms, a background in data engineering or machine learning...Big data
- ...company in Cupertino is seeking a Software Engineer to join their Services Engineering team. The role involves building scalable data processing systems and partnering with teams... ...in Java or Scala, and experience with big data technologies. The position offers a comprehensive...Big data
$181.1k - $272.1k
...started. Do you have a strong passion for using data to drive business decisions, generate... ...? We are seeking for a Senior Data Engineer to join our App Store Data team and play an... ...solving skills Hands-on experience in modern big data technologies such as: Spark/Flink, Kafka...Big dataRelocationApple
Cupertino, CA3 days ago- ...Data Pipeline Engineer ByteDance is a technology company operating a range of content platforms that inform, educate, entertain and inspire... ...Enterprise Administration, etc. Taking advantage of cutting-edge big data and AI technologies, we are aiming at building a more...Big data
Adapt Talent
Mountain View, CA4 days ago - ...Lead Data Engineer RadiumOne is seeking a Lead Data Engineer to join our Data Technology team with the responsibility of designing and implementing... ...experience ~ Knowledge and expertise of No SQL and Big Data technologies ~ Strong knowledge of Linux ~ Demonstrated...Big data
Stepping Up Solutions
Sunnyvale, CA4 days ago - ...fundamental and applied research in Intelligent data analytics, Hardware/Co-processor... ...interested in all components of a large scale Big Data Analytics system and AI Platforms,... ..., etc. We are leading researchers and engineers from worldwide to create adaptive and high...Big dataFor contractorsWorldwide
Netpace
Santa Clara, CA4 days ago - ...Research America is building a next-generation data platform to support Smart TV products and... .... They will work closely with other data engineers to implement data systems in production-... ...requirements: Broad knowledge of big data technology landscape (Hadoop, Spark,...Big data
Omega Solutions Inc
Mountain View, CA23 days ago $135k - $180k
...Responsibilities: Create and maintain optimal data processing pipeline architecture Build... ..., and loading of data using SQL and big data technologies. Identify, design,... ...stakeholders including the Executive, Engineering, and Operation teams to assist with data-...Big dataPlusAI, Inc.
Santa Clara, CA1 day ago$129.4k - $212.3k
# Senior Data Engineer## General MotorsPublished 28 Mar 2026## Role Highlights### Languages usedPythonSCALASQL### Key skillsData ScienceData... ...and relational/non-relational data storage. (ETL frameworks, big data processing, NoSQL).\\+ 3+ years of experience as a data engineer...Big dataContract workH1bLocal areaRemote workRelocationRelocation packageFlexible hoursTowards AI, Inc.
Mountain View, CA2 days ago$167k - $270.5k
Palo Alto Networks, Inc. is seeking a Principal IT Data Engineer in Santa Clara, California. This role involves architecting and maintaining... ...extensive background to support various teams utilizing Cloud and Big Data technologies. Key responsibilities include ensuring data...Big data$65 per hour
...onsite and offshore delivery models. We are currently looking for a Data Engineer to support our client in Sunnyvale, CA for a long-term contract... ...designing, developing, testing, and deploying enterprise-grade big data solutions that drive critical business initiatives. The...Big dataLong term contractLocal areaDRISHTICON
Sunnyvale, CA3 days ago$126.8k - $220.9k
...Data Engineer Apple's Media, Graphics, and Compute Technologies Group (MGC) is looking for a talented and dedicated big data engineer to join our Data Engineering team. The Data Engineering team within the MGC organization plays a critical role in supporting data-driven...Big dataRelocationApple
Cupertino, CA1 day ago- ...software development or related. 3+ years of solid Java and 2+ years experience in design, implementation, and support of solutions big data solution in Hadoop using Hive, Spark, Drill, Impala, HBase. Hands on experience with Unix, GCP and other relational databases....Big data
Samprasoft
Sunnyvale, CA5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!
Related searches
- remote data engineer Sunnyvale, CA
- entry level big data engineer Sunnyvale, CA
- big data devops engineer Sunnyvale, CA
- data engineer Sunnyvale, CA
- software data engineer Sunnyvale, CA
- big data cloud engineer Sunnyvale, CA
- junior big data engineer Sunnyvale, CA
- sr information security engineer Sunnyvale, CA
- director data engineering Sunnyvale, CA
- hadoop big data developer Sunnyvale, CA