Data Engineer

Purple Drive

Overview:

****Preferred location is Sunnyvale, CA although will accept candidates from Bentonville, AR.

Job Description:

We are seeking a skilled Data Engineer with strong expertise in Big Data, Cloud platforms, and distributed data systems . The ideal candidate will have hands-on experience in designing, building, and optimizing data pipelines, API integrations, and real-time stream-processing systems.

Key Responsibilities:

Design, develop, and optimize large-scale data pipelines and ETL workflows.
Work with Java, Python, and Scala to build scalable data solutions.
Develop APIs and integrate with systems using Node.js, GQL, and RESTful services .
Implement big data solutions leveraging Hadoop, Hive, Spark (Scala), Presto/Trino, and Data Lake architectures .
Deploy and manage workflows with Airflow, Luigi, Automic , and similar orchestration tools.
Build and maintain real-time data streaming systems using Storm, Spark-Streaming, and Kafka .
Utilize Vertex AI and Cloud services (AWS/GCP/Azure) for advanced analytics and ML integration.
Ensure system reliability, scalability, and performance in distributed environments.
Collaborate with cross-functional teams (data scientists, analysts, and engineers) to deliver high-quality data solutions.
Apply best practices in CI/CD, Kubernetes-based deployments, and monitoring.

Required Skills:

Strong programming skills in Java, Python, and Scala .
Expertise in big data frameworks : Hadoop, Hive, Spark (Scala).
Hands-on with API development (REST, GraphQL, Node.js).
Experience with stream-processing tools : Kafka, Storm, Spark-Streaming.
Proficiency with workflow orchestration : Airflow, Luigi, Automic.
Knowledge of Presto/Trino and distributed SQL query engines.
Cloud experience (AWS, GCP, or Azure), with exposure to Vertex AI .
Strong understanding of Data Lake and data warehousing concepts .
Experience with Kubernetes for container orchestration.

Apply

Vacancy posted 4 days ago

Similar jobs that could be interesting for youBased on the Data Engineer in Sunnyvale, CA vacancy

Lead Data Engineer
$251k - $310k
...U.S. states. This role follows a hybrid work schedule and reports to Sr. Staff TLM. You will: Lead, mentor, and grow a team of data engineers responsible for the Waymo Commercialization Data Lake and associated data pipelines. Define and drive the technical vision, strategy...
Suggested
Full time
Remote work
Waymo
Mountain View, CA
1 day ago
Lead Data Engineer
...NewRocket is looking for an AI Data Intelligence Lead who will design AI-powered capabilities for modernizing enterprise data. This... ...entails applying machine learning, generative AI, and advanced data engineering to enrich and optimize enterprise data. Ideal candidates will...
Suggested
Remote work
NewRocket Inc
Santa Clara, CA
1 day ago
Lead Data Engineer
...Lead Data Engineer RadiumOne is seeking a Lead Data Engineer to join our Data Technology team with the responsibility of designing and implementing scalable data infrastructure, as well as implementing data transformations developed by data scientists as scalable and...
Suggested
Stepping Up Solutions
Sunnyvale, CA
11 hours ago
Lead Data Engineer
...Lead Data Engineer Data team builds and operates the data marts and pipelines that power HR/Workforce reporting and analytics across Intuit. We're looking for a hands-on Data Engineer who can operate independently. High-visibility, high-ownership — direct exposure to...
Suggested
United IT
Mountain View, CA
11 hours ago
Lead Data Engineer
...Hi, Position: Lead Data Engineer Location: Sunnyvale, CA/ Austin, TX (Onsite) Ex-Apple is preferred but if we don't find non-ex-apple are also ok. Job Description: Role Overview We are seeking a highly skilled and strategic Lead Data Engineer...
Suggested
Diverse Lynx
Sunnyvale, CA
11 hours ago
Data Engineer
$73 per hour
...Data Engineer MAHIN-JOB-35556 We are looking for Data Engineer in Sunnyvale,CA Day 1 onsite ( 3 days a week hybrid) FTE- 130k per annum + benefits Subcon-73 $ per hour ( no layers are accepted) Job Description: Strong Python development and software...
Hourly pay
3 days per week
1 day per week
Keylent Inc
Sunnyvale, CA
3 days ago
Data Engineer
$135k - $180k
...PlusAI in Santa Clara is seeking a talented Software Engineer to create and maintain an optimal data processing pipeline architecture. You will build the required infrastructure, automate processes, and provide actionable insights into business performance metrics. The...
PlusAI, Inc.
Santa Clara, CA
1 day ago
Data Engineer
...growing, and we are looking for a highly motivated Principal Software Engineer who is a self-starter and will play a critical role in the... ...Technology team, you will be building highly scalable and reliable APIs, data-pipelines, services, and applications which will drive the next...
Local area
My3Tech Inc
Sunnyvale, CA
5 days ago
Data Engineer
...Job Title Primary Responsibilities: Expertise in SQL: You possess a deep understanding of SQL for data manipulation, querying, and performance optimization across various database systems. Proficiency in Python: Your Python skills are crucial for building...
Yochana
Sunnyvale, CA
1 day ago
Data Engineer
...test and deploy cutting edge solutions at scale, impacting millions of customers worldwide drive value from data at Walmart Scale Interact with Walmart engineering teams across geographies to leverage expertise and contribute to the tech community. Engage with...
Worldwide
Software Technology Inc
Sunnyvale, CA
1 day ago
Data Engineer
...Data Engineer Location: Mountainview, CA (Hybrid 3 days' work from office) C2C pay rate: $70-75/hr for both the roles. Role Summary: Responsible for designing and building graph data models, knowledge graphs, and data pipelines optimized for GraphRAG and reasoning...
Work at office
Kasmo Global
Mountain View, CA
1 day ago
Data Engineer II
...Employer: \t\tAMAZON WEB SERVICES, INC. Offered Position: \tData Engineer II Job Location: \t\tSanta Clara, California Job Number: \... ..., and operate large-scale, high-volume, high-performance data structures for business intelligence analytics. Implement data structures...
Local area
Amazon
Santa Clara, CA
5 days ago
Data Engineer - Hybrid
...software development experience with very high proficiency in Python, Scala, or Java and experience optimizing SQL queries on large data. Experience working with large-scale data warehouse solutions such as Teradata, Snowflake, or Redshift. Hands on experience in a Unix...
The Dignify Solutions, LLC
Sunnyvale, CA
5 days ago
Data Engineer
...Data Engineer Rootshell Enterprise Technologies Inc. is a recognized provider of professional IT Consulting services in the US. We are actively seeking a Data Engineer for one of our clients. Location: Santa Clara, CA - Onsite We are seeking a highly skilled and...
Rootshell Inc
Santa Clara, CA
1 day ago
Data Engineer
...Dexian is seeking a Data Engineer for a 100% remote, 4+ month engagement. You will build and maintain dashboards, Hive tables, and scalable ETL/ELT data pipelines for a ProdOps team. The role requires 3+ years of experience with strong SQL (Presto/Hive), Python, data warehousing...
Remote work
Worldwide
Dexian
Santa Clara, CA
13 hours ago
Data Engineer
...Location: Sunnyvale, CA (5x/ week onsite) Duration: 6 months Data Engineer Must-haves: Must have Snowflake, Advanced SQL, Dimensional Data Modelling (Slowly Changing Dimensions), Python, Vertica, Spark streaming, AWS (Computing). MUST HAVE Hands-on experience...
Q1 Technologies
Sunnyvale, CA
3 days ago
Data Engineer GCP
...Data Engineer GCP Sunnyvale, California, United States Job Title: Data Engineer Location: Sunnyvale Ca Visa: USC/GC/H1b Pay rate: W2 only no C2C Details: 5 years of SQL queries Tech stack Airflow pipeline development GCP dataproc/ BQ Expertise Hive/Spark Python/Java...
H1b
Interaction24 LLC
Sunnyvale, CA
3 days ago
Data Engineer
...Samsung Electronics Perú is seeking a Sr. Data Engineer to contribute to scalable data pipelines and support the Performance & Reach engineering organization. You will work with large advertising datasets and collaborate with cross-functional teams to deliver data solutions...
Samsung Electronics Perú
Mountain View, CA
1 day ago
Data Engineer
...time platform that transforms continuous streams of multimodal data (including structured, image, and log data) into an intelligent... ...QUALIFICATIONS Masters Degree 10+ years of experience in data engineering, including building and maintaining large-scale ETL/ELT data pipelines...
Socket
Cupertino, CA
1 day ago
Data Engineer
...PlusAI is seeking a data engineer to create and maintain optimal data processing pipeline architecture in Santa Clara, California. The role involves building infrastructure for data extraction, transforming, and loading, while also automating manual processes and optimizing...
Medium
Santa Clara, CA
1 day ago
Data Engineer
...Data Engineer Location: Austin, TX/Sunnyvale, CA Job Description We are seeking an experienced Data Engineer with proven, hands-on expertise in migrating enterprise database workloads from Oracle to PostgreSQL. The ideal candidate will lead/support end-to-end...
Futran Tech Solutions Pvt. Ltd.
Sunnyvale, CA
5 days ago
Snowflake Data Engineer
...Title: Snowflake Data Engineer Location: Sunnyvale, CA (Hybrid) Duration: 6 months (possibility of extension) Implementation Partner: Infosys End Client: To be disclosed JD: Required Skills Snowflake Keystone AI/ML skills...
Wise Skulls
Sunnyvale, CA
5 days ago
Data Engineer
$150k
...Data Engineer Specializing In Natural Language Processing (NLP) We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative...
Visa sponsorship
Institute of Foundation Models
Sunnyvale, CA
1 day ago
Data Engineer
$111.8k - $134.3k
...and CE Offline teams to execute on the following projects Key Responsibilities Migrate 2 Bureau Gateway Report Service into Partner Data Service (Experian Gateway Report Service and Equifax Gateway Report Service) Test coverage – 100% when the migration is done Onboard...
Contract work
Work at office
Aithical Hub Inc.
Mountain View, CA
4 days ago
Data Engineer
Coupang is seeking a skilled Database Engineer in Mountain View, California, to join the Customer Success team. You will be responsible for ensuring the reliability and performance of databases in a high-traffic e-commerce environment. The role requires strong expertise...
Coupang
Mountain View, CA
3 days ago
Data Engineer
• 10+ years of overall experience in data management space and at least 5 years of working in large data sets in a data lake environment : • Highly proficient in SQL, • Solid understanding of Spark including performance tuning. • Solid understanding of the AWS Platform...
Exaways Corporation
Mountain View, CA
1 day ago
Data Engineer
...Data Engineer Cupertino, CA ***Only share profile, who is interested to appear for face to face interview in Cupertino, CA*** • 8+ years of experience in Data Engineering with advanced SQL skills • Good with Python programming and experience with Data...
Q1 Technologies
Cupertino, CA
3 days ago
Data Engineer
...Role Overview: This role focuses on comprehensive data management, from strategy and source identification to transformation, modeling, and governance. The Data Engineer will be responsible for developing data pipelines, designing data models, writing code for solutions...
Prophecy Technologies
Sunnyvale, CA
11 hours ago
Data Engineer
...Apple in Santa Clara, CA is seeking an experienced Data Engineering Manager to lead data programs, architectures, and pipelines across large‑scale data products. You will guide delivery, collaborate with cross‑functional teams, and mentor engineers to ensure high quality...
Apple
Santa Clara, CA
1 day ago
Data Engineer
...Data Strategy Understands, articulates, and applies principles of the defined strategy to routine business problems that involve a single function. Data Transformation and Integration Extracts data from identified databases. Creates data pipelines and transform...
Diverse Lynx
Sunnyvale, CA
11 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!