Data Engineer
Part-time
Match Point Solutions
Job Title: Data Engineer
Location: [Your Location or "Remote"]
About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Databricks and PySpark. The ideal candidate will also have hands-on experience with Airflow for orchestration and Kafka for streaming data pipelines. You will play a key role in building, optimizing, and maintaining our data infrastructure to support data-driven decision-making across the organization.
Responsibilities:
- Design, build, and maintain scalable data pipelines using Databricks and PySpark.
- Develop and manage ETL workflows and data integration processes.
- Implement data orchestration and workflow automation using Apache Airflow.
- Build real-time and batch data streaming solutions leveraging Apache Kafka.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
- Optimize data workflows for performance, scalability, and reliability.
- Ensure data quality, integrity, and security across all pipelines.
- Monitor and troubleshoot data pipelines, ensuring minimal downtime and data loss.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- 3+ years of experience in data engineering or a similar role.
- Strong hands-on experience with Databricks and PySpark.
- Proven experience in building and maintaining data pipelines with Apache Airflow.
- Solid understanding of Apache Kafka, including producing and consuming data streams.
- Proficiency in SQL and working with large-scale data processing.
- Experience with cloud platforms (AWS, Azure, or GCP) is a plus.
- Familiarity with data modeling, warehousing, and governance best practices.
- Excellent problem-solving and communication skills.
Nice to Have:
- Experience with Delta Lake, Spark Streaming, or other big data technologies.
- Knowledge of CI/CD tools and DevOps practices for data workflows.
- Exposure to monitoring tools like Datadog, Grafana, or similar.
Location: [Your Location or "Remote"]
About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Databricks and PySpark. The ideal candidate will also have hands-on experience with Airflow for orchestration and Kafka for streaming data pipelines. You will play a key role in building, optimizing, and maintaining our data infrastructure to support data-driven decision-making across the organization.
Responsibilities:
- Design, build, and maintain scalable data pipelines using Databricks and PySpark.
- Develop and manage ETL workflows and data integration processes.
- Implement data orchestration and workflow automation using Apache Airflow.
- Build real-time and batch data streaming solutions leveraging Apache Kafka.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
- Optimize data workflows for performance, scalability, and reliability.
- Ensure data quality, integrity, and security across all pipelines.
- Monitor and troubleshoot data pipelines, ensuring minimal downtime and data loss.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- 3+ years of experience in data engineering or a similar role.
- Strong hands-on experience with Databricks and PySpark.
- Proven experience in building and maintaining data pipelines with Apache Airflow.
- Solid understanding of Apache Kafka, including producing and consuming data streams.
- Proficiency in SQL and working with large-scale data processing.
- Experience with cloud platforms (AWS, Azure, or GCP) is a plus.
- Familiarity with data modeling, warehousing, and governance best practices.
- Excellent problem-solving and communication skills.
Nice to Have:
- Experience with Delta Lake, Spark Streaming, or other big data technologies.
- Knowledge of CI/CD tools and DevOps practices for data workflows.
- Exposure to monitoring tools like Datadog, Grafana, or similar.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Data Engineer in Milpitas, CA vacancy
- ...Data & Analytics Engineer Our Information Services (ISS) department is looking for an outstanding Data & Analytics Engineer with 5+ years of experience working with data and is passionate about technology & architecture, loves hands-on development and wants to help...SuggestedLocal area
Netpace
Milpitas, CA4 days ago - ...roleGridmatic’s business is complex, and we need to build a robust data layer to be able to answer core questions around profit and loss... ...our renewable supply.We’re looking for a startup-minded data engineer who can wear a lot of hats, work with multiple teams, and build...SuggestedWork at officeHome officeFlexible hours3 days per week
$80 - $85 per hour
...solutions that drive business transformation. As a leading product engineering firm based in Silicon Valley, we provide full-cycle product development, leveraging cutting-edge technologies in AI, ML, and data analytics. Our collaborative, inclusive culture fosters...SuggestedHourly payBayrock Labs
Milpitas, CA4 days ago- ...Lead Data EngineerRADIUMONE IS A GLOBAL PROGRAMMATIC AD BUYING PLATFORMRadiumOne is the 6th largest web property in the U.S. according... ....Based in San Francisco, we are seeking a talented Software Engineering Manager.RadiumOne generates first-party data about actual customers...Suggested
Stepping Up Solutions
Sunnyvale, CA2 days ago $220.05k - $226k
Design and build secure, scalable, and reliable data processing systems and infrastructure.Develop and maintain data models and pipelines... ...Minimum qualifications:Bachelor’s degree in Computer Science, Engineering, Computer Information Systems, Mathematics, Physics, or a...SuggestedFull timeWork at office- ...Role: Lead Data Engineer Location: Mountain View, CA / Dallas, TX (Onsite) About the Role Data team builds and operates the data marts and pipelines that power HR/Workforce reporting and analytics across Intuit. We're looking for a hands-on Data Engineer...
United IT Solutions
Mountain View, CA3 days ago $114k - $180k
What this Job Entails: We are looking for a Data Engineer to join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional...Full timeTemporary workWork at officeRemote work- ...Job-ID Reference TypeContractRemote30% RemoteJob Description: The Data Science Engineering Analyst will play a pivotal role in analysing complex datasets, designing and implementing predictive models, and deriving actionable insights to drive business decisions. This...
US Tech Solutions
Mountain View, CA4 hours ago $100.79k - $160.26k
...DLA Piper is a place you can engage in meaningful work and grow your career. Let’s see what we can achieve. Together.SummaryThe Data Engineer, Solutions & Data role designs, builds, and operates data pipelines and data integration processes that translate raw data into...Full timeWork at officeRemote workVisa sponsorship$75 - $80 per hour
DescriptionResponsibilitiesArchitect, construct, and maintain robust, scalable data pipelines and complex ETL workflows.Manage and optimize data... ....Minimum Qualifications3-5 years of experience in Data Engineering.Bachelor’s degree in Computer Science, Engineering, or...Contract workTemporary workWork experience placement- ...for managing an organization.Job DescriptionPrimary Skills:Big Data experience8+ years exp in Java, Python and ScalaWith Spark and... ..., calling APIs, write SQL queries, etc.).Work closely with our engineering team to integrate and build algorithmsProcess unstructured data...
- ...Bristlecone India Limited is seeking a seasoned Lead Data Engineer to spearhead the design, development, and optimization of our data ecosystem within the Data Services – Studio department. The role demands deep expertise in Snowflake and Databricks, a proven track record...
JobCubby
San Jose, CA4 days ago - Job-ID27801708Reference26-07957Remote100% Remote Job Title: Contract Data Engineer (API & Database Focus) Role Summary: We are seeking an experienced and highly skilled Contract Data Engineer. This role is focused on building robust data pipelines, primarily involving the...Contract workRemote work
$128k - $249k
...search for talented visionaries and your search for important and impactful work lead to the same place.We are seeking a Senior Data Engineer to join the firm. The Senior Data Engineer contributes to the Enterprise Applications, Data, and AI Platforms group, designing scalable...Full timeTemporary workLocal areaRemote workFlexible hoursAfternoon shift- ...Data EngineerLocation: Mountain View CA (Onsite – Hybrid)Client: IntuitImplementation Partner: AltimetrikOverall Skills:10+ years of overall experience in data management space and at least 5 years of working in large data sets in a data lake environmentHighly proficient...
Argyle Infotech
Mountain View, CA1 day ago $185k - $230k
The OpportunityWe are looking for a Senior Data Engineer to join our Data Platform team and build the core data foundations that power analytics, experimentation, and decision-making across the company. In this role, you will design and own foundational data models, pipelines...Permanent employment- ...Data Engineer Lead We are seeking an experienced Data Engineer Lead to join our onshore team and drive the design, development, and optimization of enterprise data solutions using Microsoft Azure and Microsoft Fabric. The ideal candidate will lead data engineering...
InterSources
Palo Alto, CA4 days ago - ...Job Title Primary Responsibilities: • Expertise in SQL: You possess a deep understanding of SQL for data manipulation, querying, and performance optimization across various database systems. • Proficiency in Python: Your Python skills are crucial for building data...
Yochana
Sunnyvale, CA4 days ago $75 - $85 per hour
Position: Battery Data Engineer III Location: Cupertino, California Duration: Contract Job ID: 171364 Job Description:This will be a data engineer role for processing battery testing data to facilitate battery algorithm delivery and support battery algorithm simulations...Full timeContract work- ...OverviewWe are seeking a highly skilled Senior Data Engineer with hands-on experience in Databricks, Unity Catalog, and Azure Data Factory (ADF) to support a large-scale enterprise data modernization initiative. The ideal candidate will be responsible for designing, developing...
Publicis Media
San Jose, CA4 days ago - Position: Data Engineer IILocation: Sunnyvale, CaliforniaDuration: ContractJob ID: 175305Job Overview: We are seeking a skilled and detail-oriented Data Engineer II to join our team. The ideal candidate will be responsible for designing, building, and maintaining scalable...Full time
- ...Data EngineerMountain View, CA (Hybrid, 2 days a week onsite and day 1 onsite) Duration: 6+ months contractInterview process: 3 rounds of Zoom videoRequired skills: 10+ years of overall experience in data management space and at least 5 years of working in large data...2 days per week
Tekfortune Inc
Mountain View, CA1 day ago $55.21 - $69.01 per hour
...Description Data Engineer 1 (#2702) Full-time San Jose, CA You'll be joining Adobe on a contract opportunity, employed through NextDeavor Benefits You'll Love NextDeavor offers health, vision and dental benefits for contract employees Paid sick...Hourly payPermanent employmentFull timeContract workShift workNextDeavor
San Jose, CA16 hours ago- ...Data Engineer Location: Austin, TX/Sunnyvale, CA Job Description We are seeking an experienced Data Engineer with proven, hands-on expertise in migrating enterprise database workloads from Oracle to PostgreSQL. The ideal candidate will lead/support end-to-end...
Futran Tech Solutions Pvt. Ltd.
Sunnyvale, CA3 days ago - ...Data EngineerRole: Data Engineer Location: Palo Alto, CA Duration: Long TermJob Description: We are looking for experienced developer resources with the below key skillset.Python with oops concept exposure.Implementation knowledge covering SQL and Teradata. – Few years...
Rootshell Inc
Palo Alto, CA1 day ago $201.3k - $352.3k
...Company Description It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried... .... Our ServiceNow AI platform brings together any AI, any data, and any workflow— helping 85% of the Fortune 500® work smarter,...Permanent employmentWork experience placementWork at officeImmediate startRemote workFlexible hours2 days per weekServiceNow
Santa Clara, CA1 day ago- ...Data EngineerApex Systems has a client with an immediate opening for a fully remote Data Engineer for a 6+ Month Contract. For this role, our client is looking for someone skilled in GCP and Grafana.Position: Data EngineerLocation: 100% RemoteLength: 6+ Month ContractJob...Contract workImmediate startRemote work
Software Technology Inc
Santa Clara, CA1 day ago - Company: Tredence Inc. Position Title: Senior Data Engineer Location: 1900 Camden Ave., Suite 66, San Jose, CA 95124 Position Responsibilities: Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders...Remote work
- Job-ID17032030Reference21-063692 Data Engineer / Architect Client doesn't have much bandwidth for any handholding. They will grant required access and the candidate should work on their own with minimal Apple support. Hence, they need senior level consultant / Architect...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
Related searches