Data Engineer
Matchpoint Solutions
Job Title: Data Engineer
Location: [Your Location or "Remote"]
About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Databricks and PySpark. The ideal candidate will also have hands-on experience with Airflow for orchestration and Kafka for streaming data pipelines. You will play a key role in building, optimizing, and maintaining our data infrastructure to support data-driven decision-making across the organization.
Responsibilities:
- Design, build, and maintain scalable data pipelines using Databricks and PySpark.
- Develop and manage ETL workflows and data integration processes.
- Implement data orchestration and workflow automation using Apache Airflow.
- Build real-time and batch data streaming solutions leveraging Apache Kafka.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
- Optimize data workflows for performance, scalability, and reliability.
- Ensure data quality, integrity, and security across all pipelines.
- Monitor and troubleshoot data pipelines, ensuring minimal downtime and data loss.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- 3+ years of experience in data engineering or a similar role.
- Strong hands-on experience with Databricks and PySpark.
- Proven experience in building and maintaining data pipelines with Apache Airflow.
- Solid understanding of Apache Kafka, including producing and consuming data streams.
- Proficiency in SQL and working with large-scale data processing.
- Experience with cloud platforms (AWS, Azure, or GCP) is a plus.
- Familiarity with data modeling, warehousing, and governance best practices.
- Excellent problem-solving and communication skills.
Nice to Have:
- Experience with Delta Lake, Spark Streaming, or other big data technologies.
- Knowledge of CI/CD tools and DevOps practices for data workflows.
- Exposure to monitoring tools like Datadog, Grafana, or similar.
Location: [Your Location or "Remote"]
About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Databricks and PySpark. The ideal candidate will also have hands-on experience with Airflow for orchestration and Kafka for streaming data pipelines. You will play a key role in building, optimizing, and maintaining our data infrastructure to support data-driven decision-making across the organization.
Responsibilities:
- Design, build, and maintain scalable data pipelines using Databricks and PySpark.
- Develop and manage ETL workflows and data integration processes.
- Implement data orchestration and workflow automation using Apache Airflow.
- Build real-time and batch data streaming solutions leveraging Apache Kafka.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
- Optimize data workflows for performance, scalability, and reliability.
- Ensure data quality, integrity, and security across all pipelines.
- Monitor and troubleshoot data pipelines, ensuring minimal downtime and data loss.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- 3+ years of experience in data engineering or a similar role.
- Strong hands-on experience with Databricks and PySpark.
- Proven experience in building and maintaining data pipelines with Apache Airflow.
- Solid understanding of Apache Kafka, including producing and consuming data streams.
- Proficiency in SQL and working with large-scale data processing.
- Experience with cloud platforms (AWS, Azure, or GCP) is a plus.
- Familiarity with data modeling, warehousing, and governance best practices.
- Excellent problem-solving and communication skills.
Nice to Have:
- Experience with Delta Lake, Spark Streaming, or other big data technologies.
- Knowledge of CI/CD tools and DevOps practices for data workflows.
- Exposure to monitoring tools like Datadog, Grafana, or similar.
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Data Engineer in Milpitas, CA vacancy
- ...Job Description Launch Your Data Career with Proof, Not Promises-At SynergisticIT If you're serious about starting a high-impact, high-paying career in Data Science, Analytics, or Engineering , it's time to stop guessing and start choosing results. At SynergisticIT...Suggested
SynergisticIT
Alviso, CA1 day ago - ...Data Engineer We are seeking a highly skilled Data Engineer with strong expertise in Databricks and PySpark. The ideal candidate will also have hands-on experience with Airflow for orchestration and Kafka for streaming data pipelines. You will play a key role in building...Suggested
Match Point
Milpitas, CA2 days ago - ...Data & Analytics Engineer Our Information Services (ISS) department is looking for an outstanding Data & Analytics Engineer with 5+ years of experience working with data and is passionate about technology & architecture, loves hands-on development and wants to help...SuggestedLocal area
Netpace
Milpitas, CA2 days ago - Job Description Gather data from multiple sources including databases, spreadsheets, and external sources. Clean and preprocess data... ...& Requirements Require Master of Science in Computer Science/Engineering, Information Systems/Technology, or related field and 2 years experience...Suggested
- Payactiv, a FinTech company, seeks a Data and Analytics Engineer to own and support reporting, analytics, dashboards, and data requests. You will manage Snowflake-based workflows, ensure data accuracy, and translate business questions into outputs for stakeholder use. You...Suggested
$130k - $165k
...finances, and regain flexibility to pay for things on their own schedule. What you will do… Payactiv is seeking a hands-on Data and Analytics Engineer to own and support our business reporting, analytics, dashboarding, and marketing/operational data request workflows....Work at office- Payactiv, Inc. is seeking a hands-on Data and Analytics Engineer in Milpitas, CA to deliver reliable business reporting, dashboards, and data workflows. You will write SQL for Snowflake, build curated datasets, and partner with stakeholders to define data solutions for...
$75 - $85 per hour
Position: Battery Data Engineer III Location: Cupertino, California Duration: Contract Job ID: 171364 Job Description:This will be a data engineer role for processing battery testing data to facilitate battery algorithm delivery and support battery algorithm simulations...Full timeContract work$100.79k - $160.26k
...DLA Piper is a place you can engage in meaningful work and grow your career. Let’s see what we can achieve. Together.SummaryThe Data Engineer, Solutions & Data role designs, builds, and operates data pipelines and data integration processes that translate raw data into...Full timeWork at officeRemote workVisa sponsorship$75 - $80 per hour
DescriptionResponsibilitiesArchitect, construct, and maintain robust, scalable data pipelines and complex ETL workflows.Manage and optimize data... ....Minimum Qualifications3-5 years of experience in Data Engineering.Bachelor’s degree in Computer Science, Engineering, or...Contract workTemporary workWork experience placement- Position: Data Engineer IILocation: Sunnyvale, CaliforniaDuration: ContractJob ID: 175305Job Overview: We are seeking a skilled and detail-oriented Data Engineer II to join our team. The ideal candidate will be responsible for designing, building, and maintaining scalable...Full time
$114k - $180k
What this Job Entails: We are looking for a Data Engineer to join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional...Full timeTemporary workWork at officeRemote work- ...roleGridmatic’s business is complex, and we need to build a robust data layer to be able to answer core questions around profit and loss... ...our renewable supply.We’re looking for a startup-minded data engineer who can wear a lot of hats, work with multiple teams, and build...Work at officeHome officeFlexible hours3 days per week
- ...for managing an organization.Job DescriptionPrimary Skills:Big Data experience8+ years exp in Java, Python and ScalaWith Spark and... ..., calling APIs, write SQL queries, etc.).Work closely with our engineering team to integrate and build algorithmsProcess unstructured data...
- ..., collaborate, and communicate across media. Our team includes engineers, artists, and product thinkers, all passionate about building tools... ...and launching products together.About the RoleWe are seeking a Data Engineer to design, build, and scale the data infrastructure...Work at office
- ...About the job Lead Data Engineer Role: Lead Data Engineer Location: Fremont, CA(3 days onsite, 2 days work from home) Duration: Full Time Job Description: ~ Candidate should Provide technical expertise in needs identification, data modelling, data...Full timeWork from home
Inizio Partners
Fremont, CA13 hours ago - ...Lead Data Engineer Data team builds and operates the data marts and pipelines that power HR/Workforce reporting and analytics across Intuit. We're looking for a hands-on Data Engineer who can operate independently. High-visibility, high-ownership — direct exposure to...
United IT
Mountain View, CA2 days ago - ...Senior Lead Data Engineer Location: San Jose, CA 95125 Hybrid(Locals) Key Responsibilities: Architect, construct, test and maintain ELT pipelines using Hadoop and Spark SQL Manage and optimize the infrastructure ensuring reliability and performance Design...Local area
Argyle Infotech
San Jose, CA2 days ago $139k - $257.55k
...The Opportunity The Marketing Operations and Engineering team in Growth Marketing and Insights (GMI) handles data architecture, transformation, and engineering for GMI under the Marketing org. We manage backend data for marketing measurement tools and processes such...Temporary workCasual workLocal areaWorldwideAdobe
San Jose, CA1 day ago- ...we are looking for entry-level software programmers, Java full stack developers, Python/Java developers, data analysts/data scientists, machine learning engineers for full time positions with clients. Who Should Apply Recent computer science/engineering/mathematics...Full timeH1bRemote work
SynergisticIT
Palo Alto, CA5 days ago - Job ID: 25659978Reference Number: 25-00597Title: Data Analyst EngineerPosted Date: 2025-06-06Company: HAN Staffing Position : Data Analyst Engineer Location : Jersey City, Wilmington, Chicago, Plano, Seattle, Palo Alto. Contract : w2 Job Description: Experience : 7+ Experience...Contract work
$180k - $258.75k
...is harmonious with TRI’s robotics divisions' efforts in Diffusion Policy and Large Behavior Models.We are looking for a Senior Data Engineer to design and build the foundational data infrastructure and tools that power our autonomy research and development workflows. This...Full timeLocal areaShift work$123k - $215.25k
...StatesSalary: $123000 - $215250 annually + bonus + benefitsJob Function: Engineering & ArchitectureSchedule: Full timeShift: DayWorkplace:... ...powered by our technology, our culture, and our colleagues.Senior Data Engineer - AI & Intelligent Payment PlatformsAre you ready to...Visa sponsorship- Company descriptionOverviewWe are looking for a Senior Data Engineer with strong hands-on experience designing, building, and maintaining modern data platforms using Snowflake and Google Cloud Platform (GCP). This role will focus on data migration, data integration, and...
- Company: Tredence Inc. Position Title: Senior Data Engineer Location: 1900 Camden Ave., Suite 66, San Jose, CA 95124 Position Responsibilities: Manage end to end delivery by Investigating problem areas, working cross-functionally with product manager & other stakeholders...Remote work
$128k - $249k
...search for talented visionaries and your search for important and impactful work lead to the same place.We are seeking a Senior Data Engineer to join the firm. The Senior Data Engineer contributes to the Enterprise Applications, Data, and AI Platforms group, designing scalable...Full timeTemporary workLocal areaRemote workFlexible hoursAfternoon shift$130k - $200k
Santa Clara, CAData Engineering - Data Infrastructure /Full-time /HybridPlusAI is a Physical AI company pioneering AI-based virtual driver software for factory-built autonomous trucks. Headquartered in Silicon Valley with operations in the United States and Europe, Plus...Full time$175k - $221k
...feed ranking decision, every recommendation, every A/B test, and every ML model we ship runs on the data infrastructure you'd own. We're looking for a senior engineer to build and scale the batch and streaming pipelines at the core of that platform — someone who takes...Full timeLocal areaWork from home$185k - $230k
The OpportunityWe are looking for a Senior Data Engineer to join our Data Platform team and build the core data foundations that power analytics, experimentation, and decision-making across the company. In this role, you will design and own foundational data models, pipelines...Permanent employment- ...Job Title Primary Responsibilities: • Expertise in SQL: You possess a deep understanding of SQL for data manipulation, querying, and performance optimization across various database systems. • Proficiency in Python: Your Python skills are crucial for building data...
Omni Inclusive
Sunnyvale, CA5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
Related searches
- finance data engineer Milpitas, CA
- data engineer machine learning Milpitas, CA
- data center engineer Milpitas, CA
- data engineer analytics Milpitas, CA
- data developer Milpitas, CA
- senior data center engineer Milpitas, CA
- data science developer Milpitas, CA
- senior cloud data engineer Milpitas, CA
- data engineer Milpitas, CA
- data engineering intern summer Milpitas, CA