Senior Data Engineer / Data Platform Engineer
West Coast Consulting LLC
Job Description Position: Senior Data Engineer / Data Platform Engineer We are looking for a Senior Data Engineer with strong experience building and supporting large-scale data platforms, distributed data pipelines, and real-time data processing systems. The ideal candidate should be hands-on with Spark, Kafka, Databricks, Airflow, and cloud data technologies and comfortable working with high-volume enterprise data environments. Key Responsibilities Design, develop, and maintain scalable batch and real-time data pipelines.
Build distributed data processing solutions using Apache Spark and Spark Structured Streaming.
Develop streaming and event-driven data pipelines using Apache Kafka/Kafka Connect.
Build and optimize ETL/ELT workflows using Python, Java, and SQL.
Work with Databricks and modern lakehouse technologies such as Delta Lake or Apache Hudi.
Develop and manage data workflows using Apache Airflow.
Design data models, partitioning strategies, schema evolution, and data quality frameworks.
Monitor pipeline performance, data freshness, reliability, and operational health.
Work with cloud platforms such as Google Cloud Platform or Azure and technologies including BigQuery, Cloud Storage, Dataflow, Kubernetes, or similar.
Collaborate with analytics, ML, product, and infrastructure teams to build scalable data solutions.
Participate in architecture discussions, code reviews, troubleshooting, and performance optimization.
Implement CI/CD, automated testing, and engineering best practices for data platforms. Required Skills 8+ years of experience in Data Engineering or related roles.
Strong hands-on experience with Apache Spark / Spark Structured Streaming.
Strong experience with Kafka and real-time/event-driven data pipelines.
Strong experience with Databricks.
Strong experience with Python and SQL; Java or Scala is highly preferred.
Experience with Apache Airflow or similar workflow orchestration tools.
Experience building large-scale ETL/ELT pipelines.
Experience with cloud data platforms, preferably Google Cloud Platform or Azure.
Experience with data lakes/lakehouse architectures.
Strong understanding of distributed systems, data modeling, performance tuning, and data quality.
Experience with Git and CI/CD tools such as Jenkins, GitHub Actions, or similar. Preferred Experience with Delta Lake or Apache Hudi.
Experience with BigQuery, GCS, Dataflow, Cassandra, Presto/Trino, or similar technologies.
Experience with Kubernetes and Docker.
Experience with streaming data and CDC.
Experience working with ML/analytics data platforms.
Experience establishing data contracts, SLAs, monitoring, and observability.
Enterprise-scale data platform experience with very large datasets.
Strong communication and stakeholder-management skills.
Build distributed data processing solutions using Apache Spark and Spark Structured Streaming.
Develop streaming and event-driven data pipelines using Apache Kafka/Kafka Connect.
Build and optimize ETL/ELT workflows using Python, Java, and SQL.
Work with Databricks and modern lakehouse technologies such as Delta Lake or Apache Hudi.
Develop and manage data workflows using Apache Airflow.
Design data models, partitioning strategies, schema evolution, and data quality frameworks.
Monitor pipeline performance, data freshness, reliability, and operational health.
Work with cloud platforms such as Google Cloud Platform or Azure and technologies including BigQuery, Cloud Storage, Dataflow, Kubernetes, or similar.
Collaborate with analytics, ML, product, and infrastructure teams to build scalable data solutions.
Participate in architecture discussions, code reviews, troubleshooting, and performance optimization.
Implement CI/CD, automated testing, and engineering best practices for data platforms. Required Skills 8+ years of experience in Data Engineering or related roles.
Strong hands-on experience with Apache Spark / Spark Structured Streaming.
Strong experience with Kafka and real-time/event-driven data pipelines.
Strong experience with Databricks.
Strong experience with Python and SQL; Java or Scala is highly preferred.
Experience with Apache Airflow or similar workflow orchestration tools.
Experience building large-scale ETL/ELT pipelines.
Experience with cloud data platforms, preferably Google Cloud Platform or Azure.
Experience with data lakes/lakehouse architectures.
Strong understanding of distributed systems, data modeling, performance tuning, and data quality.
Experience with Git and CI/CD tools such as Jenkins, GitHub Actions, or similar. Preferred Experience with Delta Lake or Apache Hudi.
Experience with BigQuery, GCS, Dataflow, Cassandra, Presto/Trino, or similar technologies.
Experience with Kubernetes and Docker.
Experience with streaming data and CDC.
Experience working with ML/analytics data platforms.
Experience establishing data contracts, SLAs, monitoring, and observability.
Enterprise-scale data platform experience with very large datasets.
Strong communication and stakeholder-management skills.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer / Data Platform Engineer in United States vacancy
$180k - $250k
...You build the data infrastructure that powers trading across every major global market. This is a firm with over three... ...opportunity shows up. You join a team building a unified data platform that serves engineers across the whole organization. You design, deploy, and...SeniorWorldwide- ...Description: Required Core Skills: Python Engineering: 5+ years of hands-on experience... ..., production-grade Python applications, data pipelines, and processing frameworks.... ...plan tuning. Airflow / Google Cloud Platform Cloud Composer: Deep expertise in developing...Senior
- ...enterprise transformation across cloud, data, software engineering, and artificial intelligence . We... .... We are looking for a Senior / Staff Data Engineer to build what... ...cost implications. Pair with client platform owners in weeks 9–10 and hand over on...SeniorFull time
$180k - $220k
...Senior Data Engineer, Platform Salary range: $180,000 - $220,000 per year About the Role Two kinds of data decide whether we can do our job. What a health plan tells us happened to a member, in eligibility files, claims, pharmacy and hospital event feeds. And...SeniorFull timeFor contractorsLive in$160k - $175k
...listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer, Data Transformation Platform based in the United States. This is an opportunity to build and evolve the data infrastructure powering...SeniorFull timeHome office$180.37k - $212.2k
...the best of us, every day, as we build the emerging onchain platform — and with it, the future global financial system. To achieve... ...Team The Finance Analytics team bridges the gap between data engineering, data science, and business analytics by building scalable, impactful...SeniorWork at office- ...Job Description Job Description Title: Senior Data Engineer / ML Platform Engineer (Azure Databricks & GenAI) Location: position will sit on-site in either Warren, MI or Austin, TX 4 days per week Please only submit candidates that are able to work on-site...SeniorFull timeContract workFor contractors
- ManpowerGroup is seeking a Senior Snowflake Data Engineer in New Providence, NJ to join the Data Platform and Analytics initiatives within the Engineering organization. You will own end-to-end Snowflake data solutions, collaborating with engineering teams to deliver reliable...Senior
$164k - $295k
...redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing... ...About the Role: We're seeking an experienced Sr. Platform Data Engineer to architect, deploy, and operate the core data infrastructure...SeniorPermanent employmentWork experience placementWork at officeLocal areaRemote work- ...Senior Data Engineer - GenAI & Cloud Data Platforms: Senior Data Engineer with Generative AI expertise to design, build, and optimize scalable cloud-native data platforms and AI-powered data solutions. The ideal candidate will have strong experience in Python, AWS...SeniorFull time
- ...technology landscape, an organization’s data has never been a more important aspect in... ...goal, they are leveraging visual analytics platforms to analyze, visualize, and share... ...Contributions We are looking for seasoned Data Engineer to work with our team and our clients to...Senior
- Eliassen Group in Rockville, MD is seeking a hands-on Senior Data Engineer to design, build, and maintain an analytics platform that integrates data from Jira, Bitbucket, GitLab, Jenkins, and Splunk. You will lead data pipelines, quality remediation, and metrics reporting...Senior
- ...we operate at the layer where data becomes decisions, and... ...in-office in one of our core engineering locations: El Segundo, CA; Austin... ...talk to each other. The Data Platform team connects those sources,... ...AI agents. We're hiring a Senior Data Engineer to build that platform...SeniorFull timeWork at office
- A leading data solutions company is seeking a candidate to join their Managed Services team. This remote position focuses on maintaining... .... Responsibilities include leading cloud-native data platform implementations, optimizing performance, and ensuring robust security...SeniorRemote work
- We are seeking a driven and highly skilled Senior Data Platform/DevOps Engineer to join a leading company in the oil and gas industry. In this role, you will work alongside Software Engineers, Machine Learning Engineers, and Data Engineers to understand their technical...Senior
$159.6k - $239.4k
...a Fortune 500 company and a leading AI platform for managing people, money, and agents,... ...formed, forward-looking Cybersecurity Data Engineering & Enablement Team driving the future of... ...About the RoleWe are seeking a powerhouse Senior Data Platform Engineer to serve as our...SeniorFull timeWork at officeRemote workHome officeFlexible hours$160k - $200k
...builds, manufactures, and operates autonomous delivery systems that move essential goods where they are needed. As a Senior Data Engineer on the Data Platform team, you will build cloud data foundations that turn application and operational telemetry into trusted inputs...SeniorFull timeLocal area$160k - $250k
...willingness to challenge consensus. The Data Ecosystem team is an important part of DRW’s Unified Platform (UP) organization, building the platform capabilities... ...delivering data across DRW. As a Senior Data Engineer in our Unified Platform organization, you will...SeniorTemporary workFlexible hours$110k - $130k
...communication.We are seeking a highly motivated and experienced Senior Data Engineer to join our IT Data Engineering team and support the... ...APIs for data ingestion and consumptionDesign and implement platform automation and standardized, repeatable deployment mechanisms...SeniorWork at officeLocal areaWorldwide- ...Seeking a full-time remote Senior Data Platform Engineer, this role will manage the foundation of data-driven products for over 5,000 gyms, ensuring fast, trustworthy data, privacy, and security while collaborating with cross-functional teams to enhance AI capabilities...SeniorFull timeRemote work
- ...we are continuing to grow our Digital & Data Hub in Krakow, delivering digital, data... ....We are looking for an experienced Senior Data Engineer to design, build, and operate enterprise... ...strong focus on on-premises and hybrid data platforms. This is a hands-on engineering role...SeniorFull timeContract workWork at officeWork from home
$88.7k - $139.26k
....Your Role at Delta Faucet:Position SummaryThe Senior Analytics Engineer owns the layer between curated enterprise data and business consumption. Working primarily in... ...certification evidence, and acceptable quality.IT and data platform teams: define curated input contracts and...SeniorFull timeLocal area$240k - $335k
About the RoleWe're looking for a Senior Staff Data Engineer to be the technical backbone of our Data & ML Platform team — the foundation powering analytics, product experiences, and machine learning across Hinge Health. This is a high-ownership IC role for someone who...SeniorWork at officeLocal areaImmediate startRemote workWorldwide3 days per week$500 per month
...team is a diverse group of experienced engineers, traders, and brokerage professionals who... ...apply. About the team & role The Data Platform team builds and operates the shared... ...customers and launches new products. As a Senior Data Platform Engineer , you'll design...SeniorRemote workHome office$180.6k - $289.3k
...the world. Progress starts with you.Job DescriptionVisa's Data Platforms organization is responsible for the foundational data ecosystem... ..., analytics, and business insights across Visa.As a Senior Data Engineer, you will help design and build platforms that process and...SeniorFull timeWork experience placementWork at officeLocal area3 days per week$101.97k - $203.94k
...helping to simplify health care one person, one family and one community at a time. Position Summary We are looking for a Senior Data Platform Engineer to help build, run, and evolve our Azure Databricks data platform, which ingests, validates, and protects data for the...SeniorHourly payPermanent employmentFull timeTemporary workLocal areaRemote workShift work$149k - $198.5k
...Mission Summary: The Data Platform team is the vital link between our company's expansive data and the engineering teams that rely on it. We architect and operate the core services and infrastructure necessary to transform petabytes of raw data each month into actionable...SeniorFull timeWork at officeRemote work$165k - $200k
...artificial intelligence, mathematical optimization, data analytics, and software engineering for resilient optimization of complex systems. In... ...role was built for you. About the Role As a Senior Data Platform Engineer, you’ll own the durable view of every...SeniorFull timeContract workRemote workFlexible hours$112k - $179k
...Responsibilities Peraton is looking for a Data Architecture, Lead Associate. This... ...and extends a greenfield data and AI platform supporting federal background... ...work. Working alongside the platform's lead engineer, you'll develop and maintain the pipelines...SeniorFull timeContract workTemporary workRemote workShift work$135k - $155k
...Join this diverse team that thrives on innovation and making a real-world impact. This role sits at the intersection of platform engineering, data engineering, and data product development within financial services applications. Leading with platform engineering - and...SeniorFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer / Data Platform Engineer. Be the first to apply!
Related searches
- staff data engineer United States
- junior data engineer United States
- bi data engineer United States
- senior cloud data engineer United States
- IT data engineer United States
- big data cloud engineer United States
- junior big data engineer United States
- data engineering intern summer United States
- data engineer analytics United States
- sr data engineer United States






