Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Data Engineer / Data Platform Engineer

West Coast Consulting LLC

Job Description

Position: Senior Data Engineer / Data Platform Engineer

We are looking for a Senior Data Engineer with strong experience building and supporting large-scale data platforms, distributed data pipelines, and real-time data processing systems. The ideal candidate should be hands-on with Spark, Kafka, Databricks, Airflow, and cloud data technologies and comfortable working with high-volume enterprise data environments.

Key Responsibilities

Design, develop, and maintain scalable batch and real-time data pipelines.
Build distributed data processing solutions using Apache Spark and Spark Structured Streaming.
Develop streaming and event-driven data pipelines using Apache Kafka/Kafka Connect.
Build and optimize ETL/ELT workflows using Python, Java, and SQL.
Work with Databricks and modern lakehouse technologies such as Delta Lake or Apache Hudi.
Develop and manage data workflows using Apache Airflow.
Design data models, partitioning strategies, schema evolution, and data quality frameworks.
Monitor pipeline performance, data freshness, reliability, and operational health.
Work with cloud platforms such as Google Cloud Platform or Azure and technologies including BigQuery, Cloud Storage, Dataflow, Kubernetes, or similar.
Collaborate with analytics, ML, product, and infrastructure teams to build scalable data solutions.
Participate in architecture discussions, code reviews, troubleshooting, and performance optimization.
Implement CI/CD, automated testing, and engineering best practices for data platforms.

Required Skills

8+ years of experience in Data Engineering or related roles.
Strong hands-on experience with Apache Spark / Spark Structured Streaming.
Strong experience with Kafka and real-time/event-driven data pipelines.
Strong experience with Databricks.
Strong experience with Python and SQL; Java or Scala is highly preferred.
Experience with Apache Airflow or similar workflow orchestration tools.
Experience building large-scale ETL/ELT pipelines.
Experience with cloud data platforms, preferably Google Cloud Platform or Azure.
Experience with data lakes/lakehouse architectures.
Strong understanding of distributed systems, data modeling, performance tuning, and data quality.
Experience with Git and CI/CD tools such as Jenkins, GitHub Actions, or similar.

Preferred

Experience with Delta Lake or Apache Hudi.
Experience with BigQuery, GCS, Dataflow, Cassandra, Presto/Trino, or similar technologies.
Experience with Kubernetes and Docker.
Experience with streaming data and CDC.
Experience working with ML/analytics data platforms.
Experience establishing data contracts, SLAs, monitoring, and observability.
Enterprise-scale data platform experience with very large datasets.
Strong communication and stakeholder-management skills.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer / Data Platform Engineer in United States vacancy
  • $180k - $250k

     ...You build the data infrastructure that powers trading across every major global market. This is a firm with over three...  ...opportunity shows up. You join a team building a unified data platform that serves engineers across the whole organization. You design, deploy, and... 
    Senior
    Worldwide

    Saragossa

    Chicago, IL
    10 days ago
  •  ...Description: Required Core Skills: Python Engineering: 5+ years of hands-on experience...  ..., production-grade Python applications, data pipelines, and processing frameworks....  ...plan tuning. Airflow / Google Cloud Platform Cloud Composer: Deep expertise in developing... 
    Senior

    Omega Hires

    Phoenix, AZ
    1 day ago
  •  ...enterprise transformation across cloud, data, software engineering, and artificial intelligence . We...  .... We are looking for a Senior / Staff Data Engineer to build what...  ...cost implications. Pair with client platform owners in weeks 9–10 and hand over on... 
    Senior
    Full time

    Solvd

    Remote
    3 days ago
  • $180k - $220k

     ...Senior Data Engineer, Platform Salary range: $180,000 - $220,000 per year About the Role Two kinds of data decide whether we can do our job. What a health plan tells us happened to a member, in eligibility files, claims, pharmacy and hospital event feeds. And... 
    Senior
    Full time
    For contractors
    Live in

    Vanna Health, Inc

    Remote
    13 days ago
  • $160k - $175k

     ...listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer, Data Transformation Platform based in the United States. This is an opportunity to build and evolve the data infrastructure powering... 
    Senior
    Full time
    Home office

    Jobgether

    Remote
    6 days ago
  • $180.37k - $212.2k

     ...the best of us, every day, as we build the emerging onchain platform — and with it, the future global financial system. To achieve...  ...Team The Finance Analytics team bridges the gap between data engineering, data science, and business analytics by building scalable, impactful... 
    Senior
    Work at office

    Unchain Data

    New York, NY
    3 days ago
  •  ...Job Description Job Description Title: Senior Data Engineer / ML Platform Engineer (Azure Databricks & GenAI) Location: position will sit on-site in either Warren, MI or Austin, TX 4 days per week Please only submit candidates that are able to work on-site... 
    Senior
    Full time
    Contract work
    For contractors

    Expert In Recruitment Solutions

    Glen Allen, VA
    20 days ago
  • ManpowerGroup is seeking a Senior Snowflake Data Engineer in New Providence, NJ to join the Data Platform and Analytics initiatives within the Engineering organization. You will own end-to-end Snowflake data solutions, collaborating with engineering teams to deliver reliable... 
    Senior

    Experis-US

    New Providence, NJ
    4 days ago
  • $164k - $295k

     ...redefined modern security with the world’s most advanced AI-native platform. We work on large scale distributed systems, processing...  ...About the Role: We're seeking an experienced Sr. Platform Data Engineer to architect, deploy, and operate the core data infrastructure... 
    Senior
    Permanent employment
    Work experience placement
    Work at office
    Local area
    Remote work
    Remote
    a month ago
  •  ...Senior Data Engineer - GenAI & Cloud Data Platforms: Senior Data Engineer with Generative AI expertise to design, build, and optimize scalable cloud-native data platforms and AI-powered data solutions. The ideal candidate will have strong experience in Python, AWS... 
    Senior
    Full time

    Purple Drive

    Boston, MA
    11 days ago
  •  ...technology landscape, an organization’s data has never been a more important aspect in...  ...goal, they are leveraging visual analytics platforms to analyze, visualize, and share...  ...Contributions We are looking for seasoned Data Engineer to work with our team and our clients to... 
    Senior

    Steampunk, Inc.

    Mc Lean, VA
    4 days ago
  • Eliassen Group in Rockville, MD is seeking a hands-on Senior Data Engineer to design, build, and maintain an analytics platform that integrates data from Jira, Bitbucket, GitLab, Jenkins, and Splunk. You will lead data pipelines, quality remediation, and metrics reporting... 
    Senior

    Eliassen Group

    Rockville, MD
    4 days ago
  •  ...we operate at the layer where data becomes decisions, and...  ...in-office in one of our core engineering locations: El Segundo, CA; Austin...  ...talk to each other. The Data Platform team connects those sources,...  ...AI agents. We're hiring a Senior Data Engineer to build that platform... 
    Senior
    Full time
    Work at office

    gallatin

    El Segundo, CA
    3 days ago
  • A leading data solutions company is seeking a candidate to join their Managed Services team. This remote position focuses on maintaining...  .... Responsibilities include leading cloud-native data platform implementations, optimizing performance, and ensuring robust security... 
    Senior
    Remote work

    Jobot

    Austin, TX
    1 day ago
  • We are seeking a driven and highly skilled Senior Data Platform/DevOps Engineer to join a leading company in the oil and gas industry. In this role, you will work alongside Software Engineers, Machine Learning Engineers, and Data Engineers to understand their technical... 
    Senior

    EPAM Systems

    United States
    2 days ago
  • $159.6k - $239.4k

     ...a Fortune 500 company and a leading AI platform for managing people, money, and agents,...  ...formed, forward-looking Cybersecurity Data Engineering & Enablement Team driving the future of...  ...About the RoleWe are seeking a powerhouse Senior Data Platform Engineer to serve as our... 
    Senior
    Full time
    Work at office
    Remote work
    Home office
    Flexible hours

    Workday

    Boulder, CO
    4 days ago
  • $160k - $200k

     ...builds, manufactures, and operates autonomous delivery systems that move essential goods where they are needed. As a Senior Data Engineer on the Data Platform team, you will build cloud data foundations that turn application and operational telemetry into trusted inputs... 
    Senior
    Full time
    Local area

    Zipline

    South San Francisco, CA
    2 days ago
  • $160k - $250k

     ...willingness to challenge consensus. The Data Ecosystem team is an important part of DRW’s Unified Platform (UP) organization, building the platform capabilities...  ...delivering data across DRW. As a Senior Data Engineer in our Unified Platform organization, you will... 
    Senior
    Temporary work
    Flexible hours

    DRW

    Chicago, IL
    1 day ago
  • $110k - $130k

     ...communication.We are seeking a highly motivated and experienced Senior Data Engineer to join our IT Data Engineering team and support the...  ...APIs for data ingestion and consumptionDesign and implement platform automation and standardized, repeatable deployment mechanisms... 
    Senior
    Work at office
    Local area
    Worldwide

    IDEXX Laboratories

    Boston, MA
    1 day ago
  •  ...Seeking a full-time remote Senior Data Platform Engineer, this role will manage the foundation of data-driven products for over 5,000 gyms, ensuring fast, trustworthy data, privacy, and security while collaborating with cross-functional teams to enhance AI capabilities... 
    Senior
    Full time
    Remote work

    Virtual Vocations Inc

    United States
    3 days ago
  •  ...we are continuing to grow our Digital & Data Hub in Krakow, delivering digital, data...  ....We are looking for an experienced Senior Data Engineer to design, build, and operate enterprise...  ...strong focus on on-premises and hybrid data platforms. This is a hands-on engineering role... 
    Senior
    Full time
    Contract work
    Work at office
    Work from home

    Jacobs

    Krakow, Shawano County, WI
    9 hours ago
  • $88.7k - $139.26k

     ....Your Role at Delta Faucet:Position SummaryThe Senior Analytics Engineer owns the layer between curated enterprise data and business consumption. Working primarily in...  ...certification evidence, and acceptable quality.IT and data platform teams: define curated input contracts and... 
    Senior
    Full time
    Local area

    Masco

    Indianapolis, IN
    9 hours ago
  • $240k - $335k

    About the RoleWe're looking for a Senior Staff Data Engineer to be the technical backbone of our Data & ML Platform team — the foundation powering analytics, product experiences, and machine learning across Hinge Health. This is a high-ownership IC role for someone who... 
    Senior
    Work at office
    Local area
    Immediate start
    Remote work
    Worldwide
    3 days per week

    Hinge Health

    San Francisco, CA
    3 days ago
  • $500 per month

     ...team is a diverse group of experienced engineers, traders, and brokerage professionals who...  ...apply. About the team & role The Data Platform team builds and operates the shared...  ...customers and launches new products. As a Senior Data Platform Engineer , you'll design... 
    Senior
    Remote work
    Home office

    Alpaca

    New York, NY
    7 days ago
  • $180.6k - $289.3k

     ...the world. Progress starts with you.Job DescriptionVisa's Data Platforms organization is responsible for the foundational data ecosystem...  ..., analytics, and business insights across Visa.As a Senior Data Engineer, you will help design and build platforms that process and... 
    Senior
    Full time
    Work experience placement
    Work at office
    Local area
    3 days per week

    Visa

    San Mateo, CA
    3 days ago
  • $101.97k - $203.94k

     ...helping to simplify health care one person, one family and one community at a time. Position Summary We are looking for a Senior Data Platform Engineer to help build, run, and evolve our Azure Databricks data platform, which ingests, validates, and protects data for the... 
    Senior
    Hourly pay
    Permanent employment
    Full time
    Temporary work
    Local area
    Remote work
    Shift work

    CVS Health

    Arizona
    19 days ago
  • $149k - $198.5k

     ...Mission Summary: The Data Platform team is the vital link between our company's expansive data and the engineering teams that rely on it. We architect and operate the core services and infrastructure necessary to transform petabytes of raw data each month into actionable... 
    Senior
    Full time
    Work at office
    Remote work

    Motional

    Remote
    19 days ago
  • $165k - $200k

     ...artificial intelligence, mathematical optimization, data analytics, and software engineering for resilient optimization of complex systems. In...  ...role was built for you. About the Role As a Senior Data Platform Engineer, you’ll own the durable view of every... 
    Senior
    Full time
    Contract work
    Remote work
    Flexible hours

    DEFCON AI

    Remote
    11 days ago
  • $112k - $179k

     ...Responsibilities Peraton is looking for a Data Architecture, Lead Associate. This...  ...and extends a greenfield data and AI platform supporting federal background...  ...work. Working alongside the platform's lead engineer, you'll develop and maintain the pipelines... 
    Senior
    Full time
    Contract work
    Temporary work
    Remote work
    Shift work

    Peraton

    Remote
    7 days ago
  • $135k - $155k

     ...Join this diverse team that thrives on innovation and making a real-world impact.   This role sits at the intersection of platform engineering, data engineering, and data product development within financial services applications. Leading with platform engineering - and... 
    Senior
    Full time

    Reply

    Seattle, WA
    15 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Data Engineer / Data Platform Engineer. Be the first to apply!