Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Data Platform Engineer, Ingestion

$170k - $236k
Full-time

Minerva

ABOUT MINERVA

Minerva builds AI for marketing leaders. We work together with marketers to automate and accelerate their workflows with AI agents. Our agents augment the end-to-end workflow of the marketer by automating analysis, campaign planning, activation, measurement and reporting.

Customers like the NBA, Ramp, Wander, Hard Rock Stadium Group and Luxury Presence use Minerva to develop a better understanding of their customers and introduce AI specifically built to understand the customer into their commercial organizations. Our campaigns outperform other tools because we built the entire stack: from identity to enrichment to data integration as well as agents to action and report on the analysis.

We've raised $20M from The General Partnership, 8VC, Lingotto, NBA Investments, Topology Ventures, Future Positive, Background Capital and many others. Our team brings together operators and investors from Citadel, Dentsu, Bridgewater, Meta Superintelligence, Growthloop, Amazon and Lazard, alongside engineers and researchers from Berkeley, MIT, Stanford and Cambridge.

THE ROLE

Every new data source starts as someone else's problem: an API with rate limits, a vendor file that arrives late and gets corrected twice, a schema that changes without warning. Today, adding a new vendor feed still takes hand-built work.

You make new sources dependable and make the next one cheap. You build the ingestion pipelines for our customer and vendor data, automate onboarding from configuration to first delivery, and turn what you build into modules other Minerva teams can use without running platform infrastructure themselves.

This is a mid-level individual contributor seat on the data platform team, reporting to our Head of Data Platform. Our data product team decides which sources matter and what good data looks like. You own how it arrives and how it stays dependable.

WHAT YOU'LL DO

  • Build pipelines for messy sources. APIs, files, database extracts and approved scraping sources, at high volume, with inconsistent formats and schemas that change.

  • Make reruns safe. Incremental loads, backfills and retries that never duplicate or corrupt data. Bad records get quarantined, not loaded. Throughput and processing cost stay under control.

  • Automate onboarding. The path from source configuration and access checks through provisioning, validation, deployment and first delivery runs without hand-holding.

  • Build modules other teams use. Reusable connectors and storage components with tests, examples and clear interfaces, so a team building an API integration or a scraper gets managed ingestion, storage and monitoring without building them.

  • Keep the origin of every value. Carry source metadata through the pipeline (which vendor, which delivery, when) so we can always say where a value came from. Enforce privacy controls so restricted data can't come back through a new load or a replay.

  • Own your integrations in production. Connect each one to shared monitoring, write its recovery runbook and fix source-specific problems when they break.

OUR STACK

  • Orchestration: Dagster, with Airflow still running the workloads that haven't moved.

  • Transformation: dbt and SQLMesh.

  • Storage: Snowflake, plus a lakehouse on Spark, Iceberg, Trino and AWS Glue.

  • Code: Python on AWS.

WHAT YOU HAVE

  • Typically three or more years building and supporting production data pipelines, or equivalent ownership.

  • You've brought in data from many vendors and source types, including messy data at scale, and kept it running after launch.

  • Practical judgment about incremental processing, backfills, failures and cost. You can show the data stayed correct as volume grew.

  • You've automated repeated work or built components other engineers actually used.

  • Strong Python and SQL.

  • You pin down what a source owner actually needs, and you write down what you will and won't support.

  • You're in New York City or willing to relocate.

NICE TO HAVES

  • Connector frameworks, whether off the shelf or your own.

  • Onboarding automation or self-service data tooling.

  • Consumer, identity or contact data, and the privacy rules that come with it.

  • Dagster, Iceberg or Snowflake in production.

YOU MIGHT BE A FIT IF YOU ARE…

  • A data engineer at a data vendor or identity company who has onboarded source after source and wants to build the framework instead of the next one-off.

  • An engineer at a connector, integration or reverse-ETL company who has built connectors other people depend on.

  • A data engineer at a fintech, marketplace or adtech company who became the person every new integration ran through.

  • Or you don't fit any of these neatly, but you've taken data you didn't control and made it dependable. Apply.

WHAT SUCCESS LOOKS LIKE

  • First 90 days: a new source is live through automated onboarding you built, and another engineer has onboarded a different source with the same modules. You've measured the setup effort and tested recovery.

  • First 6 months: teams outside data platform bring in new sources through your modules, and every feed you own has alerting and a recovery runbook a second engineer has used.

  • The numbers you own: time to onboard a new source, and how often a feed needs hands-on recovery.

COMPENSATION

$170,000–$236,000 base salary, depending on level. Meaningful equity. Medical, dental and vision.

HOW TO APPLY

Skip the cover letter. Walk us through one pipeline you built for data you didn't control: the source, what broke, how you made reruns safe and how the next source got easier. A few paragraphs. Applications without it won't be reviewed.

WHERE WE WORK

We work in person from our office at 300 Kent Avenue in Williamsburg, Brooklyn.

Vacancy posted 25 days ago
Similar jobs that could be interesting for youBased on the Data Platform Engineer, Ingestion in New York, NY vacancy
  •  ...Role: Senior Data Platform Engineer Location: New York City, NY (3 days Onsite Hybrid) Prioritization - We need a senior data platform...  ...Processing : Hands-on experience in Spark Jobs, Streaming ingestion, Data Migration and performance tuning (skew, partitioning,... 
    Suggested

    W3Global

    New York, NY
    16 hours ago
  • $164k - $295k

     ...with the world’s most advanced AI-native platform. We work on large scale distributed...  ...About the Role: We are seeking an Engineer Manager to be part of the GDI (Getting Data In) group under NG-SIEM with a focus on data ingestion at petabyte-scale. The group focuses on... 
    Suggested
    Permanent employment
    Work experience placement
    Work at office
    Local area
    2 days per week
    3 days per week
    New York, NY
    a month ago
  •  ...A global financial institution is seeking a Lead Data Platform Engineer to drive adoption and enablement for a strategic enterprise cloud data...  ...adoption challenges. Provide technical guidance across ingestion, transformation, loading, data movement, orchestration, logging... 
    Suggested
    Full time
    Immediate start

    Madison-Davis, LLC

    New York, NY
    16 hours ago
  • $136k - $253k

     ...complex corporate transaction data into structured, searchable market intelligence. Its platform helps legal and finance professionals...  .... As a Senior Data Platform Engineer, you will design, build, and...  ...Design and optimize pipelines that ingest, transform, and distribute... 
    Suggested
    Full time
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    1 day ago
  •  ...future of wealth management with you.The RoleAs an AI-Native Data Platform Engineer at Farther, you will design and own the canonical data...  ...layer that makes that possible.Your ImpactDesign scalable ingestion pipelines across custodians (Schwab, Fidelity, Pershing, etc... 
    Suggested

    Farther Advisors

    New York, NY
    2 days ago
  • $180k - $250k

     ...The Role We are seeking a hands-on Senior Data Platform Engineer to build and scale enterprise data services centered on SingleStore Helios...  ...performance. Build and maintain reliable batch, streaming and CDC ingestion pipelines from relational databases, object storage and... 

    Schonfeld

    New York, NY
    14 days ago
  • SadaPay is seeking a data engineer to own a production data platform that unifies transaction records, partner feeds, and internal events for reconciliation...  ...trusted customer data access. You will work across ingestion, processing, warehouse modelling, and operational reliability... 
    Remote job

    SadaPay

    New York, NY
    2 days ago
  • $210k - $230k

     ...Description Ares Management is seeking a Vice President, Data Platform Engineering is responsible for the strategy, architecture, engineering...  ...owns the Azure and Databricks foundation that powers data ingestion, transformation, analytics, AI/ML, and data product delivery... 
    Full time
    Temporary work
    Work experience placement
    Flexible hours

    Ares Management

    New York, NY
    16 hours ago
  • $70k - $160k

    Manager, Data Platform Engineer - Microsoft Fabric At Chanel, we are focused on creating an inclusive culture that nurtures personal growth,...  ...production-ready pipelines on Microsoft Fabric/FDP, from source ingestion through curated and governed layers that can be consumed... 
    Full time
    Temporary work
    Work at office
    Flexible hours

    Chanel

    New York, NY
    16 hours ago
  • $220k - $260k

     ...Description:Preferred Qualifications:Role overviewThe Director, Data Platform & Integration Engineering Lead owns the engineering strategy and delivery of...  ..., processing frameworks and self-service, data ingestion, streaming and events, and metadata, connectivity and shared... 
    Full time
    Local area
    Work from home
    Flexible hours

    TD Bank

    New York, NY
    16 hours ago
  • $500 per month

     ...accounts. Our global team is a diverse group of experienced engineers, traders, and brokerage professionals who are working to...  ...impact, we encourage you to apply. About the team & role The Data Platform team builds and operates the shared infrastructure that... 
    Remote work
    Home office

    Alpaca

    New York, NY
    9 days ago
  •  ...Job Description Job Description Role: Senior Data Engineer (Databricks) Location: NYC, NY (or), Seattle, WA (or), Irvine, CA (or)...  ...seasoned professional whose role sits at the intersection of platform engineering, data engineering, and data product development.... 
    For contractors

    PamTen Inc

    New York, NY
    22 days ago
  • Founding Data Science Engineer, Data PlatformOur client, an innovative life sciences organisation...  ...City, is building a cutting-edge data platform to power evidence generation and AI reasoning...  ..., and the full data lifecycle from ingestion to consumption.Data ingestion, storage... 

    Huxley Associates

    New York, NY
    16 hours ago
  • $165k - $242k

    [Main menu](#)The CoreWeave Cloud Platform[CoreWeave Cloud Platform The AI cloud purpose-built...  ...to accelerate production AI]( AI data centersPurpose-built AI data centers around...  ...( Direct-to-ExpertWork directly with the engineers behind the platform]( CenterSecurity on every... 

    CoreWeave

    New York, NY
    2 days ago
  • A global fintech company in the United Kingdom is seeking a Junior Software Engineer specializing in Python to design and maintain their data platform. This full-time role requires collaboration with cross-functional teams and offers opportunities for professional growth... 
    Remote job
    Full time

    Joinrs

    New York, NY
    1 day ago
  • Tektend LLC is seeking a Senior Data Platform Engineer to design, scale, and optimize our modern enterprise data platform based in New York, NY (hybrid). In this role, you will be instrumental in architecting high-throughput data pipelines, evolving our analytical data... 
    Temporary work
    Work at office

    Tektend LLC

    New York, NY
    13 days ago
  • $127.1k - $211.9k

     ...help build and scale a strategic platform at the center of LPL's modern data ecosystem. The team is responsible...  ...the enterprise. As an AVP Software Engineer in Data Technology, you will play...  ...integration, backend services, APIs, batch ingestion, and making trusted data assets... 
    Full time
    Work at office
    Work from home
    2 days per week
    3 days per week

    LPL Financial

    New York, NY
    5 days ago
  • Myriad360 is seeking a hands-on Data Platform Engineer to implement and maintain Airflow workflows and data pipelines across our platform. You will administer access, monitor data integrity, and coordinate with analytics engineers to ensure timely, secure data delivery... 
    Remote job

    Myriad360

    New York, NY
    4 days ago
  • Role overview A senior engineering position within a Data Science and Engineering group, working on data platform infrastructure. The role is based in Warsaw, Poland, and is part of an Information Technology department serving multiple clients. Responsibilities Design,... 

    Software Mind

    New York, NY
    4 days ago
  • Thomson Reuters seeks a Senior Data Platform Engineer in New York with 7+ years in software engineering, focusing on data platforms and backend infra. You will design, build, and maintain scalable pipelines and services for large volumes of legal and financial data. Responsibilities... 

    Thomson Reuters

    New York, NY
    1 day ago
  • Software Mind in Warsaw is seeking a Senior Data Platform Engineer to join the Data Science and Engineering group. You will design, build, and maintain data platform components used across the organization, ensuring scalable and reliable data services for multiple clients... 

    Software Mind

    New York, NY
    4 days ago
  • $127.4k - $236.6k

     ..., that transforms complex corporate transaction data into structured, searchable market intelligence. Its platform helps legal and finance professionals analyze deal...  ...speed and confidence. As a Staff Data Platform Engineer, you will set the technical direction for... 
    Full time
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    4 days ago
  • $165k - $242k

     ...pioneers, CoreWeave delivers a platform of technology, tools, and...  ...operates a growing footprint of data centers covering every region...  ...As a Senior Data Platform Engineer, you will apply advanced knowledge...  ...and streaming pipelines that ingest, transform, validate, and deliver... 
    Remote job
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    2 days ago
  •  ...pioneers, CoreWeave delivers a platform of technology, tools, and...  ...operates a growing footprint of data centers covering every region...  ...Do As a Senior Data Platform Engineer, you will apply advanced knowledge...  ...and streaming pipelines that ingest, transform, validate, and deliver... 
    Full time

    Coreweave

    New York, NY
    9 days ago
  •  ...the global stablecoin payments platform for enterprises, neobanks,...  ...stablecoin infrastructure, and our data platform holds some of the...  ...looking for a Data Platform Engineer who has built and run data...  ...platform, the infrastructure for ingestion and processing in batch and... 
    Full time
    Work at office
    Work from home
    Worldwide
    Flexible hours

    Rain

    New York, NY
    4 days ago
  •  ...expertise, scale, and technology. Job Description Senior Data Platform Engineer Role Summary We are seeking a Senior Data Platform Engineer...  ...data platform, the foundational infrastructure layer that ingestion, ontology, and business application teams build on top of.... 
    Ongoing contract
    Full time
    Worldwide

    SS&C Technologies

    New York, NY
    5 days ago
  • $180k - $230k

     ...technology is an ironman suit for engineers, linemen, and vegetation...  ...operating at 10x productivity. Our platform is powered by cutting edge...  ...are a skilled and motivated Data Platform Engineer. You will:...  ...with our hardware team on ingestion and formats. Scope is a mix of... 
    Full time
    Temporary work
    Work experience placement
    Live in
    Work at office
    Flexible hours
    2 days per week

    treeswift

    New York, NY
    5 days ago
  • $95k - $125k

     ...drives the evolution of our Technology Engineering Data Product and User Experience functions....  ...power of data mobile video and social platforms our team is committed to providing a...  ...Airflow Write tested SQL and Python for ingestion transformation validation and delivery... 
    Full time
    Work experience placement
    Local area
    Flexible hours

    Dow Jones & Company

    New York, NY
    3 days ago
  •  ...construction veterans and world-class engineers to solve physical-world problems that simulations...  ...us. The role You’ll own the data platform the rest of Bedrock builds on: the...  ...of our data platform, from edge ingestion through lakehouse storage, compute, and... 
    Remote job
    Full time
    For contractors

    bedrock-robotics

    New York, NY
    5 days ago
  •  ...The Role To build the data foundation that powers Stuut's intelligence...  ...closely with our product and engineering teams to transform raw...  ...up — design pipelines that ingest, transform, and model data from...  ...decisions Scale our data platform as we grow from dozens to hundreds... 
    Full time
    Flexible hours

    Stuut

    New York, NY
    16 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Platform Engineer, Ingestion. Be the first to apply!