Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Engineering Manager, Big Data, Job Understanding

$200k - $265k

ZipRecruiter

Job Description

Job Description

We offer a hybrid work environment. Most US-based positions can also be performed remotely (any exceptions will be noted in the Minimum Qualifications below.)

Our Mission:  

To actively connect people to their next great opportunity. 

Who We Are: 

ZipRecruiter is a leading online employment marketplace. Powered by AI-driven smart matching technology, the company actively connects millions of all-sized businesses and job seekers through innovative mobile, web, and email services, as well as through partnerships with the best job boards on the web. ZipRecruiter has the #1 rated job search app on iOS & Android.

About the Team:

Recommendations Ecosystem

The Recommendations Ecosystem group owns the intelligence that connects job seekers and jobs in our marketplace. We aim to understand every job and every job seeker well enough to make matches that work for both sides. We structure and enrich the jobs corpus, build the foundational models that encode jobs and job seekers, and run the matching engines that put the right opportunities in front of the right people.

What our team owns

Our group is the authoritative source for job understanding and matching intelligence, focusing on accuracy, relevance, and trust:

  • Jobs Corpus Understanding: ZipRecruiter takes in millions of job postings from employers, partners, and feeds. We own the ingestion and enrichment pipelines that turn raw postings into structured data. We use ML classifiers to infer job attributes like education requirements and pay, and we attach company and location data. We find and suppress fraudulent listings or terms of use violations, so the marketplace stays safe and worth using. The result is a canonical view of every job that recommendations, search, and employer products build on.
  • Foundational Models and Embeddings: At the core of ZipRecruiter's matching ecosystem are models that encode everything we can learn about jobs and job seekers. We build LLM-based embedding models from job content, resumes, and behavioral signals. These are the building blocks for matching and other ML models across the company, and we serve them at scale online and in batch.
  • Matching Engines: We are responsible for the ML systems that recommend jobs to job seekers and job seekers to employers across our app, website, and email. The engines cover large-scale candidate retrieval over vector DBs and orchestration that delivers fresh, personalized recommendations.

About the Job:

ZipRecruiter's Recommendation Ecosystem Group provides the backend machine learning and data platforms that connect job seekers to the right jobs. Within this group, the Job Understanding team is the authoritative source for foundational job data, pipelines, and matching intelligence.

We are seeking an Engineering Manager to lead an experienced, high-impact engineering team. In this role, you will guide the team in modernizing core data pipelines, storage solutions, and access methods while enhancing data quality and security systems.

  • Lead, mentor, and unblock a senior team of 5 engineers responsible for core job ingestion pipelines and data stores.
  • Drive the rebuild of the job ingestion pipeline and refine how job data is exposed to consuming engineering teams.
  • Prioritize initiatives that improve underlying input data quality and handle low-structure, user-generated content.
  • Own and lead improvements to fraud detection and blocking mechanisms.
  • Partner closely with other Engineering stakeholders, Data Scientists, and a dedicated Product Manager to balance requests and roadmap priorities.
  • Ensure we can act on everything we learn about jobs in a scalable real time manner. Scale our data pipelines both for performance and maintainability so we can be ambitious about what data we collect and where we can personalize the job seeker experience. 

Minimum Qualifications:

  • 5+ years of professional software development experience.
  • 2+ years of experience as a software engineering leader
  • Deep experience with modern big data frameworks (e.g. Spark, Hadoop, Hive).
  • Streaming technologies (Kafka, Flink, Beam).
  • Strong CS fundamentals (data structures, algorithms) and coding skills.
  • Experience with containerization technologies like Docker and Kubernetes.
  • Experience working on a B2C or B2B Product
  • This role is for engineers who are talking with their Product Managers about problems to solve and turning them into engineering solutions. They are naturally curious, problem solvers, and deeply understand the product they work on.
  • Must have experience dealing with messy/unreliable data (third party data sets, data integrations, migrations) 

Preferred Qualifications:

  • Cloud platforms (GCP, AWS, Azure).
  • Modern data storage solutions (Delta Lake, HBase, Hudi).
  • Scala

As part of our team you'll enjoy:

  • Competitive compensation
  • Exceptional benefits package
  • Flexible Vacation & Paid Time Off
  • Employer-matched 401(k) plan 

The US base salary range for this full-time position is $200,000 - $265,000. Our salary ranges are determined by role, level, and location, and the range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location, role-related knowledge and skills, depth of experience, relevant education or training, and additional role-related considerations.

Depending on the position offered, equity, bonuses, commission, or other forms of compensation may also be provided as part of a total compensation package, in addition to a full range of medical, financial, and other benefits.

ZipRecruiter is proud to be an equal opportunity employer and provides equal employment opportunities (EEO) to all employees and applicants without regard to race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity or genetics.

Privacy Notice:  For information about ZipRecruiter's collection and processing of job applicant personal data for this job, please see our Privacy Notice at: -applicant-privacy-notice

Vacancy posted 12 days ago
Similar jobs that could be interesting for youBased on the Engineering Manager, Big Data, Job Understanding in San Francisco, CA vacancy
  •  ...Visit us at accenture.com.You Are:As a Data Engineering, Management & Governance professional, you will...  ...of hands-on experience on Cloud and Big Data technologies such as serverless...  ...defined differently across systems)A good understanding of governance and security best... 
    Big data
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    San Francisco, CA
    16 hours ago
  • $193.8k - $285k

    About the TeamDoorDash is a data driven organization and relies on...  ...of DoorDash success. The Data Engineering team builds database solutions...  ...engineers we are looking for managers who are passionate about Data...  ...Redshift, AWS/GCP, Hadoop/Spark/Big data, Lambda/KAPPA... 
    Big data
    Hourly pay
    Temporary work
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    4 days ago
  • $228.6k - $317.23k

     ...are passionate about enabling data teams to solve the world's...  ...business. We are the Lakeflow Engineering team, responsible for Databricks...  ...easy to operate. You will manage technical debt, including long...  ...directly with customers; ability to understand customer needs.Can ensure the... 
    Big data
    Local area
    Worldwide

    DataBricks

    San Francisco, CA
    3 days ago
  • $190k - $240k

     ...team has a unique opportunity to work on applications and data at scale, serving millions of jobseekers and tens of...  ...job seekers alike. As part of this effort, we're hiring an Engineering Manager to lead our Big Data Engineering team.  Our business is complex; we're... 
    Big data
    Full time
    Remote work
    Flexible hours

    ZipRecruiter

    San Francisco, CA
    3 days ago
  • $269.1k - $307.2k

    Director, Data Science Data is at the center of...  ...records to unlock the big opportunities that...  ...lending and model risk management require, and carry them...  ...scientists, software engineers, and product managers...  ...structures. You know understanding the data is often the... 
    Big data
    Full time
    Part time
    Work at office
    Local area
    Flexible hours

    Capital One

    San Francisco, CA
    2 days ago
  •  ...Designs and maintains Python-based data abstraction and analytical...  .... Supports production data management and model integrity, and...  ..., quantitative alpha models, big data analytics, and performance...  ...degree in Computer Science, Engineering, Information Technology, Information... 
    Big data
    Full time

    Fidelity Investments

    San Francisco, CA
    16 hours ago
  • $281k - $334k

     ...system of record to making and managing AI agents to automating away...  ...work.About the Role:Notion’s Data Foundations team builds and...  ...creating clear ownership, growing engineers and technical leaders, and...  ...-product thinking. You understand the needs behind partner requests... 
    Local area

    Notion Labs

    San Francisco, CA
    16 hours ago
  •  ...legal entity.Atlassian’s Trusted Data Platform (TDP) provides...  ...teams. It helps those teams manage persistent customer data while...  ...database platform. It gives engineering teams a supported way to build...  ...and measurable outcomes.Strong understanding of distributed systems, databases... 
    Work at office
    Local area

    Atlassian

    San Francisco, CA
    2 days ago
  • $140.3k - $260.5k

     ...society.Who We Are Our Data, Digital,and Analytics...  ...fosters a unified understanding of customers, actions,...  ...a Senior Data Product Manager, Analytical Data Products...  ...Scientists, Analysts, and Engineers) to evangelize the...  ...technical fluency with big data technologies,... 
    Big data
    Full time
    Local area
    Relocation package
    3 days per week

    Genentech

    South San Francisco, CA
    4 days ago
  •  ...healthcare for the better. About the Role As Senior Manager, Data Engineering, you will define and lead the technical strategy for Headway...  ...Science, Analytics, Product, and Engineering to deeply understand and anticipate the data needs of a fast-growing healthcare... 

    Headway - Design & Development

    San Francisco, CA
    4 days ago
  •  ...DescriptionDatabricks is looking for a Senior Manager, Infrastructure Data Science to shape the future of...  ...optimization, reliability engineering, infrastructure efficiency, and customer...  ...data engineering, data modeling, and big data technologiesLeadership skills and... 
    Big data
    Work at office
    Worldwide

    DataBricks

    San Francisco, CA
    4 days ago
  •  ...Song, Supply Chain and Engineering, and Talent, with advanced...  ...capabilities in AI and Data, Industry and Process,...  ...is seeking a Senior Manager to lead Planning transformation...  ...find new ways to understand how to make their...  ...Sciences industryExpertise in big data technologies,... 
    Big data
    Live in
    Work at office
    Local area
    Shift work

    Accenture

    San Francisco, CA
    16 hours ago
  • $235k - $275k

     ...year runway.About the RoleOur engineering team has scaled from 12 to 40+...  ...- to senior-level Engineering Managers to help us navigate this next...  ...with Product, Operations, and Data to deliver measurable impact for...  ...-on experience with Big Data, data platforms, and data... 
    Big data
    Temporary work
    Work at office
    Monday to Friday
    Monday to Thursday

    Sprinter Health

    San Francisco, CA
    1 day ago
  •  ...expertise across Cybersecurity, Digital Core, Finance, Industry and Enterprise, Song, Supply Chain and Engineering, and Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We serve approximately 9,000 clients and generated... 
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    San Francisco, CA
    2 days ago
  • $150.7k - $251.2k

     ...looking for a dynamic and experienced Manager of Data Engineering to lead our team in designing and...  ...solutions. Key responsibilities include: Understanding and analyzing business requirements...  ...Databricks and experience with Spark for big data processing. Hands-on experience... 
    Big data
    Immediate start

    EY

    San Francisco, CA
    2 days ago
  •  ...IDMissionReporting to a Director, Field Engineering, the Sr. Manager, Field Engineering will help lead a...  ...focusing on enterprise software, big data/analytics, data engineering, data science...  ...and accelerate client successStrong understanding of consumption-driven business... 
    Big data
    Worldwide

    DataBricks

    San Francisco, CA
    1 day ago
  • $267k - $375k

     ...to run Ray, so that any developer or data scientist can scale an ML application...  ...role: Anyscale is looking for an engineering leader to manage the open source Ray Data team. Ray Data...  ...running Ray Data at large scale, to understand their workloads and turn them into... 
    Full time
    Flexible hours

    anyscale

    San Francisco, CA
    8 days ago
  •  ...model innovation and systems engineering paired with a design-minded product...  ...from the quality of our data. As we push the frontier of...  ...multimodal annotation models that understand who said what, when, and how,...  ...datasets. You'll start by managing a team of 6–8 spanning core... 
    Full time
    Work at office
    Visa sponsorship
    Flexible hours

    cartesia

    San Francisco, CA
    8 days ago
  • $115.4k - $204.2k

     ...are passionate about enabling data teams to solve the world’s...  ...improve their business.Data Engineering is foundational and among the...  ...role, you will lead product management for a core Lakeflow product...  ...portfolioDevelop and deepen understanding of and expertise in Data Engineering... 
    Local area
    Worldwide

    DataBricks

    San Francisco, CA
    2 days ago
  • Principal Data Engineer For Marketing And Consumer Data Technology is at the heart of Disney...  ...pairing, and coaching — building depth in big-data engineering, identity domain...  ...Redshift, Big Query, Databricks) Solid understanding of Data Modeling techniques and Data Warehousing... 
    Big data
    Work experience placement

    The Walt Disney Studios

    San Francisco, CA
    2 days ago
  • $242.63k

     ...for a principal software engineer to lead the next generation of data infrastructure at...  ...powers mission critical big data and AI applications...  ...including compute engines, job management, resource management,...  ...best of my knowledge. I understand that any false statement... 
    Big data
    Work at office
    Local area
    Remote work
    Relocation
    Relocation package

    Pinterest

    San Francisco, CA
    4 days ago
  • $405k

     ...Engineering Manager, Research Data Platform San Francisco, CA | New York City, NY About Anthropic Anthropic...  ...the engineers who support them — to understand how they actually work, where...  ...highest-impact AI research will be big science. At Anthropic we work as a single... 
    Full time
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    8 days ago
  •  ...are passionate about enabling data teams to solve the world’s...  ...improve their business. Founded by engineers — and customer obsessed — we...  .... You will be a Sr Product Manager for Unity Catalog (). Unity...  ...large enterprise customers to understand their needs, developing long-... 
    Worldwide

    DataBricks

    San Francisco, CA
    4 days ago
  • $243k - $364k

     ...since 2010. We're looking for a Senior Manager, Data Science & Analytics to lead and grow...  ...strong partnerships across Product, Engineering, Marketing, Finance, Trust & Safety etc...  ...with leaders across the company to understand their needs, align priorities, and make... 
    Full time
    Work at office
    Local area
    Remote work
    Flexible hours

    GoFundMe

    San Francisco, CA
    5 days ago
  • $204k - $281k

     ...expansion into the Agentic use cases.The Engineering Manager (Governance Intelligence) OpportunityAs...  ...lifecycle: from problem framing and data curation through model training, A/B testing...  ....Partner with Product and Design to understand and own the desired customer outcome,... 
    Work at office
    Local area
    Worldwide
    Flexible hours

    Okta

    San Francisco, CA
    16 hours ago
  • $123.68k - $254.64k

     ...our recruiting process here.Pinterest's Data Engineering organization builds and operates the...  ...and methodologies that scale program management across the Platforms organization.Use...  ...infrastructure modernization.Advanced understanding of large-scale data platforms — batch/... 
    Work at office
    Local area
    Remote work
    Relocation
    Relocation package

    Pinterest

    San Francisco, CA
    3 days ago
  • $215k - $260k

     ...careers alongside a team of experts across energy, manufacturing, data center construction, and cloud services.If you want to do the...  ...other, come build with us at Crusoe.About the Role:As the Engineering Manager for the Data Team, you will lead a high-impact team of data... 
    Temporary work
    Work at office

    Crusoe

    San Francisco, CA
    2 days ago
  • $146k - $161k

     ...best-in-class antibodies that were engineered by Paragon Therapeutics and target the...  ...company culture. Job Title: Manager of Clinical Data Management Location: Remote OR Hybrid...  ...electronic Trial Master Files (eTMF). Strong understanding of GCP, ICH Guidelines, FDA, EMA,... 
    Interim role
    Work at office
    Remote work
    3 days per week

    Remote Services Inc.

    San Francisco, CA
    4 days ago
  • $227.5k - $237.5k

     ...Senior Manager, Data Engineering Turn/River Capital is a private equity firm that partners with software businesses to drive AI transformation and catalyze capital-efficient growth. As a team of equal parts software operators and investors, we have the people, data-... 
    Full time
    Summer work
    Work at office
    Work from home
    Home office
    Monday to Friday
    Flexible hours

    Turn/River Capital

    San Francisco, CA
    2 days ago
  • $270k - $365k

     ...of committed researchers, engineers, policy experts, and business...  .... About the Role As Data Operations Manager, you'll build and scale data...  ...combined with technical depth to understand what makes high-quality...  ...impact AI research will be big science. At Anthropic we work... 
    For contractors
    Work at office
    Local area
    Visa sponsorship
    Flexible hours
    Shift work

    anthropic

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Engineering Manager, Big Data, Job Understanding. Be the first to apply!