Data Engineer - AI Model Evaluator
$80 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Engineer (Coding Agent Experience)
Type: Contract
Compensation: $80/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex data engineering tasks.
- Review model-generated implementations involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems .
- Identify bugs, edge cases, scalability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic data engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional data engineering experience.
- Experience building ETL pipelines , data warehouses , analytics platforms , or distributed data systems .
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated data infrastructure and pipeline implementations.
Preferred
- Experience operating large-scale data platforms .
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 16 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AI Model Evaluator in New York, NY vacancy
$85 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type... ...coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations...SuggestedContract workSummer workRemote workMercor
New York, NY5 days ago$20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...external tools. Generate high-quality human evaluation data by identifying response strengths, areas... ..., and completeness of responses. Ensure model responses align with expected...SuggestedRemote jobContract workPart timeSummer work- ...implementation of scalable data architectures for cutting-edge AI and agentic systems. You... ...architectures for model management, observability... ...design of complex feature engineering workstreams, ensuring our... ...judgment around model behavior, evaluation, reliability, guardrails,...SuggestedApprenticeshipWork at officeLocal areaEasy work
McKinsey & Company
New York, NY4 days ago $86.7k - $170.9k
Position Summary Join Deloitte’s Core AI & Data practice and help organizations... ...intelligence capabilities. As a Databricks Data Engineer, you will support the design, build, and... ...architecture, integration, and data modeling practices to improve data quality, scalability...SuggestedLocal areaVisa sponsorshipDeloitte
New York, NY3 days ago$116.2k - $229.1k
Position Summary Join our AI & Engineering team in transforming technology platforms, driving... ...sector solutions in software, data, AI, network, and hybrid cloud infrastructure... ...technology & data platforms. Our delivery models are tailored to meet each client's unique...SuggestedLocal areaVisa sponsorshipDeloitte
New York, NY3 days ago- ...Financial ServicesSelling Points Lead impactful data engineering projects at a leading organization.... ...accessibility.Develop Power BI semantic models with robust data entitlement rules for... ....Drive innovation by integrating AI solutions and modern data engineering practices...
Green Key Resources
New York, NY1 day ago $176k - $179.5k
...be on the global Life Sciences AI team, fully integrated with... ...McKinsey, as a senior software engineer, building AI that transforms Pharma... ...design and manage scalable data pipelines and secure analytics... ...’s asset-based consulting model and help clients harness the full...ApprenticeshipEasy workShift workMcKinsey & Company
New York, NY4 days ago$110k - $169k
## Pricing Data EngineerApplyremote type: Hybridlocations: New York... ...:**The Pricing Data Engineer builds and maintains the data... ...solutions and the integration of AI-driven capabilities to enhance... ...services and emerging delivery models* Develop and manage data pipelines...Full timeWork at officeMcDermott
New York, NY18 hours ago$110k - $169k
...Pricing Data Engineer Build and maintain data infrastructure and tools for consistent, data‑... ...teams to modernize workflows, integrate AI, and improve pricing analysis. Essential... ...enabled legal services and emerging delivery models Develop and manage data pipelines and...Full timeWork at officeMcDermott Will & Schulte LLP
New York, NY18 hours ago$157.95k - $259.48k
...We are now building the AI layer that brings together the depth of data Catapult has collected over... ...for a Senior ML / Data Engineer to build the data... ..., transform, serve, and evaluate athlete performance data... ...and implement graph data models and schemas representing...Full timeTemporary workInternshipCatapult Sports
New York, NYa month ago$120k - $145k
...'re looking for a Recruiting Systems & Data Integration Engineer to build and operate the connections between... ...resumes, and related data so Suzy, our AI agent, can analyze and interview... ...platforms vary widely in their APIs, data models, reporting capabilities, and...Remote workengin sciences
New York, NY4 days ago$200k - $400k
Staff Data EngineerLocation: New York City, NYEmployment Type: Full... ...is looking for a Staff Data Engineer to join its engineering team.This... ...development, and modern AI/ML applications.What You'll DoDesign... ...through productionDesign data models and infrastructure that...Full timeCyberCoders
New York, NY4 days ago- ...Job Description Job Description Data Engineer Location: New York, NY Employment Type: Full-time Focus: Data Engineering, Snowflake, Data Modeling, AI Data Infrastructure, Marketing Technology About Our Client Our client is building an AI-powered platform...Full time
NxT Level
New York, NYa month ago - ...Job Title : Senior Data/Software Engineer Job Location - Address : New York, United States Skill... ...data transformation logic, data models, and analytics solutions for the Enterprise... ...accessibility, and utility to support AI capabilities and business decision-making...
Tek Leaders Inc
New York, NYa month ago $190k - $220k
...about our Total Rewards philosophy . AI is a fundamental part of how work gets... ...during the interview process. The Data Engineering team builds tools and systems that make... ...streams. ~ Strong foundation in data modeling, schema design, and data quality best practices...Full timeWork at officeLocal area2 days per week3 days per weekgusto
New York, NYa month ago- ...Overview We are seeking a skilled AWS Data Engineer to design, build, and optimize scalable... .... Solid understanding of data modeling, warehousing, and big data concepts.... ...fueled by its market leading capabilities in AI, generative AI, cloud and data, combined...Full timeLocal area
Capgemini
New York, NYa month ago - ...Assistance, Inc. (SGA) is searching for a Senior Data Engineer - Investment Technology (Python) for a... ...manage PostgreSQL and analytical data models supporting front-office investment... ...processing, you agree to be contacted by our AI recruiter via email, text, or phone. Message...Remote work
Software Guidance & Assistance
New York, NY18 hours ago $160k - $200k
...Data Engineer - Vice President About the RoleiCapital is seeking a strategic... ..., operational workflows, and AI-driven capabilities. This is... ...maintain governed semantic models, business metrics, entities,... ...calling architecturesAble to evaluate AI systems for accuracy, consistency...Full timeWork at officeRemote workiCapital Network
New York, NY4 days ago$102.5k - $187.9k
...candidate will have a strong background in data analytics, experience with Adobe CJA, and... ...in our team-led and leader-enabled hybrid model. Our expectation is for most people in external... ...trust in capital markets.Enabled by data, AI and advanced technology, EY teams help...Summer holidayFlexible hoursEY (Ernst & Young)
New York, NY4 days ago$130k - $150k
...is seeking an analytics engineer to own the ingestion... ...transformation of the data that powers the Integrity... ...party vendors — into well-modeled, tested, and documented... ...activity can be evaluated in the context of what... ...detection, alerting, and AI-driven software the Integrity...Odd jobFull timeTemporary workLocal areaRemote work1 day per weekNational Basketball Association
New York, NY1 day ago$94.95k - $118.69k
...catalog of sounds and samples and an expanding AI stack. With a rent-to-own marketplace of... ...collaborators. Are you ready? JOB TITLE: Data Engineer II LOCATION: REMOTE- US THE ROLE:... ...Build and maintain data models in SQLMesh, including incremental strategies...Work at officeLocal areaImmediate startRemote workWorldwideFlexible hoursSplice
New York, NYa month ago$60 - $90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Machine Learning Engineer — Model Evaluation & Experimentation Type: Contract Compensation: $...Contract workSummer workRemote workMercor
New York, NY5 days ago$137.7k - $229.5k
Position Summary Our Deloitte AI & Engineering team works to transform technology platforms, drive innovation, and help make a significant... ..., and fuel growth through innovation.Work You’ll DoAs a Lead Data Engineer II on the team, you will be responsible for:Support...Local areaDeloitte
New York, NY3 days ago- ...Description Job Description Senior Data Engineer II Level: IC4... ...Engineering, Data Infrastructure, AI-Enabled Data Workflows... ...independent architectural decisions, evaluate platform tradeoffs, influence... ...ingestion, transformation, modeling, and serving Make project-...Full time
NxT Level
New York, NY29 days ago - ...Description Bustle Digital Group (BDG) is looking for a talented Data Engineer to join our growing data team. You will work alongside the... ...and fast-growing team. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing...Work at officeRemote workFlexible hours
BDG
New York, NYa month ago $150k - $200k
...intensive manual processes. Stuut’s AI agent works across order... ...Microsoft. The Role To build the data foundation that powers Stuut's... ...closely with our product and engineering teams to transform raw... ...their sleeves to build pipelines, models, and systems from scratch. You...Full timeWorldwideFlexible hoursStuut
New York, NY4 days ago$110k - $165k
...turns a high-volume, multi-source data ecosystem — operational... ...telemetry — into the trusted data models, metrics, and dashboards that... ...We are hiring a Senior Data Engineer who is, in equal measure, a data... ...domains for analytics, reporting, AI, and operational use. •...New York, NYmore than 2 months ago$200k - $275k
...Description Job Description Founding Data Engineer — AI Healthcare The Opportunity Our... ..., documented, version-controlled models Establish trusted definitions for metrics... ...supporting AI model training and evaluation Partner with AI engineers on training...Immediate startNxT Level
New York, NYa month ago- ...seeking motivated and enthusiastic summer interns to join our data engineering team. An internship at Octaura provides an opportunity to acquire... ...platform, internal business intelligence and our agentic AI systems in a very fast-paced startup environment. The energy in...Summer workInternshipSummer internshipWork at office
Octaura
New York, NY4 days ago $320k - $405k
...Data Engineer, Safeguards San Francisco, CA | New York City, NY About... ...interpretable, and steerable AI systems. We want AI to be... ...Safeguards team works to monitor models, prevent misuse, and ensure... ...ML model monitoring or evaluation ~ Familiarity with data privacy...Work at officeImmediate startVisa sponsorshipFlexible hoursAnthropic
New York, NY4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - AI Model Evaluator. Be the first to apply!
Related searches
- senior data center engineer New York, NY
- senior cloud data engineer New York, NY
- senior data integration developer New York, NY
- data developer New York, NY
- entry level data engineer New York, NY
- data engineer New York, NY
- big data cloud engineer New York, NY
- data platform engineer New York, NY
- data engineer intern New York, NY
- data engineering intern summer New York, NY