Data Engineer - AI Model Evaluator
$80 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Engineer (Coding Agent Experience)
Type: Contract
Compensation: $80/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex data engineering tasks.
- Review model-generated implementations involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems .
- Identify bugs, edge cases, scalability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic data engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional data engineering experience.
- Experience building ETL pipelines , data warehouses , analytics platforms , or distributed data systems .
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated data infrastructure and pipeline implementations.
Preferred
- Experience operating large-scale data platforms .
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AI Model Evaluator in New York, NY vacancy
$400 per month
...Mercor is partnering with a leading AI research lab to support a... ...Agents project. Contributors help evaluate and improve frontier AI coding models through structured technical assessments... .... The work focuses on realistic data engineering workflows and model evaluation....Suggested- Mercor partners with a leading AI research lab to support a Frontier Code Agents project, focusing on realistic data engineering workflows and model evaluation. Contributors help evaluate and improve frontier AI coding models through structured technical assessments and...Suggested
- ...experienced Investment Banking SME to support the development, evaluation, and improvement of advanced AI models in finance. You will assess AI-generated analyses for accuracy, reasoned judgments, and data integrity, guiding model refinements and prompts for high-quality...SuggestedRemote job
- Obsidian is seeking Insurance domain SMEs to join a cutting-edge AI training program. You will evaluate AI model outputs against real underwriting practice and rubrics, guiding research and engineering teams to close knowledge gaps in underwriting, claims, and risk assessment...Suggested
$85 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type... ...coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations...SuggestedContract workSummer workRemote workMercor
New York, NY3 days ago$20 per hour
Feedinkoo is looking for a Web Developer/Designer to enhance AI models by evaluating design work, including interfaces and user experiences. This role involves reviewing AI‑generated visuals and providing feedback to improve users' experience with AI tools. Working remotely...Remote job- SME Careers is looking for a remote R Engineer to review AI-generated responses and create high-quality R and data-analysis content. The position requires a strong background in R programming, applied statistics, and excellent writing skills to document analyses. Candidates...Remote job
- Data Scientist and Model Developer, Vice President (Contract) A temporary role... ...in AML model development, AI/ML and GenAI. Location Kuala... ...development, ongoing performance evaluation and annual model reviews.... ...in developing analytic engines with detailed technical documentation...Contract workTemporary work
- ...independent assurance and evaluating the company's risk management... ....We are looking for data scientists and AI developers who will power our... ...You will be deploying your engineering, data analytics and data science... ...in frameworks for auditing models, including criteria like...
$20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...external tools. Generate high-quality human evaluation data by identifying response strengths, areas... ...and completeness of responses. Ensure model responses align with expected...Remote jobPart timeSummer work- Mercor is seeking a Generalist who can operate in English and Punjabi. This contract, remote position focuses on evaluating AI outputs and supporting model evaluation tasks. You will conduct fact-checking, assess reasoning, clarity, tone and completeness, and provide actionable...Remote jobContract work
- ...Reference Number: 25-00661Title: DATA ENGINEERPosted Date: 2025-06-1... ...Skills:Proficiency in data engineering programming languages (preferably... ...and stream), one or more data modelling techniques (Dimensional, Data... ...APIsKnowledge of Generative AI concepts (such as RAG, Vector...
$170k - $250k
...About us Thesis is building the AI-powered care team platform for infinitely scalable... ...We are looking for a highly driven Data Engineer to help design, build, and scale the data... ...normalize client data. Design robust data models: Build and maintain clean, well-...Full timeWork at office3 days per week$125k - $145k
...we're buildingOur team runs a range of data and AI projects across the GPS business.A few... ...product, servicing, and client questions.•Models that optimize pricing and foreign-... ...forThis is a hybrid role that spans data engineering, AI product enablement, and platform architecture...Full timeWork at officeFlexible hoursDay shift- ...WithAt Slalom, we co-create custom software, data, and cloud products with clients who are... ...true change.Our Data Architecture & Engineering team sees data as the foundation for decision... ..., and transformation. We embrace AI to scale with quality, innovate without fear...Temporary workLocal area
- ...investment management firm seeking a Senior Data Engineer to help transform and modernize its... ...environments into Snowflake.Design scalable data models and enterprise warehouse solutions.... ....Experience building semantic layers and AI-ready data platforms.If you meet the required...Work at officeImmediate startWork from home
- ...advanced hardware, software, and AI technology to make it possible... ...of sleep into a personalized, data-driven recovery experience. We... ...we need a world-class Data Engineer to power our next phase of hypergrowth... ...expertise in data modeling and warehousing—you think in terms...Full timeWork at officeImmediate startWorldwideSleeping nightsNight shift
$80.42k - $106.05k
...New Jersey - Onsite Hybrid Your Role 1 Cloud Data Architecture Development Design and implement scalable... ...leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its...Full timeLocal area$190k - $220k
...more about ourTotal Rewards philosophy.AI is a fundamental part of how work gets done... ...during the interview process.The Data Engineering team builds tools and systems that make... ...event streams.Strong foundation in data modeling, schema design, and data quality best practices...Full timeWork at officeLocal area2 days per week3 days per week$80.79k - $90.27k
...Description We are seeking a highly motivated Data Engineer to design, develop, and maintain... ...use cases. Develop and optimize data models, database schemas, and storage structures... ...fueled by its market leading capabilities in AI, generative AI, cloud and data, combined...Full timeLocal area- ...Degree in Computer Science, Business Analytics, Engineering, Mathematics, or related field2+ years of professional experience in data engineering, software engineering, or... ...platforms such as (Databricks, SageMaker, Vertex AI), MLOps best practices, and GenAI toolkits (LangChain...ApprenticeshipEasy work
$148.5k - $260.1k
...SalesforceSalesforce is the #1 AI CRM, where humans with... ...search, enterprise data discovery, and... ...Knowledge Graph and AI Engineering team.The SMTS will serve... ...Engineering: Develop graph data models, write complex graph... ...standards.)Evaluate & Innovate (LMTS): Conduct...Full timeContract work- ...leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are... ...Cohere is a team of researchers, engineers, designers, and more, who are all... .... As a member of our Analytics & Data Insights team, you'll tackle the...Work at officeLocal areaRemote workHome office
$80.79k - $90.74k
...Description We are seeking an experienced AWS Data Engineer to design, build, and optimize scalable... ...Snowflake. Design and optimize data models, database schemas, partitioning... ...fueled by its market leading capabilities in AI, generative AI, cloud and data, combined...Full timeLocal area- Req ID:376263NTT DATA strives to hire exceptional, innovative and passionate individuals... ...now.We are currently seeking a Senior Data Engineer - Snowflake to join our team in Remote, New... .... We are one of the world's leading AI and digital infrastructure providers, with...Full timeWork at officeRemote workFlexible hours
$140k - $348k
...translating designs into code, or iterating with AI. From idea to product, Figma empowers... ...us!We are looking for an experienced Data Engineer to partner with our Data Science and Data... ...deep understanding of Figma’s core data models and optimize data pipelines for scale.Partner...Minimum wageFull timeLocal areaRemote workFlexible hours$128k - $252.5k
...services covering valuation modeling, cost optimization,... ...We are hiring a Senior Data Scientist to... ...of tools, products, and AI services. You will work... ...interdisciplinary data scientists, engineers, architects, and... ...helping them understand and evaluate new and essential areas...Local area$90k - $135.29k
...Program for Surgical Intelligence is seeking a Data Engineer to support the development and deployment... ...execution of research and clinical AI projects across the Department of SurgeryTrack... ....Familiarity with healthcare data models and standards (such as Epic Clarity, Caboodle...TraineeshipLocal areaImmediate startFlexible hours$97.17k - $126.32k
...that mindset.About the Role:The Data Engineer designs, develops, and... ...implement logical and physical data models that promote consistency,... ...and development standards.• Evaluate emerging technologies and recommend... ...Intelligence, analytics, and AI initiatives through modern...Full timeLocal areaVisa sponsorship$137.4k - $161.7k
...impact?West Monroe is seeking a Senior Data Engineer to join our Data & AI practice. In this role, you will... ...Develop robust ELT/ETL processes, data models, orchestration workflows, and data... ...clients, exercising sound judgment in evaluating outputs.Apply AI technologies (e.g.,...Local areaImmediate startFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - AI Model Evaluator. Be the first to apply!
Related searches
- data center engineer New York, NY
- senior cloud data engineer New York, NY
- data infrastructure engineer New York, NY
- sr information security engineer New York, NY
- hadoop big data developer New York, NY
- data visualization developer New York, NY
- associate data engineer New York, NY
- remote data engineer New York, NY
- etl data engineer New York, NY
- data engineer analytics New York, NY