Data Engineer - AI Model Evaluator
$80 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Engineer (Coding Agent Experience)
Type: Contract
Compensation: $80/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex data engineering tasks.
- Review model-generated implementations involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems .
- Identify bugs, edge cases, scalability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic data engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional data engineering experience.
- Experience building ETL pipelines , data warehouses , analytics platforms , or distributed data systems .
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated data infrastructure and pipeline implementations.
Preferred
- Experience operating large-scale data platforms .
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AI Model Evaluator in New York, NY vacancy
- ...experienced Investment Banking SME to support the development, evaluation, and improvement of advanced AI models in finance. You will assess AI-generated analyses for accuracy, reasoned judgments, and data integrity, guiding model refinements and prompts for high-quality...SuggestedRemote job
- Obsidian is seeking Insurance domain SMEs to join a cutting-edge AI training program. You will evaluate AI model outputs against real underwriting practice and rubrics, guiding research and engineering teams to close knowledge gaps in underwriting, claims, and risk assessment...Suggested
$85 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type... ...coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations...SuggestedContract workSummer workRemote workMercor
New York, NY3 days ago$20 per hour
Feedinkoo is looking for a Web Developer/Designer to enhance AI models by evaluating design work, including interfaces and user experiences. This role involves reviewing AI‑generated visuals and providing feedback to improve users' experience with AI tools. Working remotely...SuggestedRemote job- SME Careers is looking for a remote R Engineer to review AI-generated responses and create high-quality R and data-analysis content. The position requires a strong background in R programming, applied statistics, and excellent writing skills to document analyses. Candidates...SuggestedRemote job
- Data Scientist and Model Developer, Vice President (Contract) A temporary role... ...in AML model development, AI/ML and GenAI. Location Kuala... ...development, ongoing performance evaluation and annual model reviews.... ...in developing analytic engines with detailed technical documentation...Contract workTemporary work
- ...independent assurance and evaluating the company's risk management... ....We are looking for data scientists and AI developers who will power our... ...You will be deploying your engineering, data analytics and data science... ...in frameworks for auditing models, including criteria like...
- Mercor is seeking a Generalist who can operate in English and Punjabi. This contract, remote position focuses on evaluating AI outputs and supporting model evaluation tasks. You will conduct fact-checking, assess reasoning, clarity, tone and completeness, and provide actionable...Remote jobContract work
- Screenshot Understanding Model Evaluator is a remote evaluation track for reviewing screenshot understanding model evaluation prompts and... ...the modeling team can use to retrain. Why this role matters AI data reviewers help turn screenshot understanding model evaluation...Hourly payFor contractorsRemote work10 hours per week
$20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...external tools. Generate high-quality human evaluation data by identifying response strengths, areas... ...and completeness of responses. Ensure model responses align with expected...Remote jobPart timeSummer work- ...Generalist to perform English and Tamil language evaluation tasks on a remote contract basis. You will... ...external tools, generating high-quality evaluation data that highlights strengths, weaknesses, and inaccuracies in model responses. The role requires strong English...Remote jobContract work
$110k - $190k
Senior Data Management Professional - Data Engineering - Commodities Data Location New York Business Area... ...we’re responsible for onboarding, modelling and maintaining data that are fit... ...leverages modern technologies including AI and machine learning to enhance...Temporary workFor contractorsWork experience placement$97.17k - $126.32k
...that mindset.About the Role:The Data Engineer designs, develops, and... ...implement logical and physical data models that promote consistency,... ...and development standards.• Evaluate emerging technologies and recommend... ...Intelligence, analytics, and AI initiatives through modern...Full timeLocal areaVisa sponsorship$90k - $135.29k
...Program for Surgical Intelligence is seeking a Data Engineer to support the development and deployment... ...execution of research and clinical AI projects across the Department of SurgeryTrack... ....Familiarity with healthcare data models and standards (such as Epic Clarity, Caboodle...TraineeshipLocal areaImmediate startFlexible hours$137.4k - $161.7k
...impact?West Monroe is seeking a Senior Data Engineer to join our Data & AI practice. In this role, you will... ...Develop robust ELT/ETL processes, data models, orchestration workflows, and data... ...clients, exercising sound judgment in evaluating outputs.Apply AI technologies (e.g.,...Local areaImmediate startFlexible hours$123k - $215.25k
...annually + bonus + benefitsJob Function: Engineering & ArchitectureSchedule: Full... ...experience engineering (Mobile and Web), AI/ML, marketing technology, enterprise communications... ..., and automation, grounded in our data technology model that prioritizes data governance. It...WorldwideVisa sponsorship$116.2k - $229.1k
...Summary Join Deloitte’s AI & Engineering practice and help... ...technology platforms, modernize data environments, and unlock value... ...architecture, integration, and modelling. • Pipeline Ownership:... ...• Team and Technology Lead: Evaluate, pilot, and integrate new big...Local areaVisa sponsorship- ...advanced hardware, software, and AI technology to make it possible... ...of sleep into a personalized, data-driven recovery experience. We... ...we need a world-class Data Engineer to power our next phase of hypergrowth... ...expertise in data modeling and warehousing—you think in terms...Full timeWork at officeImmediate startWorldwideSleeping nightsNight shift
- ...Reference Number: 25-00661Title: DATA ENGINEERPosted Date: 2025-06-1... ...Skills:Proficiency in data engineering programming languages (preferably... ...and stream), one or more data modelling techniques (Dimensional, Data... ...APIsKnowledge of Generative AI concepts (such as RAG, Vector...
$125k - $145k
...we're buildingOur team runs a range of data and AI projects across the GPS business.A few... ...product, servicing, and client questions.•Models that optimize pricing and foreign-... ...forThis is a hybrid role that spans data engineering, AI product enablement, and platform architecture...Full timeWork at officeFlexible hoursDay shift- ...investment management firm seeking a Senior Data Engineer to help transform and modernize its... ...environments into Snowflake.Design scalable data models and enterprise warehouse solutions.... ....Experience building semantic layers and AI-ready data platforms.If you meet the required...Work at officeImmediate startWork from home
$76.73k - $127.88k
...collaborative environment? As an experienced Data Engineer, you will have the ability to share new... ...with Deloitte under our Project Talent Model. Project Talent Model (PTM) is a talent model... ...operational outcomes The TeamAI & Data - AI & Engineering leverages cutting-edge...Local areaVisa sponsorship- ...WithAt Slalom, we co-create custom software, data, and cloud products with clients who are... ...true change.Our Data Architecture & Engineering team sees data as the foundation for decision... ..., and transformation. We embrace AI to scale with quality, innovate without fear...Temporary workLocal area
- QUALIFICATIONSDegree in Computer Science, Business Analytics, Engineering, Mathematics, or related field2+ years of professional experience in data engineering, software engineering, or... ...such as (Databricks, SageMaker, Vertex AI), MLOps best practices, and GenAI toolkits (...ApprenticeshipEasy work
$190k - $220k
...more about ourTotal Rewards philosophy.AI is a fundamental part of how work gets done... ...during the interview process.The Data Engineering team builds tools and systems that make... ...event streams.Strong foundation in data modeling, schema design, and data quality best practices...Full timeWork at officeLocal area2 days per week3 days per week$170k - $230k
...If you're energized by shaping data infrastructure from zero to one... ...-Funnel Agentic Intelligence & Models) — the team building the next generation of agentic AI advertising products- Pipelines... ...reporting that scales spanning Engineering, Science, PM-T, and Design across...Flexible hours$180k
Title: Data EngineerLocation: New York, NY - Hybrid 3x per weekClient: Leading US financial... ...company is looking for a hands-on Data Engineer to join their IT Infrastructure team. This... ...data lakehouse migrationsFamiliarity with AI coding tools (Cursor, Claude Code, etc.)Compensation...- ...Capital and Private EquitySelling Points Lead cutting-edge AI and data engineering initiatives in a dynamic environment. Collaborate with experts... ...architectural guidelines. Operate within a federated data model to ensure scalability and consistency across platforms. Build...
$182k - $242k
CoreWeave is The Essential Cloud for AI. Built for pioneers by... ...more at .What You'll Do:The Data Engineering Team builds and operates the... ...ecosystem. You will establish the modeling, semantic, governance, and... ...broader platform strategy.Evaluate and standardize core data technologies...Permanent employmentFull timeTemporary workCasual workWork at officeFlexible hours- ...leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are... ...Cohere is a team of researchers, engineers, designers, and more, who are all... .... As a member of our Analytics & Data Insights team, you'll tackle the...Work at officeLocal areaRemote workHome office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - AI Model Evaluator. Be the first to apply!
Related searches
- data center engineer New York, NY
- senior cloud data engineer New York, NY
- data infrastructure engineer New York, NY
- sr information security engineer New York, NY
- hadoop big data developer New York, NY
- data visualization developer New York, NY
- associate data engineer New York, NY
- data engineer freelance New York, NY
- remote data engineer New York, NY
- etl data engineer New York, NY