Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Data Engineer - AI Model Evaluator

$400 per month

Mercor Inc

About the Role Mercor is partnering with a leading AI research lab to support a Frontier Code Agents project. Contributors help evaluate and improve frontier AI coding models through structured technical assessments. The work focuses on realistic data engineering workflows and model evaluation. Spots are limited and filling quickly on a first come, first serve basis. What You'll Do Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated implementations involving ETL pipelines, data warehouses, analytics platforms, and distributed data systems. Identify bugs, edge cases, scalability issues, and failure modes. Compare outputs from multiple frontier models and assess their strengths and weaknesses. Apply professional engineering judgment to realistic data engineering scenarios. Time Commitment Sprint based project that runs in 12-24 hour stretches based on client requirement. Compensation $400 per accepted task. Typical tasks take approximately 2-3 hours after ramp-up. Compensation is tied to accepted work. Who Should Apply 2+ years of professional data engineering experience. Experience building ETL pipelines, data warehouses, analytics platforms, or distributed data systems. Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools. Ability to evaluate model-generated data infrastructure and pipeline implementations. Experience operating large-scale data platforms is preferred. #J-18808-Ljbffr Mercor Inc

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AI Model Evaluator in New York, NY vacancy
  • Mercor partners with a leading AI research lab to support a Frontier Code Agents project, focusing on realistic data engineering workflows and model evaluation. Contributors help evaluate and improve frontier AI coding models through structured technical assessments and... 
    Suggested

    Mercor Inc

    New York, NY
    3 days ago
  • Mercor is hiring Legal Experts to evaluate AI-generated responses for employment and labor law scenarios. This fully remote, hourly contract...  ...week. You will assess accuracy, provide feedback to improve model behavior and participate in calibration sessions. Requirements... 
    Suggested
    Remote job
    Hourly pay
    Contract work
    Flexible hours

    Intellerzone

    New York, NY
    3 days ago
  • Obsidian is seeking Insurance domain SMEs to join a cutting-edge AI training program. You will evaluate AI model outputs against real underwriting practice and rubrics, guiding research and engineering teams to close knowledge gaps in underwriting, claims, and risk assessment... 
    Suggested

    Obsidian

    New York, NY
    1 day ago
  • SME Careers is looking for a remote R Engineer to review AI-generated responses and create high-quality R and data-analysis content. The position requires a strong background in R programming, applied statistics, and excellent writing skills to document analyses. Candidates... 
    Suggested
    Remote job

    SME Careers

    New York, NY
    1 day ago
  • Data Scientist and Model Developer, Vice President (Contract) A temporary role...  ...in AML model development, AI/ML and GenAI. Location Kuala...  ...development, ongoing performance evaluation and annual model reviews....  ...in developing analytic engines with detailed technical documentation... 
    Suggested
    Contract work
    Temporary work

    Citi

    New York, NY
    1 day ago
  •  ...independent assurance and evaluating the company's risk management...  ....We are looking for data scientists and AI developers who will power our...  ...You will be deploying your engineering, data analytics and data science...  ...in frameworks for auditing models, including criteria like... 

    TikTok

    New York, NY
    3 days ago
  • $175k - $215k

     ...states. The mission of the Waymo AI Foundations team is to develop...  ...from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation. In this hybrid role, you...  ...Drive model development via data, eval, and systems Implement... 
    Full time
    Remote work

    Waymo

    New York, NY
    19 hours ago
  • $228.7k - $343.1k

     ...enormous scale, and one bad model can mean millions in...  ...to models applies to AI. We build the tooling that...  ..., so you critically evaluate what it produces and own...  ...write, learning the data, configs, and conventions...  ...Solid software and data engineering: production-quality... 
    Remote job
    Full time
    Local area
    Shift work

    Block

    New York, NY
    19 hours ago
  • $20 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...external tools. Generate high-quality human evaluation data by identifying response strengths, areas...  ..., and completeness of responses. Ensure model responses align with expected... 
    Remote job
    Contract work
    Part time
    Summer work

    Mercor

    New York, NY
    4 days ago
  •  ...advanced hardware, software, and AI technology to make it possible...  ...of sleep into a personalized, data-driven recovery experience. We...  ...we need a world-class Data Engineer to power our next phase of hypergrowth...  ...expertise in data modeling and warehousing—you think in terms... 
    Full time
    Work at office
    Immediate start
    Worldwide
    Sleeping nights
    Night shift

    Eight Sleep

    New York, NY
    1 day ago
  • QUALIFICATIONSDegree in Computer Science, Business Analytics, Engineering, Mathematics, or related field2+ years of professional experience in data engineering, software engineering, or...  ...such as (Databricks, SageMaker, Vertex AI), MLOps best practices, and GenAI toolkits (... 
    Apprenticeship
    Easy work

    McKinsey & Company

    New York, NY
    1 day ago
  • $180k

    Title: Data EngineerLocation: New York, NY - Hybrid 3x per weekClient: Leading US financial...  ...company is looking for a hands-on Data Engineer to join their IT Infrastructure team. This...  ...data lakehouse migrationsFamiliarity with AI coding tools (Cursor, Claude Code, etc.)Compensation... 

    Talener

    New York, NY
    5 days ago
  • $190k - $220k

     ...more about ourTotal Rewards philosophy.AI is a fundamental part of how work gets done...  ...during the interview process.The Data Engineering team builds tools and systems that make...  ...event streams.Strong foundation in data modeling, schema design, and data quality best practices... 
    Full time
    Work at office
    Local area
    2 days per week
    3 days per week

    Gusto

    New York, NY
    5 days ago
  • $170k - $230k

     ...If you're energized by shaping data infrastructure from zero to one...  ...-Funnel Agentic Intelligence & Models) — the team building the next generation of agentic AI advertising products- Pipelines...  ...reporting that scales spanning Engineering, Science, PM-T, and Design across... 
    Flexible hours

    Amazon

    New York, NY
    2 days ago
  • $110k - $190k

    Senior Data Management Professional - Data Engineering - Entities Location New York Business...  ...and extracted datasets, evaluate quality and consistency,...  ...key role in shaping how AI, LLMs, rules-based extraction...  ..., prompt evaluation, or model-assisted data workflows.•... 
    Temporary work
    For contractors
    Work experience placement

    Bloomberg

    New York, NY
    2 days ago
  •  ...25546234Reference Number: 25-00476Title: Data EngineerLocation: New York, NYPosted Date:...  ...micro-batching, and stream), one or more data modelling techniques (Dimensional, Data Vault,...  ...AWS Bedrock APIs Knowledge of Generative AI concepts (such as RAG, Vector embeddings,... 

    HAN Staffing

    New York, NY
    2 days ago
  • $110k - $190k

    Senior Data Management Professional - Data Engineering - Commodities Data Location New York Business Area...  ...we’re responsible for onboarding, modelling and maintaining data that are fit...  ...leverages modern technologies including AI and machine learning to enhance... 
    Temporary work
    For contractors
    Work experience placement

    Bloomberg

    New York, NY
    2 days ago
  • $97.17k - $126.32k

     ...that mindset.About the Role:The Data Engineer designs, develops, and...  ...implement logical and physical data models that promote consistency,...  ...and development standards.• Evaluate emerging technologies and recommend...  ...Intelligence, analytics, and AI initiatives through modern... 
    Full time
    Local area
    Visa sponsorship

    New York State Technology Enterprise Corporation (NYSTEC)

    New York, NY
    2 days ago
  • $90k - $135.29k

     ...Program for Surgical Intelligence is seeking a Data Engineer to support the development and deployment...  ...execution of research and clinical AI projects across the Department of SurgeryTrack...  ....Familiarity with healthcare data models and standards (such as Epic Clarity, Caboodle... 
    Traineeship
    Local area
    Immediate start
    Flexible hours

    Mount Sinai Health System

    New York, NY
    5 days ago
  • $116.2k - $229.1k

     ...Summary Join Deloitte’s AI & Engineering practice and help...  ...technology platforms, modernize data environments, and unlock value...  ...architecture, integration, and modelling. • Pipeline Ownership:...  ...• Team and Technology Lead: Evaluate, pilot, and integrate new big... 
    Local area
    Visa sponsorship

    Deloitte

    New York, NY
    5 days ago
  • $123k - $215.25k

     ...annually + bonus + benefitsJob Function: Engineering & ArchitectureSchedule: Full...  ...experience engineering (Mobile and Web), AI/ML, marketing technology, enterprise communications...  ..., and automation, grounded in our data technology model that prioritizes data governance. It... 
    Worldwide
    Visa sponsorship

    American Express

    New York, NY
    3 days ago
  • $137.4k - $161.7k

     ...impact?West Monroe is seeking a Senior Data Engineer to join our Data & AI practice. In this role, you will...  ...Develop robust ELT/ETL processes, data models, orchestration workflows, and data...  ...clients, exercising sound judgment in evaluating outputs.Apply AI technologies (e.g.,... 
    Local area
    Immediate start
    Flexible hours

    West Monroe Partners

    New York, NY
    1 day ago
  •  ...investment management firm seeking a Senior Data Engineer to help transform and modernize its...  ...environments into Snowflake.Design scalable data models and enterprise warehouse solutions....  ....Experience building semantic layers and AI-ready data platforms.If you meet the required... 
    Work at office
    Immediate start
    Work from home

    Solomon Page

    New York, NY
    2 days ago
  •  ...WithAt Slalom, we co-create custom software, data, and cloud products with clients who are...  ...true change.Our Data Architecture & Engineering team sees data as the foundation for decision...  ..., and transformation. We embrace AI to scale with quality, innovate without fear... 
    Temporary work
    Local area

    Slalom

    New York, NY
    2 days ago
  • $125k - $145k

     ...we're buildingOur team runs a range of data and AI projects across the GPS business.A few...  ...product, servicing, and client questions.•Models that optimize pricing and foreign-...  ...forThis is a hybrid role that spans data engineering, AI product enablement, and platform architecture... 
    Full time
    Work at office
    Flexible hours
    Day shift

    Bank of America

    New York, NY
    1 day ago
  •  ...leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are...  ...Cohere is a team of researchers, engineers, designers, and more, who are all...  .... As a member of our Analytics & Data Insights team, you'll tackle the... 
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    New York, NY
    4 days ago
  • $182k - $242k

    CoreWeave is The Essential Cloud for AI. Built for pioneers by...  ...more at .What You'll Do:The Data Engineering Team builds and operates the...  ...ecosystem. You will establish the modeling, semantic, governance, and...  ...broader platform strategy.Evaluate and standardize core data technologies... 
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    5 days ago
  •  ...Skills:Informatica workflow design and implementation requiredAWS data engineering (Glue, Lambda, EventBridge Cradle, S3) highly preferredFormer...  ...up delivery, partnering with our Science team to deliver AI solutions, or helping accounting drill down to track our business... 
    Work at office

    Ampcus

    New York, NY
    1 day ago
  • Req ID:376263NTT DATA strives to hire exceptional, innovative and passionate individuals...  ...now.We are currently seeking a Senior Data Engineer - Snowflake to join our team in Remote, New...  .... We are one of the world's leading AI and digital infrastructure providers, with... 
    Full time
    Work at office
    Remote work
    Flexible hours

    NTT DATA

    New York, NY
    1 day ago
  • $140k - $348k

     ...translating designs into code, or iterating with AI. From idea to product, Figma empowers...  ...us!We are looking for an experienced Data Engineer to partner with our Data Science and Data...  ...deep understanding of Figma’s core data models and optimize data pipelines for scale.Partner... 
    Minimum wage
    Full time
    Local area
    Remote work
    Flexible hours

    Figma

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer - AI Model Evaluator. Be the first to apply!