Data Engineer for AI Model Evaluation
$80 per hourSaidGig
Role Overview
Help evaluate and improve frontier AI coding models by using AI coding agents to complete realistic data engineering tasks, then assess the outputs for correctness, scalability, and failure modes. Work centers on end-to-end data engineering workflows including ETL pipelines, data warehouses, analytics platforms, and distributed data systems.
Key Responsibilities- Use frontier AI coding agents to implement and test complex data engineering tasks.
- Review and validate model-generated code and configurations for ETL pipelines, data warehouses, analytics platforms, and distributed data systems.
- Identify bugs, edge cases, scalability bottlenecks, and failure modes in model outputs.
- Compare solutions produced by multiple frontier models and document relative strengths and weaknesses.
- Apply professional engineering judgment to realistic data infrastructure and pipeline scenarios, producing clear, actionable feedback.
- Minimum 2 years of professional data engineering experience.
- Hands-on experience building ETL pipelines, data warehouses, analytics platforms, or distributed data systems.
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or comparable tools.
- Ability to evaluate model-generated implementations of data infrastructure and pipelines.
- Experience operating large-scale data platforms is preferred.
- Location: Remote.
- Employment type: hourly.
- Sprint-based engagement, with work organized in 12 to 24 hour stretches based on client requirements.
- Spots are limited and assignments are filled on a first come, first serve basis.
- Metadata hourly rate: $80 per hour.
- Task-based pay: $400 per accepted task.
- Typical accepted tasks require approximately 2 to 3 hours of work after ramp-up.
- Payment is tied to accepted work, compensation is awarded only for tasks that meet acceptance criteria.
- Open to applicants who meet the listed qualifications.
- Because spots are limited, apply promptly; assignments are allocated on a first come, first serve basis.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Data Engineer for AI Model Evaluation in United States vacancy
- Role Description We are hiring a Senior Solutions Engineer to help shape and scale our AI Data Solutions, working with leading AI labs, frontier model developers, and enterprise AI teams on complex data, evaluation, and model development workflows. This is a senior, customer...SuggestedFull timeFor contractors
L10n People Ltd
Remote1 day ago $60 - $90 per hour
...Role Overview Drive research-grade data analyses and create evaluation tasks that reveal where frontier generative AI models fail. You will author realistic, multi-skill analysis... ...of experience in a research, research-engineering, or intensive data-analysis role....SuggestedHourly payFull timeFreelanceRemote workSaidGig
United States7 days ago$238k - $302k
...across 15+ U.S. states. The Large Model Evaluation team is at the nexus of Waymo’s AI ambition . With advancements in... ...for quantitatively-minded engineers to research and propose new ways... ...development and deployment. Build data pipelines for signal discovery, data...SuggestedFull timeRemote workWaymo
San Francisco, CA11 hours ago$85 per hour
...Role Overview Evaluate and improve frontier AI coding models by completing structured technical assessments that mirror realistic machine learning engineering workflows, model training and inference systems, MLOps, and LLM application scenarios. Key Responsibilities...SuggestedHourly payRemote workSaidGig
United States2 days ago$120 per hour
...Government Quantitative Professionals evaluate AI model outputs and provide structured, domain-... ...quantitative roles such as Mathematician or Data Scientist. Relevant real-world... ...techniques to practical problems in science, engineering, business, or security, and sharing...SuggestedHourly payTemporary workPart timeImmediate startRemote workFlexible hoursSaidGig
United States6 days ago- ...Role Overview Medical professionals evaluate AI-generated medical content and use their clinical and field experience to improve model outputs. No prior AI experience is required. In this role you will assess model responses, create realistic prompts that reflect clinical...Hourly payPart timeImmediate startRemote workFlexible hours
SaidGig
United States6 days ago $208k - $300k
...Machine Learning Engineer - Model Evaluations, Public Sector The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, and multimodal pipelines... ...PyTorch. Background in algorithms, data structures, and object-oriented programming...Full timeScale Ai
Washington DC11 hours ago$40 - $65 per hour
...Contribute domain expertise to evaluate and harden frontier large language models by crafting adversarial... ...how next-generation AI systems learn, reason,... ...prior experience in human data environments such as... ..., annotation, or prompt engineering. Preferred: deep familiarity...Remote jobHourly payFor contractorsSaidGig
Frontier County, NE1 day ago$39 per hour
...where you will play a crucial role in evaluating generative musical AI models in collaboration with a leading AI... ...the quality of musical training data. Write music in the specified domain... .../mix quality, using audio-engineering terminology. Annotate songs in detail...Hourly payPart timeImmediate start10 hours per weekSaidGig
United States25 days ago$60 - $80 per hour
...foundational Large Language Models, by designing realistic... ...solutions, and evaluating model outputs with rigorous... ...you within a leading AI lab''s extended workforce... ...Guide research and engineering teams to close knowledge... ...and accuracy in training data. Qualifications At...Hourly payWeekday workSaidGig
United States6 days ago$30 - $90 per hour
...Build and maintain backend services in Go while testing and evaluating alpha-stage AI coding models. This part-time, remote contract role combines hands-on... ...endpoints using a modern Go stack Perform rigorous data validation and implement comprehensive error handling...Hourly payContract workPart timeFor contractorsRemote workSaidGig
United States2 days ago$20 per hour
...DataAnnotation is committed to creating quality AI. Join our team to help train AI chatbots while... ...chatbots. You will develop complex prompts to test AI models, write high-quality responses to demonstrate excellence, and evaluate different model outputs based on accuracy and...Hourly payFull timeContract workPart timeFor contractorsSelf employmentFreelanceRemote workDataAnnotation
Wyoming, OH4 days ago$75 per hour
...Role Overview Lead the clinical evaluation of medical AI by designing and executing assessments that... ...realistic clinical problems, find where models fail or lack medical knowledge, and... ...ready systems by combining high-quality data, training pipelines, and expert...Contract workRemote workFlexible hoursSaidGig
United States1 day ago$60 per hour
A tech company focused on AI is seeking quantitative professionals to evaluate AI-generated analytical work and help advance AI development. Responsibilities include statistical analysis, predictive modeling, and providing feedback to shape AI systems. The role offers flexible...Hourly payRemote workFlexible hoursDataAnnotation
Helena, MT2 hours ago$60 per hour
A leading AI development company is seeking quantitative professionals to evaluate AI-generated analyses and develop solutions in various quantitative fields. This fully remote position allows for flexible scheduling and competitive pay up to $60/hour. Candidates should...Remote workFlexible hoursDataAnnotation
Providence, RI4 days ago$40 per hour
A leading AI training company is seeking a Quantitative Researcher to enhance AI models through rigorous evaluation and problem-solving in mathematics. This remote role involves measuring chatbot progress and assessing performance across various mathematical fields. Ideal...Hourly payRemote workFlexible hoursDataAnnotation
Jackson, MS4 days ago$60 per hour
A leading AI development firm is seeking experienced quantitative professionals to contribute... ...to AI systems. The role involves evaluating AI-generated work and solving technical problems... .... Candidates should have a background in data science, economics, or similar fields,...Hourly payRemote workFlexible hoursDataAnnotation
Nashville, TN4 days ago$60 - $150 per hour
...Role Overview Provide legal subject-matter expertise to improve and evaluate AI systems, by designing realistic legal tasks, reviewing model outputs, and giving domain-specific feedback that advances frontier AI research. This is an open application to join a Law Expert...Hourly payContract workImmediate startRemote workSaidGig
United States4 days ago$85 per hour
...GIS analysis, and renewable energy siting to evaluate AI-generated geospatial outputs and develop expert training data that improves AI understanding of environmental... ...Analyst, Interconnection Specialist, Solar Design Engineer, or equivalent positions in the energy...Hourly payContract workPart timeWork at officeRemote workFlexible hoursSaidGig
United States6 days ago$40 per hour
A data annotation company is seeking professionals in quantitative fields to enhance AI development. This fully remote role allows individuals to set flexible schedules while evaluating AI-generated analyses and solving complex quantitative problems. Candidates should have...Hourly payRemote workFlexible hoursDataAnnotation
Oregon, WI3 days ago$40 per hour
A leading AI development firm in Michigan is seeking experienced quantitative professionals to evaluate AI-generated analyses and contribute to the evolution of AI models. Candidates should have a background in data science, statistics, or similar fields, with at least...Hourly payRemote workDataAnnotation
Lansing, MI4 days ago$40 per hour
...A forward-thinking AI team is seeking quantitative professionals to evaluate and improve cutting-edge AI systems. The role involves working on AI-generated analyses and providing critical feedback for model enhancement. Candidates should have a strong quantitative background...Hourly payRemote workFlexible hoursDataAnnotation
Helena, MT11 hours ago$60 per hour
...developing cutting-edge AI systems, while enjoying... ...AI development. AI models are increasingly capable... ...AI models on tasks like evaluating AI-generated quantitative... ...systems reason about data, models, and scientific... ...Science, Mathematics, Engineering, or similar); a master'...Hourly payFull timeRemote workFlexible hoursDataAnnotation
Little Rock, AR4 days ago$40 per hour
A data science team is seeking experienced quantitative professionals to evaluate AI-generated work and contribute to the development of cutting-edge AI systems. This fully remote position offers flexible scheduling and competitive hourly pay starting at $40+. Ideal candidates...Hourly payRemote workFlexible hoursDataAnnotation
Madison, WI3 days ago$110 per hour
...Role Overview Join a Physician Expert Network to provide clinical expertise to AI research teams and companies. Members contribute medical knowledge to train and evaluate AI models, design realistic clinical tasks, and give domain-specific feedback that advances medical...Hourly payContract workRemote workSaidGig
United States10 days ago$40 per hour
...An innovative AI development company is seeking experienced quantitative professionals to contribute to AI advancements. This fully remote role involves evaluating AI-generated analyses and ensuring they are technically accurate and valid in real-world scenarios. Candidates...Hourly payRemote workFlexible hoursDataAnnotation
Santa Fe, NM11 hours ago$40 per hour
A leading AI development company is seeking experienced quantitative professionals for a remote role. Candidates will evaluate AI-generated quantitative work, solve complex problems, and provide valuable feedback. The ideal candidate has 2+ years of experience in a quantitative...Hourly payFull timeRemote workFlexible hoursDataAnnotation
Topeka, KS4 days ago$80 - $110 per hour
...expertise at the frontier of physics to help develop and evaluate next-generation AI systems for scientific reasoning. You will apply active research... ...physics problems used to train and evaluate advanced AI models. Evaluate model outputs on tasks including analytical...Hourly payPart timeImmediate startRemote workSaidGig
United States7 days ago- ...Physicians apply clinical judgment and frontline medical experience to evaluate AI-generated medical content, ensuring clinical accuracy, sound... ...planning. Assess clarity, relevance, and safety of model outputs in realistic care scenarios. Provide detailed, constructive...Full timeFor contractorsPrivate practiceRemote workFlexible hours
SaidGig
United States6 days ago $40 per hour
...A leading AI development firm is seeking experienced quantitative professionals to evaluate AI-generated analysis and solve complex technical problems. Ideal candidates will have 2+ years in quantitative roles, knowledge of statistical methods, and experience with analytical...Hourly payRemote workFlexible hoursDataAnnotation
Oklahoma City, OK4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer for AI Model Evaluation. Be the first to apply!
Related searches
- big data cloud engineer United States
- big data engineer United States
- big data developer United States
- remote data engineer United States
- bi data engineer United States
- big data devops engineer United States
- finance data engineer United States
- sr information security engineer United States
- junior big data engineer United States
- entry level big data engineer United States