Systems Engineer - AI Model Evaluator [Remote]
$85 per hourMercor
New York, NY
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Systems Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex engineering tasks.
- Review model-generated code, architecture decisions, and systems implementations.
- Identify bugs, edge cases, performance bottlenecks, and failure modes.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic systems engineering scenarios.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- 2+ years of professional systems engineering experience.
- Experience with distributed systems , networking , operating systems , storage systems , infrastructure software , or database internals .
- Regular use of AI coding agents such as Cursor , Claude Code , Codex , Windsurf , Gemini CLI , or similar tools.
- Ability to evaluate model-generated systems designs and implementations.
Preferred
- Experience building highly scalable and performance-critical systems.
Compensation & Legal
- $400 per accepted task
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Systems Engineer - AI Model Evaluator [Remote] in New York, NY vacancy
$400 per month
...contributors for a project with a leading AI research lab. The role involves using... ...agents to finalize and assess complex engineering tasks. Contributors will review code,... ..., and compare outputs from different models. Strong systems engineering experience is essential,...Suggested$60 per hour
...to developing cutting-edge AI systems, while enjoying the flexibility... ...advance AI development. AI models are increasingly capable of... ...art AI models on tasks like evaluating AI-generated quantitative analysis... ...Science, Mathematics, Engineering, or similar); a master's or...SuggestedHourly payFull timeRemote workFlexible hoursDataAnnotation
Brooklyn, NY17 hours ago$40 per hour
A leading AI development company is seeking experienced quantitative professionals to evaluate AI-generated quantitative analysis and solve technical problems. This fully remote... ...while contributing to cutting-edge AI systems. The ideal candidate will have hands-on experience...SuggestedHourly payRemote workDataAnnotation
New York, NY17 hours ago- ...A leading AI development firm is seeking experienced quantitative professionals to evaluate AI-generated analyses and contribute to training robust AI models. This remote role offers flexibility and competitive... ...impact on the future of AI systems designed for data reasoning....SuggestedRemote work
DataAnnotation
New York, NY17 hours ago $40 per hour
A cybersecurity technology company is seeking experienced professionals to evaluate AI-generated security content and design technical solutions. This remote role offers flexibility with project selection and scheduling. Ideal candidates should possess over 2 years of hands...SuggestedRemote jobHourly pay$40 per hour
A technology firm is seeking experienced cybersecurity professionals to evaluate AI-generated security content and solve technical challenges in a remote capacity. Candidates should have over 2 years of hands-on experience in various cybersecurity fields, strong writing...Remote jobHourly pay$60 per hour
A leading firm in AI development is seeking experienced quantitative professionals to evaluate AI-generated analyses and shape future AI models. This role is fully remote, offering flexible scheduling... .... Join us to help advance AI systems built for reasoning and analytics....Remote jobFlexible hours- A leading AI development company is seeking experienced quantitative professionals for remote work evaluating AI-generated quantitative analysis. Ideal candidates will have a robust... ...technical feedback to enhance AI systems. Join us to shape the future of analytics...Remote job
$40 per hour
...cybersecurity sector is seeking experienced professionals to evaluate AI-generated security content. In this remote role, you will work... ...schedule, solve technical cybersecurity problems, and help train AI models. Preferred candidates will have at least 2 years of experience...Remote jobHourly payFlexible hours$40 per hour
A leading AI training company is seeking experienced cybersecurity professionals to evaluate AI-generated security content and solve technical problems. This role allows you... ...choice of projects, fostering contributions to AI system improvements. Candidates must have at least 2...Remote jobHourly pay- A leading data annotation company is seeking an Aerospace Engineer to join their remote team. This role involves training AI models by measuring progress, evaluating logic, and solving problems to improve model quality. A Master's or PhD in Aerospace Engineering or related...Remote jobHourly payFlexible hours
$40 per hour
A leading AI development firm is seeking quantitative professionals to evaluate AI projects and provide feedback. You will work remotely from anywhere in the US, Canada... ...flexible working hours and the chance to shape future AI systems. #J-18808-Ljbffr DataAnnotationRemote jobHourly payFlexible hours$40 per hour
A leading AI development company is seeking experienced quantitative professionals to join their remote team. The role involves evaluating AI-generated work, designing quantitative problems, and providing impactful feedback. Candidates with at least 2 years of quantitative...Remote jobHourly pay- ...cybersecurity services company is seeking experienced professionals to evaluate AI-generated security content and solve technical cybersecurity... ...coding skills. Responsibilities include assessing AI security models and providing critical feedback. A preferred bachelor's degree...Remote job
$40 per hour
...experienced cybersecurity professionals for a remote position. Responsibilities include evaluating AI-generated security content and solving technical challenges to train AI models. Candidates should have at least 2 years of hands-on cybersecurity experience, some coding...Remote jobHourly payFlexible hours$40 per hour
A dynamic AI development company is seeking experienced quantitative professionals in Georgia to evaluate AI-generated work and provide vital feedback. This role is fully remote, allowing candidates from various countries to contribute flexibly. Ideal applicants will have...Remote jobHourly pay$40 per hour
...tech firm is seeking experienced quantitative professionals to evaluate AI-generated analyses and enhance AI development. The role offers... ...skills. This opportunity impacts the evolution of AI systems focused on reasoning and data analytics. Applicants can work alongside...Remote jobHourly pay$40 per hour
A leading AI development company is seeking experienced quantitative professionals to evaluate AI-generated analysis and shape future AI systems. The role offers fully remote work with a flexible schedule and competitive pay starting at $40+ per hour. Candidates should...Remote jobHourly payFlexible hours$40 per hour
A leading technology company is seeking experienced cybersecurity professionals to help train AI models. In this remote role, you will evaluate AI-generated security content and solve technical cybersecurity problems. Candidates must have 2+ years of hands-on experience...Remote jobHourly pay$40 per hour
A cybersecurity firm is seeking experienced professionals to evaluate AI-generated security content and improve AI systems. Candidates should have 2+ years in cybersecurity, including roles like penetration testing and incident response. The position offers flexible hours...Remote jobHourly payFlexible hours$40 per hour
...cybersecurity firm is looking for experienced professionals to evaluate AI-generated security content. In this role, you will solve... ...technical cybersecurity problems and provide feedback to improve AI systems. Candidates should have over 2 years of experience in...Remote jobHourly pay- A tech company specializing in AI is seeking an Aerospace Engineer to join their team in the United States. The role involves training AI models by measuring progress, evaluating outputs, and solving diverse problems to enhance model quality. Candidates should have an expert...Remote jobHourly payFlexible hours
$40 per hour
...firm is looking for experienced cybersecurity professionals to evaluate AI-generated security content and solve technical problems.... ...US, Canada, UK, Ireland, Australia, and New Zealand, with opportunities to shape AI security models. #J-18808-Ljbffr DataAnnotationRemote jobHourly payFlexible hours$40 per hour
A leading cybersecurity firm is seeking experienced professionals to evaluate AI-generated security content and solve technical problems. This remote role offers flexible scheduling and project selection, with hourly pay starting at $40+. Candidates should have 2+ years...Remote jobHourly payFlexible hours- A cybersecurity technology firm is seeking experienced cybersecurity professionals to train AI models. This role involves evaluating AI-generated cybersecurity content, solving technical problems, and providing essential feedback. Candidates should have over 2 years of...Remote jobFlexible hours
- ...cybersecurity technology firm is seeking experienced professionals to evaluate AI-generated security content and solve technical cybersecurity... ...and providing feedback to shape future AI security models. Ideal candidates should have over 2 years of hands-on cybersecurity...Remote jobFull timePart time
$40 per hour
A leading AI development firm is seeking experienced quantitative professionals to evaluate AI-generated analyses and design quantitative challenges for AI systems. This role allows for fully remote work from locations including the US and Canada. Candidates should have...Remote jobHourly payFlexible hours- A leading AI training company is seeking medical experts to evaluate AI chatbots' responses to complex healthcare problems in a REMOTE position. Applicants need to hold a medical degree or be in-progress towards one. Responsibilities include ensuring medical accuracy of...Hourly payRemote work
DataAnnotation
New York, NY2 days ago $30 per hour
...A prominent AI data company is seeking Advanced Mandarin Speakers for evaluating AI models and completing training tasks. This remote role offers competitive pay of $30/hr for tasks requiring one hour of focused work. Candidates should be fluent in Mandarin with the ability...Remote workWork from homeFlexible hoursProlific - UK Job Board?
New York, NY4 days ago$40 per hour
...data annotation company is seeking a Statistician to join their team. This remote role involves training AI models by posing complex mathematical problems, evaluating outputs, and assessing the model's performance. Candidates must be detail-oriented and proficient in...Remote jobHourly pay
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Systems Engineer - AI Model Evaluator [Remote]. Be the first to apply!
Related searches
- visual systems engineer New York, NY
- system support engineer New York, NY
- systems project engineer New York, NY
- system engineer contract New York, NY
- application system engineer New York, NY
- system test engineer New York, NY
- senior windows systems engineer New York, NY
- lead system engineer New York, NY
- system performance engineer New York, NY
- senior staff systems engineer New York, NY