AI Evaluation Engineer for Coding Agents
Repovive, Inc.
##### ###### ##### ### # # ### # # ######## ## ## ## ## ## ## # # # # # ####### #### ##### # # # # # # # ###### # ## ## ## ## # # # # # #### # ###### ## ### # ### # ###### $ curl repovive.com/jobs/69ed18d7682d4cf1d9e87166 ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ █████████████████████████████████████████████████ ##### ###### ##### ### # # ### # # ######## ## ## ## ## ## ## # # # # # ####### #### ##### # # # # # # # ###### # ## ## ## ## # # # # # #### # ###### ## ### # ### # ###### ##### ###### ##### ### # # ### # # ######## ## ## ## ## ## ## # # # # # ####### #### ##### # # # # # # # ###### # ## ## ## ## # # # # # #### # ###### ## ### # ### # ###### $ curl repovive.com/jobs/69ed18d7682d4cf1d9e87166 ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ █████████████████████████████████████████████████ ##### ###### ##### ### # # ### # # ######## ## ## ## ## ## ## # # # # # ####### #### ##### # # # # # # # ###### # ## ## ## ## # # # # # #### # ###### ## ### # ### # ###### $ curl repovive.com/jobs/69ed18d7682d4cf1d9e87166 ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ █████████████████████████████████████████████████ Repovive © 2025 Repovive, Inc. All rights reserved. Back to Jobs Apply Now Compensation Not listed Posted April 25, 2026 Required Skills AI evaluation data pipelines agent instrumentation Requirements Mid/Senior Visa Sponsorship Not mentioned Relocation Not mentioned About the Role Build evaluation and quality systems for Cursor's coding agents. Interested in this role? Apply directly on Cursor's website Apply for this Position #J-18808-Ljbffr
Vacancy posted 19 hours ago
Similar jobs that could be interesting for youBased on the AI Evaluation Engineer for Coding Agents in New York, NY vacancy
- ...the Team The Codex Core Agent team builds the kernel of Codex... ...We're looking for applied AI engineers to help bring Codex agents from... ...behaviors across real-world coding tasks and long-horizon workflows... ...better real-task data into evaluation and research. Work with...Suggested
OpenAI
New York, NY3 days ago - ...Title-Resources-Guaranty is seeking a product-minded AI Engineering Lead to build and scale production AI agents, driving significant operational improvements. You... .... The ideal candidate will possess strong coding expertise in C# and TypeScript/React and have hands...SuggestedRemote workFlexible hours
Title Resources Guaranty Company
New York, NY4 days ago - Jack & Jill in New York City is looking for an AI Engineer to join a high-caliber team and build a state-of-the-art coding agent. The role involves designing complex AI architectures, improving code quality, and scaling the AI stack. Ideal candidates should have 3+ years...Suggested
- BeaconFire Inc. is looking for an AI Developer specializing in building and productionizing Vertex AI-based RAG systems. You will design... ..., vector databases, and cloud services like AWS and GCP. Strong coding practices and collaboration are key, alongside a commitment to...Suggested
- ...OpenCall OpenCall's voice AI handles calls for multi... ...hiring an AI Prompt & Agent Developer to own... ...accelerate the work. Build evaluation harnesses. You'll develop... .... The best prompt engineers are good writers. You notice... ...calls." Comfort with code. You don't need to be a...SuggestedDay shift
- ...Senior AI Engineer — Inference & Agent Systems United States Title: Applied AI Engineer — Inference & Agent Systems Location: United States What... ...shipped inference systems at A real-time AI product (search, coding assistant, chat at scale) You've shipped inference...
Arcana Analytics Inc.
New York, NY3 days ago $175k - $230k
AI Chopping Block, Inc. is seeking a Customer Engineer, Agent Builder to execute AI agent builds for strategic customers. This highly technical role requires 5+ years... ...in a customer-facing position, with strong coding and API integration skills. The ideal candidate will...Flexible hours$175k - $230k
Decagon is hiring a Customer Engineer specializing in AI agent builds in New York. This role involves executing builds for enterprise customers and... ...facing technical role. Candidates should be comfortable with coding, APIs, and engaging with senior stakeholders. The...$150k - $200k
A leading advisory firm in New York is seeking a Lead Engineer specialized in AI-enabled product development. This role requires significant... ...with modern web frameworks, and the ability to orchestrate coding agents for efficient development. The ideal candidate will...$125k - $150k
...interest of our team. SUMMARY The AI Engineer builds agent-based systems and intelligent... ...support engineering workflows including code generation, test creation, documentation... ...Improve agent reliability through evaluation pipelines, observability metrics, and...Full timeWork at officeLocal areaAEG Presents
Brooklyn, NY3 days ago- ...A leading AI research accelerator in San Francisco is seeking experienced software engineers for a contract role. Responsibilities include evaluating AI-generated code and collaborating with teams to enhance coding solutions. Ideal candidates will have over 5 years of...Contract workRemote work10 hours per weekFlexible hours
Turing Inc
New York, NY4 days ago - ...About the job AI ENGINEER Role Description We're looking... ...architect and build the intelligent agents that power this system. You'... ...policy information Create evaluation frameworks and feedback loops... ...roadmap Collaborate on code reviews and technical design...Full timeRelocation
Barker Staffing Solutions, LLC
New York, NY5 days ago $73.8k - $220.4k
...Accenture's Global Responsible AI team within the Global Data &... ...if you're an experienced RAI Engineer with a Responsible AI background... ...practices. Detecting, evaluating, and applying relevant RAI dimensions... ...data preparation, design, coding, testing, deployment, and support...Work experience placementLive inWork at officeLocal areaAccenture
New York, NY2 days ago$200k - $300k
...AI Engineer Title of Role: AI Engineer Location: New York, onsite Company Stage... ...in prompt engineering and develop evaluation frameworks to enhance AI tooling. Proactively... ...of 1 year of hands-on experience coding in Python and deploying AI systems in production...Work at officeRecruiting from Scratch
New York, NY2 days ago$170k - $200k
...Get AI-powered advice on this job and more exclusive features. This... ...is looking for a Senior AI Engineer to help drive the next phase of... ...identify and implement AI use cases Evaluate and apply LLMs and other AI technologies beyond basic coding tools Help shape AI strategy...Full timeFreelanceRemote workQuantix Search
New York, NY4 days ago$142.32k - $213.48k
...information, please review .**Senior AI Engineer, Banking Technology – C13 (VP... ...LLM‐driven workflows and AI coding tools (e.g., Devin, GitHub... ...and improvement.**Evaluation & Optimization:** Define and... ...prototype, and integrate advances in agent‐based, autonomous, and generative...Full timeCitibank (Switzerland) AG
Jersey City, NJ1 day ago- ...Conversica is seeking a Senior AI Software Engineer to design, build, and scale production-grade... ...applied AI engineering, including agent evaluation, interpretability, data layer design,... ...lifecycle (i.e., during design, analysis, coding, deployment, QA, etc.) Strong...Full timeRemote work
Conversica
New York, NY4 days ago $35k
...Set: Python (advanced, production-grade coding) Generative AI (LLMs, prompt engineering, fine-tuning, RAG) Agent development using frameworks such as ADK,... ...Azure / GCP (especially AI/ML services) Model evaluation, guardrails, and Responsible AI practices...Full timeFor contractorsPhoton
New York, NY5 days ago- ...health! At dexter health, we build AI-powered software for care... ...looking for a high-agency AI Engineer to help us build new AI features... ...tools such as Claude Code, Codex, Cursor, Copilot, or similar... ..., and fallback behavior Build evaluation loops, tests, and quality checks...Remote work
dexter health
New York, NY4 days ago - ...Job Title AI Software Engineer Location US - Remote Job Type Fulltime Job Description... ...As a Software Engineering evaluator, you will create cutting‑... .... This includes curating code examples, providing precise... ...performance benchmarks. Build agents that can verify code quality...Full timeRemote work
SWITS DIGITAL Private Limited
New York, NY4 days ago $100k - $120k
...Job Description Junior AI Engineer Location: New York, US... ...function-calling patterns so agents can safely interact with internal... ...Write clean, maintainable code with strong engineering hygiene... ...documentation. Quality, evaluation, and responsible deployment...Full timeeClercx
New York, NY5 days ago$170k - $205k
...Senior AI Engineer Hybrid (NYC Metro) About SecurityScorecard:... ...are reviewing the production code that runs it. You will work at... ...latency, cost, output quality, and evaluation Use AI-native development... ...agentic AI systems, multi-agent orchestration, or retrieval-...SecurityScorecard
New York, NY3 days ago- ...About the AI Strike Team The AI Strike Team exists to reimagine... ...improvements in both engineering velocity and product quality... ...security reviews, and automated code quality agents that maintain stability... ...increasing development velocity. Evaluated and integrated modern AI...Work at officeRemote workNight shift
hyperexponential
New York, NY4 days ago $150k - $250k
...About Distyl AI Distyl is an applied AI technology... ...AI systems using Evaluation-Driven Development -an... ...production. AI Evaluation Engineers focus on designing and... ...production Python code, build evaluation pipelines... ...inform prompt design, agent logic, model selection,...3 days per weekDistyl AI
New York, NY1 day ago- ...seeking a highly skilled Senior AI Engineer with deep expertise in... ...Qdrant. Implement and Orchestrate Agents: Utilize frameworks like MCP,... ...cost optimization, and model evaluation. Work with AWS Services:... ...Gateway, IAM, CloudWatch. Strong coding ability in Python or similar...Local area
Quantanite
New York, NY4 days ago - ...Feedinkoo is looking for experienced software engineers to contribute their expertise to AI evaluation and improvement projects. This role involves applying software engineering skills to assess AI systems with no prior AI experience required, as training will be provided...FreelanceRemote work
Feedinkoo
New York, NY1 day ago $150k - $300k
...Job Title AI Engineer Salary $150k – $300k + Equity Company Description Vibecode - Seed-stage... .... You will build a state‑of‑the‑art coding agent that transforms natural language into native... ...‑based coding AI agent. Develop robust evaluation frameworks to measure and improve code...Jack & Jill
New York, NY20 hours ago- A leading technology firm is seeking an experienced software engineer to enhance AI-driven coding solutions. This role involves evaluating AI-generated code, collaborating with teams, and designing verification mechanisms. Candidates should have over 5 years of experience...Contract workRemote work10 hours per weekFlexible hours
Turing Inc
New York, NY1 day ago - ...architect and implement the core AI pipeline that powers Accel's... ...system generates. • On the engineering side, you'll build and... ...everything together. • Beyond pure coding, you'll be expected to think... ...output quality and building evaluation steps, catching failure modes...Internship
Accel Learning
Secaucus, NJ3 days ago - ...Perplexity is looking for an Applied AI Engineer to design, build, and iterate on cutting-edge agents powering our core experience in... ...: data analysis, modeling, evaluation, offline/online A/B testing,... ...) and experience using agentic coding tools for large scale parallel...
Perplexity AI Inc.
New York, NY19 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Engineer for Coding Agents. Be the first to apply!
Related searches