Remote Software Engineer, AI Benchmarking & Evaluation
Epoch AI
New York, NY
- Remote job
Epoch AI is seeking a Software Engineer to develop and evaluate AI benchmarks. This role involves implementing benchmarks, creating new evaluations, and collaborating with researchers to provide trustworthy insights. The ideal candidate has over two years of experience in software engineering and a passion for AI. This fully remote position offers competitive pay and flexible work hours. Benefits include comprehensive health insurance, generous paid time off, and opportunities for professional development. #J-18808-Ljbffr Epoch AI
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Remote Software Engineer, AI Benchmarking & Evaluation in New York, NY vacancy
- 24-MAG LLC invites experienced Senior Software Engineers to join a part-time, remote consulting engagement focused on evaluating AI-generated software, code benchmarking, and research-driven verification. You will curate code, craft reference solutions, and build automated...Remote jobPart time
- ...Role Overview Evaluate and benchmark the coding abilities of frontier AI models by reviewing AI-generated solutions,... ...validating them against real-world software engineering tasks, and shaping high-... ...next week. Work location: Remote, United States only. Perks:...Remote workContract workFor contractors
SaidGig
United Statesmore than 2 months ago $50 per hour
...Role Overview Evaluate, benchmark, and help improve the coding capabilities of advanced AI models by assessing AI-generated solutions against real software engineering problems. You will review code for correctness... ...week. Work location, remote, candidates must be located...Remote workContract workFor contractorsSaidGig
United States3 days ago- ...Overview Design and evaluate high-quality datasets... ...multiple languages, assess AI-generated... ...capabilities across the software engineering lifecycle. Key Responsibilities... ...model training and benchmarking. Evaluate AI-... ...fit. Location: Remote, candidates must be based...Remote workFor contractors10 hours per weekFlexible hours
SaidGig
Canadamore than 2 months ago - ...high-quality datasets and evaluation pipelines that improve and benchmark large language models for code generation and software engineering tasks. You will curate... ...code, evaluate and refine AI-generated solutions... ...Work Terms Location: Remote. Engagement type: Contractor...Remote workFull timeFor contractors10 hours per weekFlexible hours
SaidGig
United Statesmore than 2 months ago - ...partner is looking for a Senior Software Engineer – LLM Evaluation based in United States.... ...solutions, and evaluate AI-generated software against... ...and contribute to benchmarks that make AI evaluation more... .... You will collaborate remotely with technical and research...Remote workFull timePart timeFor contractors10 hours per weekFlexible hours
jobgether
United States4 days ago - ...world's fastest-growing AI companies,... ...feedback, and rigorous evaluation frameworks. Our contributors... ..., agentic workflows, software engineering, and multimodal AI,... ...coding-agent benchmark tasks using production... ...Work in a fully remote environment. Opportunity...Remote workContract workTemporary workFor contractorsFreelanceWorldwide
Turing
Remote29 days ago - Turing is seeking a Software Engineering Evaluator to create cutting-edge datasets for training and benchmarking large language models. You will curate code examples and provide precise... ...Rust, and Go as needed. You will evaluate AI-generated code for efficiency, scalability,...Remote job
- ...Responsibilities Review and refine AI-generated prompts,... ...Validate algorithms and software concepts for technical... ...difficulty, or language Support benchmarking efforts to evaluate and compare model... ...Strong experience in software engineering, technical research, or...Remote workPart time
Crossing Hurdles
United States1 day ago $100 - $150 per hour
...Apply your software engineering expertise to assess AI-generated slides, spreadsheets, and documents for real-world... ...clarity. Key Responsibilities Evaluate AI-generated work products in a software... ...is preferred. Work Terms ~ Remote, hourly engagement....Remote workHourly payWork at officeSaidGig
United States17 days ago- To support a research initiative on AI evaluation tasks, remote software engineers will review and assess programming challenges and evaluation harnesses, providing feedback on their technical accuracy and realism in a contract role. Key responsibilities Review and assess...Remote workContract work
$70 - $150 per hour
...Role Overview Help shape how frontier AI systems are trained and evaluated by applying production software engineering judgment to real-world code, designs, requirements... ..., and security engineering. Work Terms Remote, hourly engagement. This listing is a standing...Remote workHourly payImmediate startSaidGig
United States24 days ago$50 per hour
...Overview Help build and evaluate the datasets and... ...models for real-world software engineering. You will collaborate... ...solutions, and correct AI-generated code across... ...industry performance benchmarks. Design and build... ...Terms Location: Remote. Engagement type:...Remote workFull timeFor contractors10 hours per weekFlexible hoursSaidGig
United States3 days ago$197.3k - $225.1k
AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability)At Capital One, we are creating responsible and reliable AI systems, changing banking... ....* Design, develop, test, deploy, and support AI software components including foundation model training,...Full timePart timeLocal area- ...steps. Our partner is looking for a Software Engineer L5/L6 — Model Evaluations & Data Curation (MEDC) based in the... ...infrastructure that accelerates how AI teams create, evaluate, and improve... ...for full-time salaried employees. ~ Remote work opportunity within the United States...Remote workFull timeFlexible hours
jobgether
United States3 days ago - ...Role: AI Evaluations Engineer (Senior) Location: US (remote) Duration: 12 Months + Key Responsibilities Design and operate end-to-end... ...-Judge framework; HITL evaluation pipelines Benchmarking frameworks (tau-bench, SWE-bench, GAIA) and custom...Remote work
VeeRteq Solutions Inc.
New York, NY2 days ago $50 per hour
A leading AI research organization is seeking PhDs in Chemistry or related fields for a remote contract. The role involves designing advanced problems to... ...Responsibilities include developing solutions, evaluating AI outputs, and refining benchmarks. The pay rate is $50+/hour,...Remote jobContract work$100 - $150 per hour
...technical talent with leading AI research labs.... ...our investors include Benchmark , General Catalyst ,... ...Dorsey . Position: Software Engineer Expert Type: Contract... ...50/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts...Remote workContract workSummer workWork at officeMercor
San Francisco, CA10 days ago$70 - $90 per hour
...technical talent with leading AI research labs.... ...our investors include Benchmark , General Catalyst ,... ...90/hour Location: Remote Role Responsibilities Evaluate the quality, correctness... ...and reproducibility of software-engineering benchmark tasks . Assess...Remote workContract workSummer work$85 per hour
...technical talent with leading AI research labs.... ...our investors include Benchmark , General Catalyst ,... ...DevOps / SRE / Cloud Engineer (Coding Agent Experience... ...5/hour Location: Remote Role Responsibilities... ...agents to complete and evaluate complex infrastructure...Remote workContract workSummer workMercor
San Francisco, CA10 days ago$45 - $75 per hour
...Software Engineer – Coding Benchmarks & RL Environments Experience: Professional Software Engineering... ...Job Type: Contractor Location: Remote Compensation: $45–$75/hour... ...realistic coding tasks used to train and evaluate frontier AI coding agents . The core...Remote workFor contractors10 hours per weekCodefeast
United States3 days ago- Senior Software Engineer, ApplicationsAcuityMD is the AI platform for MedTech trusted by over 500... ...backed by Stepstone Group, Benchmark, Redpoint Ventures,... ...professional experience when evaluating candidates.Obsessed... ...-skilling opportunities.Remote work: AcuityMD is committed...Remote workFor contractorsWork at officeWork from homeHome officeFlexible hours
AcuityMD
Boston, MA4 days ago $184k - $287.5k
...unlimited potential of AI to define the next era... ...for an experienced Software Engineer to develop our core libraries... ..., and stability.Benchmark the latest agents to identify... ...quality—backed by evaluations, benchmarking, and... ..., TX, Austin; US, NC, Remote; US, TX, Remote; US, NY...Remote workFull timeNvidia
Santa Clara, CA5 days ago$160k - $180k
...About the RoleAs a Senior Software Engineer on the MarTech... ..., building production AI agents that solve real... ...sufficient for success in a remote and largely... ...reliability, observability, evaluation, and cost efficiency of... ...regression suites, and offline benchmarks wired into CI/...Remote workFull timeLocal areaJones Lang LaSalle
Chicago, IL2 days ago$160k - $190k
...agentic coding skills and AI-native workflows (prompt engineering, Spec-Driven... ...the Role Senior Software Engineer to lead development... ...datasets. Build evaluation harnesses and cross-provider benchmarks for cost, latency,... ...~ Ability to work remotely and set your own hours...Remote workFull timeFor contractorsImmediate startVibehackers
Eastern, KY2 days ago- ...research accelerator for frontier AI labs and a trusted partner... ...who specialize in software engineering, logical reasoning, STEM, multilinguality... ...As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large...Remote workFor contractorsFlexible hours
Turing
United States3 days ago - ...for a Senior Full Stack Software Engineer to build and evolve... ...AWS Serverless AI-Assisted Engineering... ...graceful degradation Evaluate what you ship. For AI-... ...regression suites, and quality benchmarks; monitor for... ...software. This is a fully remote position. Life Line...Remote workContract work
Life Line Screening
Cleveland, OH1 day ago $145k - $175k
...Full-Stack Software Engineer We are looking for a Full-Stack... ...systems like AI-powered pricing intelligence... ...ultimate goal and evaluating effort and priority in... ...independent, third-party benchmarking against national... ...performance) ~30-day remote work from anywhere...Remote workFull timeWork at officeRelocationRelocation packagePlug Technologies Inc
Los Angeles, CA2 days ago- ...Project Overview Evaluate and refine AI-generated code to ensure that it is efficient, scalable... ...against industry performance benchmarks. Build agents that can verify the quality... .... Hypothesize on steps in the software engineering cycle (prototyping, architecture design...Remote workFull timeFor contractorsFlexible hours
Turing
Dallas, TX4 days ago $175k - $215k
...Sr. Full-Stack Engineer Team: Engineering... ...building the most AI-native startup team... ...senior full-stack software engineer who will... ...ultimate goal and evaluating effort, risk, and... ...independent, third-party benchmarking against national... ...) ~30-day remote work from anywhere...Remote workFull timeWork at officeRelocationRelocation packagePlug Technologies Inc
Los Angeles, CA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote Software Engineer, AI Benchmarking & Evaluation. Be the first to apply!
Related searches
- software engineer full time New York, NY
- graduate software developer no experience New York, NY
- software support engineer New York, NY
- software developer apprenticeship New York, NY
- software engineer healthcare New York, NY
- network software engineer New York, NY
- software engineer co-op New York, NY
- software engineer internship New York, NY
- senior software engineer New York, NY
- software system engineer New York, NY