Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Software Engineer, Coding Evaluation & Training Data

Full-time

Surge AI

About Us

Our mission is to raise AGI with the richness of human intelligence — curious, witty, imaginative, and full of unexpected brilliance.

Surge was founded by engineers and researchers who dreamed of building the next generation of AI. Our platform powers the most capable models in the world, in partnership with the AI models you use every day.

At Surge, we believe the path to AGI isn't just about scaling compute—it's about embracing the unlimited ceiling of human intelligence and creativity in the data that shapes these systems. Our platform combines elite human expertise with cutting-edge tools for scalable oversight, from building rich RL environments to conducting rigorous evaluations that go beyond benchmarks. We've run a profitable business from day one without raising venture funding.

The Role

As a Software Engineer, Coding Evaluation & Training Data at Surge, you'll sit at the intersection of software engineering and product to build and run the systems that teach frontier models how to code.

You’ll own end-to-end coding data projects for top AI labs, designing tasks, RL environments, and evaluation schemes that reflect real-world software engineering. This is an ideal role for a software engineer who wants to keep using their engineering skill set every day, but is most excited by training data quality, agentic evaluation, and system design (humans + models + tools) rather than traditional product feature work.

What You'll Do

  • Own end-to-end coding data projects, from initial scoping and pilot design through execution, iteration, and scale-up.

  • Design agentic training workflows and task structures that mirror real-world SWE work (e.g., refactoring, debugging, code review, large-repo navigation, tool use).

  • Define and iterate on rubrics, golden sets, and reward signals that capture true engineering value, not just “does it compile.”

  • Evaluate data and worker output with strong SWE taste, and make calls about what meets the bar for frontier training.

  • Design and run qualification processes for coding workers, including hands-on assessments of their coding ability.

  • Set up or partner on complex technical environments (e.g., containers, repos, test harnesses, sandboxes, code execution infrastructure).

  • Partner with technical staff at our clients to translate high-level training goals into concrete projects and technical environments.

  • Collaborate closely with Surge engineering, product, and operations to improve our coding data products, internal tools, and execution processes.

What We're Looking For

  • 2–6+ years of professional software engineering experience building and maintaining real systems.

  • Strong coding ability in at least one mainstream language and comfort working in production codebases.

  • High “taste” for good engineering: you care about correctness, code quality, and how real engineering teams actually work.

  • Ability to reason about and debug technical environments, including containers, dependencies, and automated test setups.

  • Interest in owning projects end-to-end, including scoping, workflow design, execution, and continuous improvement.

  • Excellent written and verbal communication skills, with the ability to speak credibly with senior client engineers and translate fuzzy goals into actionable plans.

  • Excitement about AI/ML systems and the role of data, evaluation, and reward design in improving agentic coding capabilities.

Vacancy posted 7 days ago
Similar jobs that could be interesting for youBased on the Software Engineer, Coding Evaluation & Training Data in Remote vacancy
  • $174k - $252k

     ...blocks, such as streaming evaluation, autorater pipelines,...  ...secure per-customer data isolation and access...  ...of experience with coding in Python.3 years of...  ...Python and C++.Google's software engineers develop the next-...  ...relevant education or training. US: $174000 - $25200... 
    Data
    Training

    Google

    New York, NY
    3 days ago
  •  ...professional services and engineering talent solutions....  ...background in software engineering and...  ...specialized AI security evaluation project. In this...  ...You will analyze code paths, identify...  ...evaluation and AI training TECHNICAL FOCUS:...  ...and trace relevant data flows, action paths... 
    Data
    Training
    Contract work
    Remote work

    Gramian Consulting Group

    Remote
    8 days ago
  •  ...and improve AI-generated code, build software checks, and evaluate coding systems across the...  ...and improve code used to train and benchmark AI models....  ...three years of software engineering experience and strong skills...  ...more reliable training data and AI coding systems. #J... 
    Data
    Training
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • $204k - $259k

     ...states. The Planner Evaluation team works on one of...  ...the quality of the software that drives the car....  ...looking for experienced data-minded software engineers and data scientists...  .... Champion code health and best practices...  ..., relevant training and education, and skill... 
    Data
    Training
    Full time
    Remote work

    Waymo

    San Francisco, CA
    3 days ago
  •  ...Evaluate and improve AI-generated code for accuracy and quality. About Turing Turing is one of the world...  ...quality human feedback, evaluation, and training data. Role Overview We are looking for experienced Software Engineers to help evaluate, benchmark, and... 
    Data
    Training
    Contract work
    Part time
    For contractors
    Freelance
    Remote work

    Turing

    San Francisco, CA
    a month ago
  •  ...research with high-quality data, advanced training pipelines, plus top AI...  ...researchers who specialize in software engineering, logical reasoning, STEM,...  ...As a Software Engineering evaluator, you will create cutting-edge...  .... This includes curating code examples, providing... 
    Data
    Training
    Contract work
    For contractors
    Flexible hours

    Turing

    Remote
    6 days ago
  •  ...research with high-quality data, advanced training pipelines, plus top AI...  ...researchers who specialize in software engineering, logical reasoning, STEM,...  ...As a Software Engineering evaluator, you will create cutting-edge...  .... This includes curating code examples, providing... 
    Data
    Training
    Contract work
    For contractors
    Flexible hours

    Turing

    Remote
    6 days ago
  • $100 - $150 per hour

     ...accelerate model development through high-quality training data, evaluations, and engineering talent. Engagement Details...  ...We’re looking for experienced, hands-on software engineers to help evaluate and improve AI coding models . Rather than primarily building... 
    Data
    Training
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start
    Remote work

    Turing

    Remote
    7 days ago
  • $135k - $155k

     ...to create operational software that empowers educators...  ...in specific areas. Evaluation is our teacher and staff...  ...are hiring a Software Engineer III to own significant...  ...-powered features, the data integrations that connect...  ..., and maintainable code, and help strengthen our... 
    Data
    Full time
    Remote work

    Education Advanced, Inc.

    Remote
    18 days ago
  • $104k - $208k

     ...leveraging real-world software engineering expertise and open-...  ...optimization, and technical evaluation. For this project,...  ...intelligence training project focused on high...  ...scalability, and overall code quality. Optimize...  ...advanced algorithms, data structures, and... 
    Data
    Training
    Hourly pay
    Contract work
    For contractors
    Remote work

    ELITEBRAIN

    Remote
    5 days ago
  • $208k - $315k

     ...AI is the frontier AI data lab, helping teams build...  ...About The Role-Senior Software Engineer (Req # 722)  Snorkel...  ...tasks. The Evaluation Engineering team owns...  ...building high-quality post-training and benchmark datasets...  ...used globally across coding, mathematics, law, medicine... 
    Data
    Training
    Full time
    Local area
    Remote work

    Snorkel AI

    Remote
    5 days ago
  •  ...looking for a Sr. Full-Stack Software Engineer based in United States. Join an AI training project where your full-stack...  ...technical problem-solving, debugging, code evaluation, and cross-language...  ...alongside the services, APIs, and data models that support them.... 
    Data
    Training
    Hourly pay
    Part time
    For contractors
    Work at office
    Remote work
    Flexible hours

    Jobgether

    Remote
    3 days ago
  • $170k - $200k

     ...automatically optimizes cloud data warehouses for...  ...We're an AI-first engineering team. We use...  ...As a Senior Software Engineer, you'll design...  ...data Use AI coding tools effectively...  ...move faster Evaluate new tools and models...  ...compensation, and training. We strive to create... 
    Data
    Training
    Full time
    Remote work
    Home office
    Visa sponsorship

    Keebo

    Remote
    10 days ago
  • $99k - $125k

     ...Purpose This Senior Software Engineer will build next generation...  ...a source of direction, training and guidance for less...  .... Reviews progress and evaluates results. Recommends changes...  ...of specifications, coding of new programs, physical design of data records, data files, and... 
    Data
    Training
    Full time
    Work experience placement
    Work at office
    Afternoon shift

    Save A Lot

    Remote
    11 days ago
  • $150k - $215k

     ...looking for a Senior Software Engineer to provide technical leadership...  ...’s Scanning Engine & Data Collection team. This...  ...workflows, ensuring code is maintainable, well-...  ...scanning platform, evaluating tradeoffs and...  ...recruitment, selection, training, promotion, transfer,... 
    Data
    Training
    Full time
    Fixed term contract
    Work experience placement
    Work at office
    Remote work
    Home office
    Flexible hours

    Coalition, Inc.

    Remote
    14 days ago
  • $104k - $208k

     ...initiatives that leverage real-world software engineering expertise to improve the...  ..., system architecture, and code evaluation across multiple programming...  ...development to support AI training through complex coding...  ...services, APIs, and data models. Analyze and troubleshoot... 
    Data
    Training
    Hourly pay
    Contract work
    For contractors
    Remote work
    Worldwide
    Flexible hours

    ELITEBRAIN

    Remote
    5 days ago
  • $125k - $180k

     ...-driven financial software company that aims...  ...includes inspired data scientists, financial...  ...As a Senior Engineer on the True Integrations...  ...working with LLM coding agents (Claude) to...  ..., tool calling, evaluation and guardrails. You...  ...company meetings, training, offsites, and collaborative... 
    Data
    Training
    Full time
    Fixed term contract
    Work at office
    Local area
    Remote work
    Flexible hours

    TrueML

    Remote
    6 days ago
  • $105k - $185k

     ...Job Title: Full Stack Software Engineer Job Description The...  ...services. Design, code, benchmark test, debug...  ...documentation, test scripts, and training materials to support...  ...with external data sources and SQL Server...  ..., screening, and evaluating candidates. AI helps assess... 
    Data
    Training
    Permanent employment
    Contract work
    Interim role
    Remote work

    Actalent

    Dahlgren, VA
    13 hours ago
  • $286k - $312k

     ...labs are building coding agents that do real engineering work. Before these...  ...pull request: run it, evaluate it. Pair with...  ...straight back to the labs training the next best-in-...  ...of professional software engineering experience...  ...fastest-growing AI data business in history... 
    Data
    Training
    Weekly pay
    Full time
    Contract work

    Handshake

    Remote
    6 days ago
  • $75k - $90k

     ...hiring an Associate Software Engineer to build the...  ...agents, MCP servers, orAI coding tools such as Claude Code...  .... Experience training users or running user...  ...where AI fits, how we evaluate it, and how we deploy...  ...including how apps share data, sign users in, and connect... 
    Data
    Training
    Full time
    Temporary work
    Internship
    Work at office
    Local area
    Flexible hours

    Valiant Solutions, LLC

    Remote
    3 days ago
  • Write, debug, and optimize code, and evaluate AI-generated solutions for correctness,...  ...efficiency, readability, security, and engineering quality. Create challenging...  ...and have a solid understanding of data structures, algorithms, software design, and systematic debugging.... 
    Data
    Training
    Ongoing contract
    Freelance
    Remote work
    Flexible hours

    Alignerr Corp.

    Portland, OR
    4 days ago
  •  ...accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality,...  ...precise, self-contained questions. Experience evaluating AI-generated scientific responses or working... 
    Data
    Training
    Contract work
    For contractors
    Freelance
    Remote work
    Flexible hours

    Turing

    New York, NY
    3 days ago
  •  ...a good chance our software is working behind...  ...Native Senior Software Engineer is a change agent...  ..., including coding agents, spec-driven...  ...customer feedback and data to decide what...  ...industry experience and training. ~6+ years of...  ...building and evaluating AI harnesses, MCP... 
    Data
    Training
    Full time
    Work from home

    Versaterm

    Remote
    7 days ago
  • $100.7k - $167.8k

     ...The Role As a Software Engineer on Bloomerang's PDLC Operations...  ...Figma, Pendo, and our data platforms such as...  ..., help content, and training from verified sources,...  ...traceability between code and work items Document...  ...classification, and evaluation of AI answer quality... 
    Data
    Training
    Permanent employment
    Full time
    Remote work
    Relocation package
    Flexible hours

    Bloomerang

    Remote
    11 days ago
  •  ...looking for a Senior Software Engineer to join a small, technically...  ...modeling, spectral data analysis, and machine...  ...workflows — including training data preparation, model integration, and evaluation — into operational...  ...workflow diagrams, and code structure so tools can... 
    Data
    Training
    Immediate start
    Remote work

    GRVTY

    Dayton, OH
    3 days ago
  • $167.8k - $246.8k

     ...the Role As a Senior Software Engineer, Machine Learning, you...  ...will use our robust data and machine learning infrastructure...  ..., offline model training, online shadowing/...  ...in using AI coding tools (e.g., Claude Code...  ...our recruiting team by evaluating job related qualifications... 
    Data
    Training
    Hourly pay
    Work at office
    Local area
    Flexible hours

    DoorDash USA

    San Francisco, CA
    3 days ago
  • $105.7k - $160.3k

     ...and maintainable software solutions. Working...  ...makes significant engineering decisions within the...  ...frameworks, APIs, data patterns, software...  ...issues; and evaluates trade-offs involving...  ...and AI agents for coding, testing, analysis...  ...factors, such as: training, transferable skills... 
    Data
    Training
    Temporary work
    Work experience placement
    Summer work
    Casual work
    Internship
    Seasonal work
    Work at office
    Remote work
    Work from home
    Work visa
    Flexible hours

    Compeer Financial

    Mankato, MN
    5 days ago
  • $131k - $170k

     ...Senior Software Engineer - Land.com Job Description Overview...  ...engineering judgment to evaluate where AI creates...  ...SOLID principles, secure coding practices, and sound quality...  ...scripts, application data access patterns, and...  ...growth with internal training, and tuition reimbursement... 
    Data
    Training
    Full time
    Work at office
    Local area
    Work from home

    CoStarGroup

    Austin, TX
    11 days ago
  • $91.62k - $134.38k

     ...Assurance Automation Engineer, you will work...  ...collaborate closely with software development...  ...support test creation, code review,...  ...and test context, evaluate AI-generated suggestions...  ...usage, sensitive-data handling, intellectual...  ...unconscious bias training for our leaders to... 
    Data
    Training
    Full time
    Remote work
    Relocation
    Flexible hours

    Entrust

    Shakopee, MN
    4 days ago
  • $98.16k - $159.27k

     ...Description: The Software Engineer-III (Full Stack/API Integration...  ...; includes planning evaluating recommending designing...  ...development tools and coding assistants to improve...  ...application and data issues and cross-capability...  ...including cross training within own team Keeps... 
    Data
    Training
    Full time
    Work at office
    Local area
    Work from home
    Flexible hours
    Shift work

    TD Bank

    Mount Laurel, NJ
    25 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineer, Coding Evaluation & Training Data. Be the first to apply!