Software Engineer | AI Training Data & Evals Lab

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Engineer | AI Training Data & Evals Lab based in United States.

This is a broad engineering role at the intersection of platform development, AI evaluation, and experimentation infrastructure.
You will build and operate production systems that researchers and operators rely on to develop and assess frontier AI capabilities.
Your work will include evaluation harnesses, backend services, data pipelines, training environments, and tools that make experimentation more reliable and repeatable.
You will have meaningful ownership from the start, with an expectation of delivering production improvements and becoming a trusted owner of core systems.
The environment is lean, async-first, and highly collaborative, with an emphasis on clear communication, sound technical judgment, and dependable execution.
You will work on practical engineering challenges closely connected to AI research while helping transform expert work into high-quality training and evaluation data.
This opportunity is well suited to a strong builder who enjoys autonomy, distributed systems, and working on infrastructure that directly influences how AI systems are evaluated and improved.

• Build and maintain evaluation harnesses that measure the performance of AI models and agents on real-world tasks.
• Improve evaluation reliability, coverage, and signal quality through better rubrics, task design support, and scoring approaches.
• Develop tools that enable researchers and operators to run experiments efficiently without repeatedly rebuilding the same workflows.
• Build and maintain APIs and backend services supporting human-in-the-loop workflows, task routing, and quality-control processes.
• Improve data pipelines that transform expert work into structured training and evaluation datasets.
• Strengthen system observability, scalability, and operational reliability through effective logging, metrics, monitoring, and debugging capabilities.
• Write clear, maintainable production code and actively participate in code reviews, architecture discussions, and technical design decisions.
• Document technical decisions and system behavior clearly so that other engineers and collaborators can build upon and operate the systems effectively.
• Take ownership of core systems from development through production operation, with an expectation of delivering meaningful improvements within the first 30–90 days.

Requirements:

• Strong software engineering fundamentals with professional experience in Node.js and TypeScript.
• Strong coding ability in Python and/or Go.
• Demonstrated experience building, deploying, and owning production systems, including APIs, backend services, and data pipelines.
• Solid understanding of distributed systems, scalability, reliability, and engineering trade-offs.
• Experience working with AWS or GCP and modern infrastructure technologies such as containers and Kubernetes.
• Proven track record of shipping and maintaining production systems that other people depend on, rather than working exclusively on prototypes.
• Strong written communication skills and the ability to collaborate effectively in an asynchronous, distributed environment.
• Comfortable taking ownership of ambiguous technical problems, making sound engineering decisions, and following projects through to production.
• Experience with evaluation frameworks, experimentation platforms, or machine-learning tooling is a plus.
• Experience with data pipelines, workflow orchestration, or internal platforms for research and operations teams is a plus.
• Experience working in early-stage environments or high-ownership B2B SaaS and platform teams is a plus.

Benefits:

• Full-time, fully remote position with a LATAM focus and meaningful overlap with U.S. time zones.
• Compensation of $7,000–$10,000 USD per month, based on experience.
• Significant ownership and opportunities to grow into larger systems, deeper technical leadership, and projects central to the organization’s growth.
• Lean, async-first working environment focused on clear writing, sound judgment, and strong follow-through.
• Opportunity to work on research-adjacent engineering challenges at the frontier of AI while building practical production platforms.
• Direct impact on the training data and evaluation systems used by leading AI labs.
• Structured hiring process including a practical take-home assignment, team review, technical screen, real-world work trial, and final offer stage.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1