Senior Python Engineer , AI Agents Evaluation
Temporary
Turing
About Turing
Turing is one of the world's leading AGI infrastructure companies, working with frontier AI labs to accelerate model development through high-quality training data, evaluations, and engineering talent. We are staffing a frontier AI data initiative building the infrastructure and training data used to develop and evaluate AI agents. You will be assigned to one of two tracks: connectors or tasks.
About the Role
We are looking for experienced Python Engineers to work with leading AI research labs on evaluating and improving next-generation AI coding agents. You will review agentic coding trajectories, assess technical correctness, debug and validate outputs, and use coding agents such as Claude Code, Codex, or Cursor as part of your workflow.
What You'll Do
- Review agentic coding trajectories, including prompts, tool calls, intermediate actions, code changes, execution results, and final outputs.
- Evaluate whether AI agents correctly understand and execute software engineering tasks.
- Analyze agent behavior to identify technical errors, inefficient approaches, and opportunities for improvement.
- Read, debug, test, and validate Python code produced or modified by AI agents.
- Work with LLM-powered agents, workflows, and development environments to evaluate model capabilities.
- Use coding agents such as Claude Code, Codex, Cursor, or similar tools to accelerate analysis, debugging, and validation.
Requirements
- 5+ years of professional software engineering experience, with strong hands-on expertise in Python.
- At least 6 months to 1 year of practical AI/LLM engineering experience building agents, agent loops, LLM-powered applications, data pipelines, or similar systems.
- Hands-on experience using AI coding agents such as Claude Code, Codex, Cursor, or equivalent tools as part of regular software development workflows.
- Ability to understand and evaluate agentic workflows, including tool usage, intermediate execution steps, code modifications, and resulting outputs.
Perks of Working With Turing
- Work with leading AI research labs and contribute to the development of cutting-edge AI systems.
- Fully remote opportunity.
- Collaborate with a global network of talented engineers and AI professionals.
- Flexible engagement with exposure to frontier AI development and evaluation.
Offer Details
- Engagement: Full-time contractual opportunity
- Location: Remote
- Availability: 40 hours per week with required overlap with the global team
- Contract Duration: 1 month
Evaluation process
- Around 25 mins of AI interview
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Senior Python Engineer , AI Agents Evaluation in Remote vacancy
- ...companies, working with frontier AI labs to accelerate model... ...high-quality training data, evaluations, and engineering talent. We are staffing a... ...to develop and evaluate AI agents. You will be assigned to one... ...are looking for experienced Python Engineers to work with leading...SeniorFull timeContract workTemporary workRemote workFlexible hours
Turing
Remote2 days ago $80 - $100 per hour
...build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding... ...experience (non-negotiable) ~ Expert Python — clean, performant, well-... .... Nice to have Senior or Lead-level profile with a...SeniorRemote jobFull timeContract workFor contractorsG2i
Miami, FL8 hours ago$80 - $100 per hour
...client is seeking experienced Senior Python Developers to help build the... ...the client who is a leading AI platform that enables... ...high-quality human feedback, AI evaluation, and model alignment. The... ...opportunity for experienced software engineers who enjoy solving complex...SeniorHourly payWeekly payContract workRemote work10 hours per weekLifted, an Upwork Company™
Texas City, TX10 days ago- ...applications and next steps. Our partner is looking for a Senior Automation Engineer (Python, AI Agents & Automation) based in United States . This role... ...that Jobgether will process your personal data to evaluate your candidacy and share relevant information with...SeniorFull timeRemote work
jobgether
United States5 days ago - ...world's fastest-growing AI companies, accelerating... ...feedback, and rigorous evaluation frameworks. Our... ...agentic workflows, software engineering, and multimodal AI,... ...build realistic coding-agent benchmark tasks using production... ...experience in Python (mandatory) along with...SeniorContract workTemporary workFor contractorsFreelanceRemote workWorldwide
Turing
Remote2 days ago $45 - $50 per hour
...Summary The role of the Senior Python Engineer is to design, develop, and deploy machine learning... ...to-end data workflows, and integrating AI capabilities into broader application... ...engineering to support model accuracy Evaluate model performance and iterate to...SeniorHourly payFull timeWork at officeRemote workWork visaFormativgroup
Remote8 hours ago$204k - $259k
...Driver Understanding and Evaluation (DUE) team at Waymo is... ...components such as agents and a realistic simulator... ...researchers and software engineers who are passionate... ...modeling and generative AI into robust, production... ...programming skills in Python and experience with ML...SeniorFull timeWaymo
Remote8 hours ago$118.8k - $205.6k
...with speed and confidence.The Nexpart engineering team builds and evolves the... ...this marketplace. We are seeking a Senior Python Engineer, AI to help design, develop, and maintain... ...improves reliability and accuracyDevelop evaluation and quality frameworks for AI outputs...SeniorImmediate startRemote workVisa sponsorship$216k - $270k
About Scale AIScale AI is the data foundation... ...build production AI agents that automate... ...one of the hardest engineering challenges.As a Senior Frontier Agent Engineer... ...customers to design, evaluate, and deploy intelligent... ...experience.Strong Python programming skills.Experience...SeniorFull time- ...designing and scaling reusable AI-driven workflows,... ...language models and autonomous agents. You will build foundational... ...emphasizes system reliability, evaluation rigor, and thoughtful tradeoffs... ...components used broadly across engineering teams. Key Responsibilities...SeniorFull time
Calliere
Remote8 hours ago $170k - $260k
...this level of clearance. Are you a Senior Python Software Engineer who is ready for a new challenge that... ..., Analytics, Machine Learning, AI, and DevOps Containerization. We also... ...from the current manual system security evaluation and authorization process to a new model...SeniorFull timeContract workRemote workWork from homeRelocation packageShift workGliaCell Technologies LLC
Linthicum Heights, MD5 days ago$144.7k - $261.3k
...DescriptionAbout the TeamThe Evaluation Foundations team, part of Embodied AI's Scaling Foundations,... ...vehicle development. We engineer high-performance tools... ....About the RoleAs a Senior Engineer in the Embodied... ...key technologies:Backend: Python, node.js, high-throughput...SeniorFull timeLocal areaRemote workWork from homeRelocation packageFlexible hours$200k - $400k
...leading conversational AI platform empowering every... ...Douglas to deploy AI agents that power personalized... ...experimental, frontier-style engineering. The team continuously... ...loops through offline evaluation and online... ...About the Role As a Senior Software Engineer on the...SeniorFull timeWork at officeLocal areaDecagon
New York, NY8 hours ago- ...: we are building LLM evaluation and training datasets to... ...on realistic software engineering problems. One of our approaches... ...’s fastest-growing AI companies accelerating... ...following languages: Python Proficiency with... ...developer tools or automation agents. Perks of...SeniorContract workFor contractorsFreelanceRemote work
- ...Planera to build Manny, our AI scheduling assistant,... ...platform. You will own agent features end to end:... ...LangGraph/LangChain agent, engineering prompts and tools,... ...a high bar with a real evaluation and observability stack... ...Strong proficiency in Python building production services...SeniorRemote work
Planera Inc
United States2 days ago $202.5k - $247.5k
...sharing localhost or running AI workloads in production... ...time. About the Agent Team Our Agent... ...runs entirely on AWS. Engineers develop by using remote... .... Compensation Senior Software Engineer... ...actual compensation will be evaluated based on factors including...SeniorPermanent employmentFull timeWork at officeLocal areaImmediate startRemote workHome officeFlexible hoursNgrok
Remote8 hours ago$170k - $220k
...TetraScience is the Scientific Data and AI Company building Tetra OS, the... ...Role We are seeking a highly skilled Senior .NET Engineer to join our dynamic team. The ideal candidate... ...compensation is determined by a thorough evaluation of factors including the candidate’s...SeniorFull timeContract workImmediate startRemote workVisa sponsorshipFlexible hoursTetrascience
Remote8 hours ago$141.02k - $204.53k
...secure your future. Responsibilities As the Senior AI/ML Engineer - Validation & Evaluation within AI Validation & Monitoring (AVM), you will... ...processing, and Generative AI, with proficiency in tools like Python, TensorFlow, PyTorch, sci-kit-learn, Keras, etc....SeniorFull timeWork at officeRemote workFlexible hoursWeekend workMayo Clinic
Rochester, MN4 days ago$204k - $259k
...of-the-art Generative AI to create a training... ...Driver. The Simulator Evaluation team faces the... ...We are looking for a Senior Software Engineer to build the metrics... ...throughput pipelines (C++/Python) capable of processing... ...or experience with agent-based modeling and behavioral...SeniorFull timeRemote workWaymo
San Francisco, CA8 hours ago- ...Netomi is hiring a Senior Product Manager to own the product... ...roadmap for Autonomous Agents in the conversational AI domain. This role is responsible... ...Data Science to define evaluation criteria, benchmark... ...functional alignment across Engineering (Runtime, Studio, Channels...SeniorFull time
Netomi
Remotea month ago - General Motors, through Embodied AI, seeks a Senior Engineer to measure and visualize AV model performance. You will design and implement evaluation workflows, collaborate across Data, Infra and validation teams, and connect offline evaluation with real-world behavior....SeniorRemote job
- ...acts like a teammate: a 24/7 AI threat hunter that... ...Position Overview As a Senior Backend Engineer - Agents at Nebulock, you will work... ...Impact Continuous iteration, evaluation, and cross-team collaboration... ...fundamentals ~ Proficiency in Python , Java , Go , Rust ,...SeniorFull time
Nebulock
Remote24 days ago - Geisinger is seeking a Senior BI Analyst to partner with AI Evaluation & Monitoring, collecting, cleaning, and interpreting data to support data-driven decisions... ...8+ years of relevant experience, strong SQL and Python skills, and experience with production monitoring, AI...SeniorRemote job
- Rex.zone is hiring a Senior Data Annotator to produce and review high-quality training data for AI systems across NLP, computer vision, and multimodal evaluation. This is a Remote, full-time role focused on annotation guidelines compliance, content safety labeling, and...SeniorRemote jobFull time
- ...Senior AI Agent Engineer We are Wizeline is a global AI-native technology solutions provider that... ...environments (TypeScript/Node, Go, or Python) with strong API design, testing, and... ...function/tool calling, model selection, and evaluation - Retrieval-Augmented Generation (RAG...SeniorFull timeFlexible hours
Wizeline
Remote3 days ago $224k - $356.5k
...future of autonomous driving, and evaluation is how we know the drive is getting... ...how our organization develops AI drivers!We are looking for a senior engineer to own the engine that makes it all... ...data pipelines behind them.Strong Python and data engineering skills for production...SeniorFull timeRemote work- ...technology products.As a Senior Lead Software Engineer - ML Engineer for Agent Platform at JPMorgan... ...communicate, remember, and get evaluated in a secure, stable, and... ...enterprise-authorized AI-assisted engineering... ...programming language(s); strong Python plus a systems language...Senior
$242k - $290k
...developing Foundation Models for ML Agents and planning, applying them... ...and Planning Machine Learning Engineer you will work on the bleeding... ...pipelines Fluency in Python with a basic understanding of... ...use artificial intelligence (AI) tools to support parts of the...SeniorFull timeTemporary workRelocation packageZoox
Remote8 hours ago- ...ITTConnect is seeking an experienced Senior AI / ML Engineer (Python) to work for a client that is a... ...workflows (prompt engineering, RAG, evaluation) for voice and chat use cases Lead... ...with Agentic AI and multi-agent orchestration Experience with LLMOps...SeniorFull time
Ittconnect
Remote8 hours ago $60 per hour
...contributors for a part-time QA project focused on autonomous AI agents. This flexible remote opportunity requires strong analytical and... ...with structured data formats. Candidates will review evaluation tasks, identify inconsistencies, and help define expected AI behaviors...Part timeRemote workFlexible hoursMind Rift
Kansas City, MO3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Python Engineer , AI Agents Evaluation. Be the first to apply!
Related searches
- python developer part time Remote
- full stack / python developer (remote) Remote
- entry level python developer Remote
- python developer data analytics Remote
- python developer remote Remote
- python engineer Remote
- python programmer Remote
- work from home python developer Remote
- freelance python developer Remote
- senior python developer Remote