Remote Python Engineer (AI Benchmarking & Task Design)
24-MAG
New York, NY
- Remote job
24-MAG LLC is offering a fully remote, full-time consulting opportunity for experienced software engineers with strong Python skills. You will design realistic, multi-step coding tasks and build verifiable Python reference solutions, including environment setup and tests. The role evaluates AI coding assistants and requires rigorous documentation and peer-review contributions. The ideal candidate has 1+ year in software or research engineering, strong Python, and fluent Git/IDE workflows. #J-18808-Ljbffr 24-MAG
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote Python Engineer (AI Benchmarking & Task Design) in New York, NY vacancy
$80 - $100 per hour
...You'll Be Doing Design and build the coding benchmarks and evaluation... ...to test frontier AI models on real software engineering work: Design coding... ...programming tasks — reasoning, debugging... ...) ~ Expert Python — clean, performant... ...Location: Fully remote — work from anywhere...Remote jobFull timeContract workFor contractorsG2i
Miami, FL1 day ago- ...seeking an Electrical Engineering & RF/Circuit Design Expert to design... ...graduate-level problems for benchmarking AI research. You will build challenging tasks using scikit-rf or... ...software experience, strong Python skills, and comfort with Linux and remote compute sandboxes....Remote work
$70 - $85 per hour
...Design graduate-level computational engineering challenges that evaluate how advanced AI systems use real scientific software... ...original tasks involving simulations... ...rigorous benchmark problems, not routine... ...data. Write Python-based problem... ...and remote compute sandboxes...Remote workHourly paySaidGig
United States17 days ago- 24-MAG LLC is offering a full-time remote consulting opportunity for experienced software engineers with strong Python development, debugging, and AI-assisted coding experience. You will design, implement, and review multi-step tasks to evaluate frontier AI models in realistic...Remote jobHourly payFull time
$60 - $90 per hour
...Role Overview Design and deliver realistic... ...-step software engineering challenges used... ...generative AI coding agents.... ...will create tasks that represent... ...engineering work across Python implementation,... ...evaluation, or benchmark and task... ...per week, fully remote within the United...Remote workHourly payFreelanceSaidGig
United States26 days ago$70 - $100 per hour
...Role Overview Design original, graduate-level... ...whether advanced AI systems can perform... ...focuses on creating tasks that require running... ...validators in Python. Test each problem... ...terminal environment with remote compute sandboxes.... ...Familiarity with benchmark or evaluation...Remote workHourly payContract workSaidGig
United States17 days ago- ...is offering a project-based opportunity to craft and evaluate AI coding tasks. You will build a realistic developer environment—a virtual... ...infrastructure, tickets, docs, and conversations—to test AI agents. You design tasks from intermediate states, define what constitutes a...
- ...AI Benchmark and Task Designer is a remote review track for evaluating AI outputs across ai benchmark and task designer creative review workflows. Reviewers grade craft, constraints, and tooling adherence; flag production-readiness issues; and document the corrected approach...Remote jobHourly payFor contractors10 hours per week
- ...goML is hiring a Backend Developer to design and build scalable APIs, backend services, and Python packages powering AI-driven platforms. You will work across Python ecosystems including FastAPI, Django, and Flask, building robust services and SDKs for internal and external...Remote work
goML
United States3 days ago - ...To support innovative AI projects, the part-time freelance Python Engineer will design and evaluate tasks for AI coding agents, ensuring they complete assignments safely and correctly while working remotely. Key responsibilities Build realistic developer environments...Remote workPart timeFreelance
Virtual Vocations Inc
United States6 days ago - ...To support innovative AI projects, the freelance Software Engineer - Python will design coding tasks for AI coding agents, creating Docker environments and writing pytest tests while working remotely on a project-based basis. Key responsibilities Invent realistic developer...Remote workFreelance
Virtual Vocations Inc
United States3 days ago - ...Python Engineer What if your Python expertise could... ...influence how AI writes code for millions... ...This is a fully remote, flexible contract... ...that set the benchmark for AI outputs... ...language features, design patterns, and performance... ...of meaningful, task-based work Make...Remote workHourly payOngoing contractContract workFreelanceFlexible hours
Alignerr
United States3 days ago $20 per hour
...Senior Python Engineer Arionkoder is an AI consulting firm helping companies build, embed, and own AI systems... ...ML engineers, and domain experts to design and deliver innovative solutions.... ...Holidays Caregiver Time Off A dynamic remote work culture. You can work from anywhere...Remote workPermanent employmentLocal areaArionkoder
United States11 hours ago- ...Senior Engineer Opportunity Hungry for a challenge? That's good, because... ...within our Enterprise AI team. Working with a group of other Engineers to design, build, and deploy cutting-edge... ...BigQuery). Strong experience in Python and fluency in another language....Remote work
hackajob
United States3 days ago $70 - $85 per hour
...As an Electrical Engineering & RF/Circuit Design Expert, you will... ...developing a large-scale benchmark aimed at evaluating advanced AI systems''... ...typical data-labeling tasks; you will design... .... Strong Python programming skills... ...environment with remote compute sandboxes...Remote workSaidGig
United States3 days ago- A leading AI solutions firm is seeking a Python Developer to work on a Reinforcement Learning Gym project. You will design and optimize FastAPI services, collaborate... ...with software engineering best practices. This role offers fully remote work and a flexible schedule...Remote workFlexible hours
- ...is the Agentic AI Assistant platform... ...and automate tasks. Powered by the... ...’ Reasoning Engine and natural language... ...motivated to design and maintain... ...architecture, eval benchmark creation,... ...programming in Python and/or GolangFamiliarity... ...(flexible, remote, or required in...Remote workWork at officeImmediate startFlexible hours
$70 - $85 per hour
...Design graduate-level computational seismology and geophysics... ...whether advanced AI systems can complete... ...focuses on creating original benchmark problems grounded in... ...solution validators in Python. Qualifications... ...terminal environment with remote compute sandboxes. Ability...Remote workHourly paySaidGig
United States13 days ago$70 - $85 per hour
...Role Overview Design graduate-level computational... ...whether advanced AI systems can use... ...Design more open-ended tasks in which the system... ...validators in Python. Qualifications... ...terminal environment with remote compute sandboxes.... ...Familiarity with benchmark or evaluation...Remote workHourly paySaidGig
United States4 days ago- ...Computational Statistics and Applied Mathematics Expert to design graduate-level problems for AI benchmarks. You will craft tasks requiring real scientific software and multi-... ...target difficulty is met. This role supports remote work and part-time hours (15-20 per week), with...Remote workPart time
- ...Machine Learning Engineer The Machine... ...Developer designs, builds, ships,... ...evolving — where "AI-driven" once meant... ..., performance benchmarking, and automated... ...into agent-sized tasks; and review... ...Experience with Python is strongly preferred... ...(flexible, remote, or required in...Remote workWork at officeLocal areaFlexible hours
ServiceNow
United States2 days ago - ...States Digital Space LLC offers a remote contract role focusing on fine-tuning... ...complex problems for evaluation tasks. You will collaborate with LLM researchers on benchmarks spanning undergraduate to PhD topics, enabling cutting-edge AI projects while working...Remote jobContract work
$60 - $90 per hour
...talent with leading AI research labs.... ...investors include Benchmark , General Catalyst... ...Machine Learning Engineer — Model Evaluation... ...Location: Remote Commitment:... ...Responsibilities Design tasks by transforming... ...~ Proficiency in Python and Git . ~...Remote workHourly payWeekly payFull timeContract workFor contractorsSummer workMercor
Remote1 day ago- ...Principal Engineer, Enterprise AI Platforms Optum Tech is a... ...Office of AI, you will design, build and operate... ...flexibility to work remotely from anywhere within... ...on: golden datasets, task-level benchmarks, offline and online... ...level proficiency in Python and at least one additional...Remote workContract workWork at office
UnitedHealth Group
Eden Prairie, MN4 days ago - ...accelerator for frontier AI labs and a trusted... ...specialize in software engineering, logical reasoning, STEM... ...against industry performance benchmarks. Build agents that can... ..., architecture design, API design, production... ...a software engineering task. Required Skills Several...Remote jobFor contractorsFlexible hours
$60 - $65 per hour
Rose International in Tampa, FL seeks a Python AI and ML Engineer for a hybrid contract role. Three days onsite and two remote, six months, USD 60-65/hour. You will lead design and implementation of Python-based frameworks, build AI agents and autonomous workflows, and...Remote workContract work$70 - $100 per hour
...Overview You will design graduate-level computational... ...whether advanced AI systems can perform... ...software. Your tasks will include... ...solution validators in Python, ensuring reproducible executions in remote compute sandboxes.... .... Background in benchmark or evaluation design...Remote workHourly payFor contractorsSaidGig
United States3 days ago- ...Onsite and 2 days Remote Job Description: Python Developer - Data Protection... ...by architecting, engineering, deploying, and... ...(DPS) fleet design, engineer and operate... ...Development & Operational tasks involved in data... ...security controls AI Software...Remote workWork at officeWorldwideRotating shift3 days per week
Denken Solutions, Inc.
Alpharetta, GA2 days ago - 24-MAG LLC is seeking an experienced QA/test engineer to design robust benchmark test cases, review complex tasks, and debug Python environments for frontier AI evaluation work. The role is fully remote within the United States and requires strong attention to detail and...Remote job
$80 - $100 per hour
...experienced Senior Python Developers to help build... ...who is a leading AI platform that enables... ...contribute to AI research by designing programming benchmarks, evaluating AI-... ...experienced software engineers who enjoy solving... ...Information Fully remote contract opportunity....Remote workHourly payWeekly payContract work10 hours per weekLifted, an Upwork Company™
Texas City, TX29 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote Python Engineer (AI Benchmarking & Task Design). Be the first to apply!
Related searches
- freelance python developer New York, NY
- python programmer New York, NY
- python developer data analytics New York, NY
- python developer part time New York, NY
- python developer New York, NY
- python developer internship New York, NY
- python developer remote New York, NY
- full stack / python developer (remote) New York, NY
- senior python developer New York, NY
- python engineer New York, NY