Senior Python Engineer for LLM Evaluation [Remote]
$50 per hourSaidGig
United States
- Remote job
Role Overview
As a Senior Python Engineer focused on LLM Evaluation, you will play a pivotal role in creating advanced datasets for training and benchmarking large language models. Collaborating closely with researchers, you will curate code examples, provide precise solutions, and make necessary corrections, primarily utilizing Python while also engaging with other languages such as JavaScript, C/C++, Java, Rust, and Go.
Key Responsibilities
- Curate code examples, build solutions, and correct code primarily in Python, with additional work in JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
- Evaluate and refine AI-generated code to ensure efficiency, scalability, and reliability.
- Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks.
- Develop agents and automated verification tools in Python to verify code quality and identify error patterns.
- Hypothesize on steps in the software engineering cycle and evaluate model capabilities on them.
- Design verification mechanisms to automatically validate solutions to software engineering tasks.
Qualifications
- Minimum of 3 years of software engineering experience.
- Strong expertise in Python, with in-depth knowledge of frameworks, tooling, and best practices for production-grade software.
- Experience in building full-stack applications and deploying scalable software using modern languages and tools.
- Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
- Excellent oral and written communication skills for clear, structured evaluation rationales.
Work Terms
- Flexible engagement, with a minimum commitment of 10 hours per week, up to 40 hours per week.
- Contractor status (no medical/paid leave).
- Initial duration of 1 month, with potential extensions based on performance and fit.
- Candidates must be based in the United States.
Compensation
Compensation details will be discussed during the interview process.
Eligibility
- Completion of an AI video interview is required as part of the evaluation process.
- The application process takes approximately 15, 30 minutes.
Vacancy posted 18 days ago
Similar jobs that could be interesting for youBased on the Senior Python Engineer for LLM Evaluation [Remote] in United States vacancy
- ...pipelines, plus top AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality, multimodality, and... ...pedigree. Project Overview What Does a Typical Day Look Like? Evaluate and refine AI-generated code across backend and frontend...SeniorRemote jobFor contractorsFlexible hours
$40 per hour
...position focuses on developing evaluation and training datasets for... ...to tackle real-world software engineering challenges. The role involves... ...codebases locally to assess LLM performance in bug-fixing scenarios... .... Strong proficiency in Python or similar programming...SeniorContract workFor contractorsSaidGig
United States18 days ago- Driverai is seeking an Applied Data Scientist with expertise in LLM evaluation to join its innovative team in Austin, TX. This role focuses on building the evaluation function from scratch and requires a strong background in statistics and machine learning. The successful...SeniorRemote job
- Omaze is looking for a Senior Applied AI Scientist to... ...role focuses on AI system evaluation, measurement, and... ...data science with product engineering. You will work on evaluation frameworks for LLM features and analyze production... ...a strong background in Python, SQL, and experimental...Senior
$204k - $259k
Neura Market is seeking an experienced Software Engineer in Mountain View, California. You will develop innovative machine learning solutions for autonomous driving and contribute to advanced evaluation systems. The ideal candidate holds a Bachelor's or Master's degree...Senior$204k - $259k
...inference, hierarchical learning, and robust evaluation. This role follows a hybrid work schedule and you will report to a Senior Staff Software Engineer. You will: Work with a creative team... ...Proficient programming skills (eg: Python, C/C++) We prefer: ML infra...SeniorFull timeTemporary workRemote work- ..., and deliver top-notch technology products. As a Senior Lead Software Engineer - Python/AWS/AI/LLM at JPMorganChase within the Asset Wealth Management... ..., including full workflow toolset such as tracing, evaluations, and guardrails; * Must have strong programming...SeniorFull timeFor contractors
- ...Job Title: Senior Python Engineer Location: Sunnyvale, CA Role Overview We are seeking... ...develop advanced AI/ML solutions, including LLM inference, similarity search, and... ...or disability. All applicants will be evaluated solely on the basis of their ability,...Senior
Yantran LLC
Mountain View, CA1 day ago - ...companies’ adoption of frontier research for their next generation of LLM products. Join us if you: Wish to work on the premier... ...privacy for the sake of ML advancement. Responsibilities Own LLM evaluation processes and methods with a focus on generating benchmarks...Local areaShift work
$180k - $270k
...rely on. Possess strong software engineering skills (especially in Python) and have experience building reliable... ...systems, data pipelines, or evaluation harnesses that can run at scale against... ...what "good" looks like for a Speech LLM, translating capabilities (like ASR...Full timeWork at officeWorldwide$200k - $225k
...Senior Python Engineer Remote - USA The Role We're looking for a Senior Software Engineer... ...pipelines supporting training, inference, and evaluation Partner closely with ML Engineers,... ...architected backend systems ML and LLM capabilities are seamlessly integrated...SeniorRemote workFlexible hoursWizard
United States4 days ago$40 per hour
...This position focuses on developing and evaluating large language model (LLM) training datasets to address real-world software engineering challenges. The role emphasizes building... ...overall experience. Strong proficiency in Python. Experience with Git, Docker, and...SeniorContract workFor contractorsSaidGig
United States18 days ago- ...Senior Python Engineer Ciklum is looking for a Senior Python Engineer to join our team full-time... ..., handling critical functions such as LLM routing, telemetry, and per-tenant cost... ...Real Estate (CRE) workflows Evaluation Literacy: Experience building programmatic...SeniorFull timeRemote work
Ciklum
United States2 days ago $175k - $275k
...-Time in Austin, TX Remote (any location) - Senior - Product & Engineering - $175k - $275k Applied Data Scientist, LLM Evaluation Introduction At Driver, we’re building systems... ...outputs are open-ended text. Proficient in Python and the scientific/data stack (pandas, NumPy...Remote jobFull timeFlexible hours- Senior AI/ML Engineer - LLM & Agentic AI Systems - Hybrid We are looking for a Senior AI/ML Engineer... ...strategies Support LLM fine‑tuning, model evaluation, and continuous model improvement... ...orchestration platforms Strong Python development experience for AI/ML pipelines...SeniorFull time
$80k - $175k
...Nasdaq: INOD) is a global data engineering company. We believe that data... ...by providing the data, evaluation frameworks, and human expertise... ...team of technical experts in LLM training, post-training, and... ...Strong programming skills in Python and experience building production...Full timeFixed term contract$141.8k - $258.6k
...leveraging multimodal capabilities. You will design and manage data annotation processes, work with ML Engineers, and develop LLM auto-judges for AI model evaluation. The ideal candidate has a BA/Master’s in a relevant field and at least 2 years of experience in survey...- ...Senior Data Scientist The AI & Data Analytics division is... ...innovative training strategies) and evaluation (T&E, robustness) for key... ...Hands on experience with LLM/LVM/Foundation Model and... .... Software engineering foundations: Python, ML frameworks (PyTorch/TensorFlow...SeniorWork experience placementRemote work
PNNL
Richland, WA3 days ago - Hippocratic AI is seeking an experienced ML Engineer in Palo Alto to design and build evaluation frameworks for large language models. The role involves... ..., with strong software engineering skills in Python and experience in LLM evaluation. Join a team focused on transforming...Senior
$225k
This employer is hiring an AI Solutions Engineer for a direct hire role based in Kennett Square... .... You’ll be working heavily with Python, modern web technologies, and large language... ...What You Will Be Doing Tech Breakdown AI / LLM-based development (RAG, agents, workflows...Senior2 days per week- Lendistry is seeking a Senior AI Engineer to enhance our AI capabilities and lead the development of LLM features. This hands-on role involves collaborating... ...will have a strong background in Python, LLM applications, and AI system evaluation. Our comprehensive benefits and...Senior
$175k - $225k
Innodata is seeking an Applied Research Scientist to advance LLM evaluation and post-training methods. This remote role involves spearheading... ..., extensive experience in ML/AI, and strong coding skills in Python. The position offers a competitive salary of $175,000 - $225,0...Remote job- Centific Global Solutions, Inc. is seeking a Research Scientist for LLM Evaluation & Post-Training in Seattle, WA. This full-time role involves... ...experience in applied ML research, and strong proficiency in Python and ML frameworks. This position offers hybrid working options...Full time
- ...Vision Technologies is seeking an innovative LLM Engineer to design and operationalize workflows... ...Strong ML background and proficiency in Python and PyTorch are essential, alongside a commitment to production readiness and evaluation rigor. #J-18808-Ljbffr Bright Vision...SeniorRemote job
$77k - $202k
...in data and analytics engineering focus on leveraging advanced... ...business needs. As a Senior Associate, you analyze... ...languages like Python, Java, Scala Proficient... ...prompt engineering for LLM outputs Designing thorough... ...with team members. We evaluate these factors...SeniorFull timeH1bPwC
Minneapolis, MN18 hours ago- Collective Health is seeking a Senior Software Engineer in Test for AI Agentic Systems... ...quality ownership for LLM-based claim adjudications, developing evaluation frameworks, and automating testing... ...candidates will have expertise in Python and AI observability, paired...SeniorWork at office2 days per week
$140k - $215k
...and emerging threats. This role entails analyzing LLM architectures and developing methodologies to enhance... ...of transformer technologies, experience in evaluating AI systems for security, and proficiency in Python. The position allows flexible work with in-office requirements...SeniorWork at officeFlexible hours$281k - $356k
...improve and speed up the evaluation and onboard developer... ...researchers and software engineers who are passionate... ...models and Generative AI (LLM/VLM) solutions. These solutions... ...areas Proficiency in Python and standard ML... ...leadership, influencing senior stakeholders, and driving...SeniorFull time- RAVENNA in Seattle is seeking experienced engineers passionate about AI to design and implement LLM powered systems. Ideal candidates will have at least five... ...on robust, scalable AI workflows. You will develop evaluation frameworks, maintain high engineering standards, and...SeniorFlexible hours
$204k - $259k
Senior Research Scientist, Foundation Model (LLM/VLM) Waymo Position type: Full‑time Location: New York Waymo is... ...and run experiments training and evaluating large deep learning models Present... ...or simulation Proficiency in Python 3+ years of experience with modern...SeniorFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Python Engineer for LLM Evaluation [Remote]. Be the first to apply!
Related searches
- python developer remote United States
- c++-python developer United States
- python developer United States
- python sql developer United States
- qa engineer python United States
- python developer with big data United States
- python engineer United States
- python developer internship United States
- entry level python developer United States
- freelance python developer United States