Harness Engineer - AI Prompt & Evaluation Architect
$375kVirio
Virio is seeking a Harness Engineer in San Francisco. You will have the responsibility for the intelligence layer that integrates with our AI models and product. Your role involves refining system prompts, designing evaluation frameworks, and collaborating with product teams. The position offers a compensation package ranging from $375K to $625K, including a base salary and cash bonus, alongside benefits such as medical insurance and equity opportunities. #J-18808-Ljbffr Virio
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Harness Engineer - AI Prompt & Evaluation Architect in San Francisco, CA vacancy
- ...Switzerland to develop high-quality reasoning questions for AI models. Your expertise in biology, physics, or chemistry... ...Italian and English. The role involves creating detailed prompts, establishing evaluation standards, and conducting testing on models. A commitment...Suggested
Obsidian
San Francisco, CA1 day ago $180k - $300k
A leading AI company in San Francisco seeks an experienced candidate to own and elevate core AI systems that power their services... ...role requires expertise in TypeScript and Python, focusing on prompt engineering for text, image, and layout generation. With a hybrid work...Suggested$130 per hour
...seeking an Internal Medicine Expert to design clinically realistic prompts and scenarios. Ideal candidates are board-certified physicians... ...to $300 per hour and involves responsibilities such as grading AI-generated responses and providing feedback to enhance model performance...SuggestedHourly payOngoing contractRemote workMercor Inc
San Francisco, CA14 days ago- ...Known - Conversational AI Engineer, System Prompt and Orchestration ~ San Francisco, CA (In... ...Orchestration & Context Optimization: Architecting the core system prompts and managing... ...Quality: Developing custom evaluation frameworks to measure "conversational...Suggested
Known, Inc
San Francisco, CA2 days ago - Shared Context Lab is seeking a Founding Engineer in San Francisco, United States, to work directly with the CEO and CTO in building core... ...This opportunity is ideal for someone passionate about shaping the future of AI-powered software. #J-18808-Ljbffr Shared Context LabSuggested
- Nohi, based in San Francisco, is seeking a Senior Full-Stack Engineer to lead the architecture and implementation of core systems such as products, orders, and payments. You will drive the end-to-end development, ensuring reliability across both frontend and backend. The...
$315k
We are looking for Research Engineers to build “gold standard” evaluations for catastrophic risks, in order to understand what AI Safety Level (ASL) to assign to models. Research leads... ...experience training, working with, and prompting models For all workstreams, experience...Currently hiringWork at officeImmediate startHome officeVisa sponsorshipRelocation package- ...building the world’s first AI native BPO, starting with healthcare... ...looking for an exceptional Harness Engineer to build systems around our... ...approaches, and large‑scale evaluations. You either have healthcare... ...how it was built, the prompts you used, and why you chose...3 days per week
- ...chats with other countries. AI then powers the worlds’ response... ...needed to rigorously tailor harnesses and prompts to individual AI models.... ...training tuned endpoints. Evaluating and improving embedding and... ...stage startup or as a founding engineer. This job listing is for a...Full timeVisa sponsorship
$375k
Harness Engineer Location: San Francisco (in-person, in-office, full... ...that sits between our AI models and product—the system prompts, tool definitions, context management, and evaluation framework that makes... ...metrics to measure it. Architect the abstraction layer between...Full timeWork at officeRelocation package- ...you’re a senior construction engineering professional who thrives on precision... ...how the next generation of AI systems understand... ...construction work. You’ll challenge and evaluate advanced language models on... ...and refine AI-generated prompts, answers, explanations, calculations...For contractorsRemote work
YO IT Consulting
San Francisco, CA2 days ago - ...Nexxa.ai is building artificial super intelligence... ...Job Title: Senior AI Architect Location:... ...looking for a Senior AI Engineer who has spent the last... ...latency, and cost Define evaluation, monitoring, and... ...preference modeling, eval harnesses) Ability to reason...
Nexxa.AI
San Francisco, CA4 days ago $138k - $225k
...Customer Success, Marketing, and Engineering to create a unified... ...efficiency. We're hiring an AI Architect to design, build, and launch... ...and prototyping, through evaluation and rollout, to monitoring... ...applying AI evaluation strategies, prompt engineering techniques, and...For contractorsWork at officeWork from homeLinkedIn
San Francisco, CA3 days ago- ...foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone... ...parallel agents. Build agent harnesses to improve open model (... ...same for open source! Automate prompt optimization techniques like DSPy...Work at officeImmediate start
- ...Senior AI Architect – Multi-Agent Systems & Platform Infrastructure... ...Systems & Orchestration / Head of Engineering Seniority: Senior-Level (... ...wealth workflows. Design prompt engineering pipelines,... ...Develop and refine test plans, evaluation pipelines, and debug tools...Full timeWork at officeRemote work
Nivalto
San Francisco, CA4 days ago $142.6k - $261.5k
...working world. ServiceNow– ServiceNow AI Architect Manager In the digital economy,... ...selecting methods, techniques, and evaluation criteria for obtaining results.... ...environments Strong foundation in prompt engineering, including crafting effective prompts...Summer holidayWorldwideFlexible hoursEY
San Francisco, CA2 days ago$171.6k - $392.1k
...working world. ServiceNow – ServiceNow AI Architect Senior Manager In the digital... ...capabilities with ServiceNow or evaluating how AI can streamline service delivery... ...for AI-driven solutions Skill in prompt engineering and Retrieval-Augmented Generation (RAG...Summer holidayWorldwideFlexible hoursEY
San Francisco, CA2 days ago$180k - $260k
...investment firm in San Francisco is seeking a Model Behavior Architect to enhance their answer engine by collaborating with research, design, and engineering... .... Candidates should demonstrate a strong passion for AI, be familiar with Python, and possess a high-level...Full time$171.6k - $392.1k
...working world. ServiceNow - ServiceNow AI Architect Senior Manager In the digital... ...enterprise capabilities with ServiceNow or evaluating how AI can streamline service... ...pipelines for AI‑driven solutions Skill in prompt engineering and Retrieval‑Augmented Generation (...Summer holidayWorldwideFlexible hours- B Capital is looking for a Lead Agentic Data Systems Engineer in San Francisco to architect and maintain autonomous data products. This hands-on role involves... ...into actionable systems, and leveraging generative AI to drive efficiency. Applicants should have over 5 years...
- ...Luma AI Job Posting Luma's mission is to build multimodal AI to expand... ..., you'll partner with researchers, engineers, and technical artists to evaluate our models against real-world creative... ...performance across diverse tasks, prompts, and modalities. Identify key...
Luma AI
San Francisco, CA3 days ago $320k - $405k
...Cross-functional Prompt Engineer Anthropic's mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and... ...and policy concerns Develop behavioral evaluations in collaboration with product teams and alignment...Full timeWork at officeImmediate startVisa sponsorshipFlexible hourslatitude
San Francisco, CA1 day ago$180k - $220k
David Joseph & Company is looking for a Software Engineer to develop datasets and evaluation systems that enhance AI models performance. This role involves designing data slices, running experiments, and collaborating with leading AI research teams. The ideal candidate...- A pioneering AI research company in San Francisco seeks a Research-Hardware Codesign Engineer. This hybrid role involves shaping AI silicon architecture, debugging system discrepancies, and writing quantization kernels. Candidates should have a strong background in Python...
$230k - $385k
...AI Systems Engineer - Codex Core Agents About The Team The Codex Core... ...team builds the agent harness that turns model capability... ...systems around the model: prompting and interpreting model outputs... ...how models are trained and evaluated, making this one of the highest...OpenAI
San Francisco, CA22 hours ago- Champ AI is building a multimodal work-agent orchestration... ...continuously improve with evaluations and feedback loops. The... ...looking for a Deployment Architect / Forward Deployed Engineer to own the path from "first... ...the product. Configure prompts, tools, evals, and guardrails...Contract workLive inDay shift
- ...Bilingual Italian STEM Expert to work remotely for a duration of 3-6 months. The role involves designing STEM-based prompts in Italian and English, defining evaluation standards, and ensuring scientific rigor in assessments. Candidates must have native-level fluency in Italian...Remote job
$114.2k - $142.7k
...data processing, and software engineering, our office is a truly... ...looking for a Mechanical Engineer, Harness Design to support the end-to-... ...information as described therein. AI in Our Interviewing Process... ...prohibited unless explicitly prompted by an interviewer or...Full timeTemporary workFor contractorsWork at officeLocal areaRemote workHome office3 days per weekPlanet Labs PBC
San Francisco, CA3 days ago$200k - $300k
...reliable, interpretable, and steerable AI systems. We want AI to be safe and... ...group of committed researchers, engineers, policy experts, and business... ...experience with LLMs including advanced prompt engineering, agent development, evaluation frameworks, and deployment at...Work at officeVisa sponsorshipFlexible hours$214k - $300k
Monograph is seeking an engineer to build and improve AI evaluation systems aimed at increasing shipping quality for AI tools. You will enhance scalable eval runners, improve benchmarks, and ensure reliability in distributed systems. Strong engineering fundamentals and...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Harness Engineer - AI Prompt & Evaluation Architect. Be the first to apply!
