AI Safety & Evaluation Lead - LLM Systems
Health4TheWorld
This is currently a volunteer position. Contributors will gain hands-on experience building real-world AI systems for public health impact.
About the Role
This unpaid volunteer role oversees evaluation, safety, and responsible AI governance for LLM-powered healthcare chatbot systems.
Commitment
- 4–6 hours per week
- Remote
- Flexible schedule
Qualifications
- Background in data science, NLP, machine learning, or AI research
- Experience with LLM evaluation, RAG, and prompt engineering
- Understanding of model validation and bias mitigation
- Interest in healthcare ethics and responsible AI
Responsibilities
- Design evaluation framework for chatbot outputs
- Develop medical safety test cases
- Measure hallucination rates and factual accuracy
- Implement guardrails and bias monitoring
- Document governance practices
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI Safety & Evaluation Lead - LLM Systems in Palo Alto, CA vacancy
- Hippocratic AI is seeking an experienced ML Engineer... ...to design and build evaluation frameworks for large language... ...automated evaluation systems, and collaborating with... ...model accuracy and safety. Candidates must hold an... ...and experience in LLM evaluation. Join a team...Suggested
$213.48k - $319.72k
...and profound opportunity for AI to drive positive change in the... ..., T. Rowe Price, and other leading investors. About the Role As the Autonomous Driving System Safety Lead on the Systems Engineering... ...and assess evidence from evaluation and validation as arguments for...SuggestedWork experience placementImmediate startFlexible hoursNuro
Mountain View, CA29 days ago$101k - $127k
Systems Safety Engineer The goal of a Systems Safety Engineer at Wisk is to accurately assess risk to inform decision‑making, resulting in... ...and reliability, and identify, communicate, and continuously evaluate product safety risk. This is a hybrid work opportunity;...SuggestedFull timeWork experience placementWork at officeLocal areaImmediate startFlexible hours3 days per week$190k - $241k
A leading autonomous vehicle technology company in Mountain View is seeking a Safety Management Systems Manager to support safety policies and promote a positive safety culture. The ideal candidate will possess a minimum of 5 years of experience in safety management, technical...SuggestedRemote job- ...Founded in 2017, Wayve is the leading developer of Embodied AI technology. Our advanced AI software... ..., enhancing the usability and safety of automated driving systems. Our vision is to create... ...experience Risk management tools and evaluation Cross‑functional project...SuggestedFull timeWork at officeWork from home
- ...multilingual data and evaluation partner for foundation... ...enterprises deploying GenAI systems globally. They deliver... ...systems that ensure AI models perform reliably... ...challenges, including safety, bias, inclusivity, and... ...Management & Risk Mitigation Lead incident reporting,...
- Advanced Micro Devices is seeking a Trustworthy AI, Safety Architect in Santa Clara, California. This hybrid role involves shaping the deployment... ...of safe AI across diverse platforms. The ideal candidate will lead global AI strategies, design safety guardrails, and build...
- AMD is seeking a Trusted AI, Safety Architect to lead safe AI innovation globally. This hybrid role requires developing governance strategies and leading AI safety initiatives across diverse industries. The ideal candidate will have strong expertise in SLMs/LLMs, a solid...
$193.93k - $291.15k
...driver, combining cutting‑edge AI with automotive‑grade hardware... ...future. About the Team The Safety Office is a key part of our Public... ...Safety Governance & Standards Lead, you will lead Nuro’s safety... ...standards, Safety Management Systems (SMS), safety case development...Work at office$190k - $241k
...simulation across 15+ U.S. states. The Waymo Safety team works to promote and help to... ...SMS Policy Maintenance: Support the SMS Lead in the development, periodic review, revision... ...years of experience in Safety Management Systems (SMS), Organizational Development, or Corporate...Full timeRemote work$184k - $356.5k
...products. Responsibilities include performing system-level analysis and defining performance... ...field and 8+ years of experience in safety-critical systems. The position offers a competitive... .... Join us as we innovate in the realm of AI and autonomous vehicles! #J-18808-Ljbffr...$130k - $190k
...developed an artificial intelligence (AI) powered technology stack... ...are looking for a Functional Safety Hardware Engineer to join us... ...we bring the Kodiak Autonomy System to market. We first published... ...applications. As part of the evaluation process, we provide Endorsed with...Temporary workWork at officeVisa sponsorshipFlexible hoursKodiak
Mountain View, CA29 days ago- Rivian in Palo Alto is seeking a Manager for ADS Safety Case, Regulatory. This critical role involves leading the development of safety cases for Rivian’s Level 3 and 4 autonomous driving systems. The selected candidate will work between Autonomy, Engineering, and Regulatory...
$125.5k - $230.2k
Location: Anywhere in Country Manager - AI Safety and Trust Architect The opportunity We’re... ...explainability, Formal Methods and complex system behaviors. Knowledge of the use of causal... ...and troubleshooting skills. Ability to lead AI safety architectural product functions...Summer holidayFlexible hours- We're building safety-enhancing technology for aviation that will save lives. Automated aviation systems will enable a future where air transportation is safer, more convenient and fundamentally transformative to the way goods — and eventually people — move around the planet...Permanent employmentRemote work
- A leading autonomous flight company is looking for a Staff Systems Safety Engineer in Mountain View, CA. This hybrid position involves working with cross-functional teams to assess and mitigate safety risks for innovative aircraft systems. Responsibilities include developing...Flexible hours
- A cutting-edge aerospace technology company is seeking a Systems Safety Engineer in Mountain View, California. The role involves applying safety methodologies and supporting the development of certifiable safety-critical flight systems. Candidates should have over 10 years...Remote job
$167k - $208k
...and more time being there. If you want to be part of shaping the future of mobility, please read on. We are looking for a Staff Systems Safety Engineer to join our team. The goal of a Systems Safety Engineer at Wisk is to accurately assess risk to inform decision‑making...Full timeWork at officeLocal areaImmediate startFlexible hours3 days per week$95 - $110 per hour
...client in Mountain View, CA. This role involves leading the development of machine learning models for prompt injection and safety, optimizing performance across various... ...PyTorch, and knowledge of hybrid deployment systems. The position is a 12-month contract with a...Contract work- The Fountain Group is looking for an experienced Machine Learning Engineer in Mountain View, CA, to lead the development of prompt safety models for agentic AI systems. Responsibilities include designing and deploying classifiers, optimizing hybrid deployment pipelines,...
- At Rhoda AI, we're building the full-stack foundation for the next generation of humanoid... ...of large-scale learning, robotics, and systems, with a research team that includes researchers... ...reporting directly to the Head of Safety and Certification. We're looking for a technical...
- Wisk Aero LLC is seeking a Staff Systems Safety Engineer to join their team in Mountain View, California. In this role, you will assess risks and ensure safety for autonomous aircraft systems, working in a hybrid model that involves 3 days a week onsite. Your responsibilities...3 days per week
$168k - $258.75k
A leading technology company in Santa Clara, CA, is looking for a Senior Functional Safety Manager to lead the safety efforts for AI-powered autonomous driving systems. The role involves ensuring compliance with ISO standards, mentoring team members, and collaborating with...Remote job- ...Lang LaSalle Incorporated is looking for an EH&S Specialist in Mountain View, CA to develop and implement environmental health and safety programs. The role focuses on compliance with regulations and fostering a safety-first culture. Candidates should have a Bachelor's...
- Aeva is looking for an experienced Systems Engineer in Mountain View, California, to help develop next-generation Lidar sensor products... ...with automotive standards such as ASPICE and Functional Safety. Ideal candidates will have 5+ years in Systems Engineering, expertise...Flexible hours
$174k - $252k
...Engineer, Infrastructure, Ads Safety Apply X Note: By applying to... ...infrastructure, or distributed systems or networks, or experience with... ...systems that integrate with AI Agents, Large Language Models... ...track record of successfully leading major system re-architectures...Full time- ...principles Ensure clear safety accountability and... ..., and manufacturing Lead root cause analysis (RCA... ...structured methodologies Evaluate operational,... ...rigging, and high-voltage systems Coordinate emergency... ...artificial intelligence (AI) tools to support parts...Work at office
Pivotal
Palo Alto, CA3 days ago - A leading technology firm in Mountain View, California, is seeking a Senior Quality Assurance... ...Reliability Engineer to oversee product safety and reliability. In this role, you will... ...design engineering and quality management systems, allowing them to thrive in a fast-paced,...
$207k - $300k
...Engineer, Workspace Abuse Safety Protection corporate_... ...years of experience in leading technical project... ...model deployment, model evaluation, data processing, debugging... ...production/operational ML systems. Experiences in... ...business successful through AI by combining cutting-...Full time$154.9k - $209.6k
A leading technology company is seeking an experienced Systems Engineer to contribute to the development of next-generation Lidar sensor products. The ideal candidate will have 5+ years in Systems Engineering, a strong background in sensor technology, and expertise in...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety & Evaluation Lead - LLM Systems. Be the first to apply!
Related searches
- safety technician Palo Alto, CA
- construction safety Palo Alto, CA
- safety Palo Alto, CA
- safety intern summer Palo Alto, CA
- vp safety Palo Alto, CA
- safety trainer Palo Alto, CA
- safety analyst Palo Alto, CA
- safety physician Palo Alto, CA
- safety specialist Palo Alto, CA
- aviation safety assistant office automation Palo Alto, CA