AI Safety & Evaluation Lead - LLM Systems
Health4TheWorld
This is currently a volunteer position. Contributors will gain hands-on experience building real-world AI systems for public health impact.
About the Role
This unpaid volunteer role oversees evaluation, safety, and responsible AI governance for LLM-powered healthcare chatbot systems.
Commitment
- 4–6 hours per week
- Remote
- Flexible schedule
Qualifications
- Background in data science, NLP, machine learning, or AI research
- Experience with LLM evaluation, RAG, and prompt engineering
- Understanding of model validation and bias mitigation
- Interest in healthcare ethics and responsible AI
Responsibilities
- Design evaluation framework for chatbot outputs
- Develop medical safety test cases
- Measure hallucination rates and factual accuracy
- Implement guardrails and bias monitoring
- Document governance practices
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI Safety & Evaluation Lead - LLM Systems in Palo Alto, CA vacancy
- Hippocratic AI is seeking an experienced ML Engineer... ...to design and build evaluation frameworks for large language... ...automated evaluation systems, and collaborating with... ...model accuracy and safety. Candidates must hold an... ...and experience in LLM evaluation. Join a team...Suggested
$213.48k - $319.72k
...scalable driver, combining cutting-edge AI with automotive-grade hardware. Nuro... ...the Role As the Autonomous Driving System Safety Lead on the Systems Engineering team, you will... ...mitigated. Review and assess evidence from evaluation and validation as arguments for safety...SuggestedWork experience placementNuro
Mountain View, CA9 days ago$90 - $130 per hour
...ML Engineer, LLM Safety & Security Sustainable Talent is partnering with Nvidia... ...and engineering teams, focused on AI Safety for LLMs, including multi-lingual... ...datasets and moderator models for evaluating LLM models and end-to-end systems for Content Safety, ML Fairness....SuggestedFull timeContract workWork experience placementRemote workSustainable Talent
Santa Clara, CA4 days ago$118.4k - $173.4k
...also spearhead new and critical safety initiatives, providing the strategic... ...efficient execution of the EHS System across all domestic sites, including... ...and mentorship opportunities. Lead efforts to optimize EHS processes for evaluating and optimizing vendor and...SuggestedFull timeFor contractorsFor subcontractorLocal areaGFiber
Mountain View, CA2 days ago$174k - $252k
...pipeline in a frontier AI environment.... ...and anomaly detection systems at scale, taking safety defenses or mitigations... ...Experience collaborating on or leading applied ML projects, including LLM training, inference,... ...involves building novel evaluations, red-teaming,...SuggestedFull time$237k - $329k
...apply the latest advancements in AI, collaborating with experts... ...DeepMind, Core, and Trust and Safety to develop model mitigations that... ...Google . Responsibilities Lead and scale a team of... ...optimization, develop encoder systems, etc.) and design safety mitigations...Full timeWorldwide$159.4k - $245k
...breakthrough hardware and battery systems to intuitive design,... ..., and next-generation safety and entertainment... .... What You’ll Do Lead TPM execution for validation... ...experience in AI/ML, robotics, or autonomous... ...participate in a company vehicle evaluation program, through which...RelocationRelocation packageFlexible hours$101k - $127k
...tiny cookie so that you're not asked to make this choice again.Systems Safety Engineer page is loaded## Systems Safety Engineerlocations:... ...and reliability; and identify, communicate, and continuously evaluate product safety risk.This is a hybrid work opportunity. At Wisk...Full timeWork experience placementLocal areaImmediate startFlexible hours3 days per week$193.93k - $291.15k
...driver, combining cutting-edge AI with automotive-grade hardware... .... About the Team The Safety Office is a key part of our Public... ...Safety Governance & Standards Lead, you will lead Nuro's safety... ...standards, Safety Management Systems (SMS), safety case development...Work at officeNuro
Mountain View, CA8 days ago- Advanced Micro Devices is seeking a Trustworthy AI, Safety Architect in Santa Clara, California. This hybrid role involves shaping the deployment... ...of safe AI across diverse platforms. The ideal candidate will lead global AI strategies, design safety guardrails, and build...
- ...Founded in 2017, Wayve is the leading developer of Embodied AI technology. Our advanced AI software... ..., enhancing the usability and safety of automated driving systems. Our vision is to create... ...experience Risk management tools and evaluation Cross‑functional project...Full timeWork at officeWork from home
$190k - $241k
A leading autonomous vehicle technology company in Mountain View is seeking a Safety Management Systems Manager to support safety policies and promote a positive safety culture. The ideal candidate will possess a minimum of 5 years of experience in safety management, technical...Remote job- A leading technology company is seeking candidates for a role in AIML related to Machine Learning and Applied Research. You will be part of a team shaping safety policies for generative AI products and collaborate closely with machine learning researchers, engineers, and...
$190k - $241k
...simulation across 15+ U.S. states. The Waymo Safety team works to promote and help to... ...SMS Policy Maintenance: Support the SMS Lead in the development, periodic review, revision... ...years of experience in Safety Management Systems (SMS), Organizational Development, or Corporate...Full timeRemote work- AMD is seeking a Trusted AI, Safety Architect to lead safe AI innovation globally. This hybrid role requires developing governance strategies and leading AI safety initiatives across diverse industries. The ideal candidate will have strong expertise in SLMs/LLMs, a solid...
$174k - $252k
...Software Engineer, Generative AI, Ads Safety corporate_fare Google place... ...g., model deployment, model evaluation, optimization, data... ...engineering, applied ML, autonomous systems, and large-scale systems design... ...code. Participate in, or lead design reviews with peers...Full time- ...multilingual data and evaluation partner for foundation... ...enterprises deploying GenAI systems globally. They deliver... ...systems that ensure AI models perform reliably... ...challenges, including safety, bias, inclusivity, and... ...Management & Risk Mitigation Lead incident reporting,...
$147k - $211k
Software Engineer, Commerce Content Safety Mountain View, CA, USA... ...machine learning, generative AI and AI algorithms. Responsibilities... ...frameworks for model evaluation and performance tracking. Collaborate... ...the effectiveness of our systems, and guide future improvements...Full time$100k - $110k
...Safety Strategy And Compliance Manager Welo Data,... ...multilingual data and evaluation partner for foundation... ...enterprises deploying GenAI systems globally. They deliver... ...systems that ensure AI models perform reliably... ...operational controls. Lead incident reporting, investigation...Temporary work$121.3k - $213.7k
...Python and Spark. Understand complex software and data processing systems and be able to break the systems down into smaller components in... ...+ years experience programming in Python Experience leveraging AI-powered development tools to streamline daily workflows,...Relocation- Rivian in Palo Alto is seeking a Manager for ADS Safety Case, Regulatory. This critical role involves leading the development of safety cases for Rivian’s Level 3 and 4 autonomous driving systems. The selected candidate will work between Autonomy, Engineering, and Regulatory...
- Pivotal, based in Palo Alto, CA, is seeking a Safety Manager to oversee safety management systems for our revolutionary eVTOL aircraft. The role involves developing safety strategies, conducting audits, and ensuring compliance with FAA regulations. Ideal candidates will...
$125.5k - $230.2k
Location: Anywhere in Country Manager - AI Safety and Trust Architect The opportunity We’re... ...explainability, Formal Methods and complex system behaviors. Knowledge of the use of causal... ...and troubleshooting skills. Ability to lead AI safety architectural product functions...Summer holidayFlexible hours$181.1k - $318.4k
...Senior ML Engineer, Responsible AI and Safety Join Us in Shaping the... ...you passionate about making AI systems safer, more inclusive, and... ...features. In this role, you will lead the responsible AI lifecycle... ...directly influence how we evaluate, align, and monitor the safety...RelocationApple
Cupertino, CA2 days ago$101k - $127k
Wisk is seeking a Systems Safety Engineer in Mountain View, California. The position focuses on assessing risks to ensure safety in autonomous flight. Responsibilities include performing safety assessments, defining safety targets, and collaborating with various teams....Flexible hours- ...automotive’s next chapter. From operating systems to zonal controllers to cloud and connectivity... ...our expertise across connectivity, AI, security and more, we’ll map a new way forward... ...This role is embedded in the Functional Safety Engineering team with a specific focus on...Full timeFlexible hours
- Decisive Point in Mountain View, CA is looking for a System Safety Engineer to define and manage safety architecture for autonomous driving systems. The ideal candidate should have over 5 years of relevant experience, expertise in engineering, and knowledge of automotive...
- A leading autonomous flight company is looking for a Staff Systems Safety Engineer in Mountain View, CA. This hybrid position involves working with cross-functional teams to assess and mitigate safety risks for innovative aircraft systems. Responsibilities include developing...Flexible hours
$130k - $190k
...developed an artificial intelligence (AI) powered technology stack... ...are looking for a Functional Safety Hardware Engineer to join us... ...we bring the Kodiak Autonomy System to market. We first published... ...applications. As part of the evaluation process, we provide Endorsed with...Temporary workWork at officeVisa sponsorshipFlexible hoursKodiak
Mountain View, CA8 days ago- A cutting-edge aerospace technology company is seeking a Systems Safety Engineer in Mountain View, California. The role involves applying safety methodologies and supporting the development of certifiable safety-critical flight systems. Candidates should have over 10 years...Remote job
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety & Evaluation Lead - LLM Systems. Be the first to apply!
Related searches
- safety tech Palo Alto, CA
- patient safety associate Palo Alto, CA
- aviation safety assistant office automation Palo Alto, CA
- patient safety Palo Alto, CA
- safety attendant Palo Alto, CA
- safety trainer Palo Alto, CA
- safety Palo Alto, CA
- safety physician Palo Alto, CA
- safety technician Palo Alto, CA
- safety intern summer Palo Alto, CA