AI Governance and Evaluation Executive

Job Description - AI Governance and Evaluation Executive (981995) Job Description AI Governance and Evaluation Executive - ( 981995 ) Description

WHY UT SOUTHWESTERN?

With over 75 years of excellence in Dallas-Fort Worth, Texas, UT Southwestern is committed to excellence, innovation, teamwork, and compassion. As a world-renowned medical and research center, we strive to provide the best possible care, resources, and benefits for our valued employees. Ranked as the No. 1 hospital in Dallas-Fort Worth according to U.S. News & World Report, we invest in you with opportunities for career growth and development to align with your future goals. Our highly competitive benefits package offers healthcare, PTO and paid holidays, on-site childcare, wage, merit increases and so much more. We invite you to be a part of the UT Southwestern team where you’ll discover a culture of teamwork, professionalism, and a rewarding career!

JOB SUMMARY

The AI Governance and Evaluation Executive is the program and operations leader responsible for implementing, scaling, and continuously improving UTSW's enterprise approach to responsible AI governance and evaluation. Reporting to the CAIO, this role executes the governance and evaluation roadmap and adoption plan across the health system, research and academic mission, biomedical education, and administrative and business functions. *** The leader in this role converts enterprise direction into annual priorities and repeatable workflows for intake, classification, evaluation, review, vendor assessment, deployment conditions, monitoring, reassessment, and stakeholder support. The role maintains enterprise portfolio visibility, drives reuse and consistency, and coordinates cross-functional execution and works closely with Compliance, Information Security, and Information Technology Services on AI risk management, while those functions retain formal ownership of their respective risk-management domains.

BENEFITS

PPO medical plan, available day one at no cost for full-time employee-only coverage Paid Time Off, available day one Retirement Programs through the Teacher Retirement System of Texas (TRS) Paid Parental Leave Benefit Wellness programs Tuition Reimbursement Public Service Loan Forgiveness (PSLF) Qualified Employer Master's Degree in Public Health, Biomedical Informatics, Data Science, Computer Science, Engineering, Research Administration, or a related field. Experience 8 years of progressively responsible experience in AI, digital health, healthcare operations, technology governance, clinical informatics, data analytics, research administration, compliance, or a related field. 4 years leading people, programs, or complex cross-functional initiatives. Demonstrated experience operating cross-functional governance, evaluation, assurance, or enterprise programs. Demonstrated experience implementing enterprise roadmaps, operating models, or new capabilities, and translating priorities into measurable outcomes. Preferred Education PhD or additional advanced degree in a related field. Experience Experience in an academic medical center, integrated health system, life sciences organization, or similarly complex regulated environment. Experience with AI/ML evaluation, deployment, governance, responsible AI, or assurance. Experience evaluating AI vendors and coordinating due diligence. Technical fluency across LLM ecosystems, MLOps, retrieval and vector technologies, AI APIs and integrations, lifecycle monitoring, and incident-response interfaces sufficient to engage technical teams and assess governance evidence. Experience building or scaling a new team or function and managing tools, vendors, contracts, or external resources. Familiarity with generative AI, LLMs, multimodal models, foundation models, and agentic AI. Working knowledge of healthcare AI evaluation and lifecycle monitoring, including quantitative methods and model cards. Familiarity with NIST AI RMF, ISO/IEC 42001, and related responsible AI frameworks or standards.

JOB DUTIES

Lead Governance and Evaluation Operations and Roadmap Execution Independently run the day-to-day enterprise Responsible AI Governance and Evaluation program on behalf of the CAIO. Translate program strategy into an annual operating roadmap, including priorities, dependencies, resource needs, adoption activities, and measurable program outcomes. Operationalize AI policies, standards, workflows, decision rights, review pathways, and documentation requirements. Coordinate governance across institutional and domain-specific structures including work with UTSW executives and senior leadership. Prioritize work based on institutional value, risk, dependencies, and capacity, and surface tradeoffs for decision. Ensure governance is risk-proportionate, timely, consistent, and designed to enable responsible innovation. Identify complex, novel, or high-consequence cases requiring escalation to the CAIO or appropriate governance body. Manage End-to-End Governance Workflow Operate overall intake, triage, AI risk classification, evaluation, multidisciplinary review, decision, deployment conditions, monitoring, and reassessment workflows. Coordinate enterprise governance committee agendas, case materials, decisions, action items, and follow‑up. Maintain authoritative records of approvals, conditions, exceptions, reassessments, and governance precedents. Develop expedited and standardized pathways for low-risk or well-understood AI uses. Create and maintain reusable review patterns, templates, evidence packages, evaluation assets, and precedents that improve consistency and reduce duplicate work. Provide clear guidance, consultation, education, and stakeholder support to promote early and ongoing engagement and adoption of governance requirements. Track service levels, bottlenecks, and outstanding governance actions and drive closure. Operationalize AI Evaluation and Assurance Implement evaluation requirements proportionate to AI risk, intended use, and deployment context. Coordinate and ensure assessment of performance, reliability, safety, bias, security considerations, and unintended consequences as applicable. Coordinate independent evaluation of consequential AI systems as appropriate. Lead assessment of vendor evidence and claims and identify when supplemental local testing or independent evaluation is required. Facilitate prospective validation, simulation, and other assurance activities as warranted. Translate evaluation findings into governance conditions, remediation actions, monitoring requirements, and escalation. Connect evaluation plans and findings to measurable outcomes where appropriate. Support Governance of Generative and Agentic AI Ensure application of governance and evaluation practices to foundation models, LLMs, multimodal models, and agentic AI. Coordinate review of tool and data access, identity and authorization, human oversight, autonomous actions, memory, agent‑to‑agent interactions, observability, guardrails, testing, and fail‑safe mechanisms. Ensure UTSW maintains practical governance approaches across commercial frontier models, institutionally hosted models, open‑weight models, and internally developed AI. Convert emerging model and agent capabilities, research, and standards into practical review criteria, test protocols, and playbooks. Coordinate AI Portfolio, Vendor and Adoption Operations Maintain enterprise visibility of AI systems, vendors, accountable owners, intended uses, risk classifications, evaluation status, and monitoring obligations. Coordinate vendor due diligence and evidence collection with Procurement, Legal, Compliance, Information Security, IT, Data, and domain teams, supporting build‑buy‑partner decisions without assuming ownership of procurement or technical architecture. Track portfolio priorities, review readiness, dependencies, decisions, and open conditions and prepare concise materials for decision forums. Responsible for continuous adoption through training, guidance, communications, and integration of governance requirements into business, research, clinical, and technology workflows. Promote common methods, reusable evidence, and standard controls to reduce duplication and improve enterprise consistency. Partner on AI Risk Management Work directly with Compliance and Information Security to identify and assess AI‑related risks and to incorporate AI considerations into their established enterprise risk‑management processes. Lead coordination of AI governance requirements, evidence, mitigations, and escalation with the risk‑owning functions and other control #J-18808-Ljbffr 109000 - Office for Enterprise AI