Director of Engineering, IAEF

Overview

The Founding Vice President, Engineering, leads the architecture, delivery, and engineering team for Independent AI Evaluation Foundation (IAEF), a platform for evaluating the trustworthiness of artificial intelligence (AI). Reporting to the CEO of IAEF, this leader combines technical ownership, team development, security leadership, and external representation.

Independent AI Evaluation Foundation (IAEF)’s mission is to identify and reduce barriers to evaluating the efficacy, transparency, and safety of specific educational AI-centric solutions used by students and opportunity seekers in their educational context. IAEF is a fiscally sponsored project of Digital Harbor Foundation.

Digital Harbor Foundation is dedicated to fostering learning, creativity, productivity, and community through education with a vision of digital equity for everyone. Driven by our sincere belief that access to opportunity is a basic right, we take bold yet practical actions to support making a better future now. Through a portfolio of projects focused on developing leadership within communities, we support those closest to challenges to take deliberate actions based in design thinking approach, backed by data analysis, grounded in a practice of collective impact, and driven by a commitment to racial equity.

Education and Experience Requirements

Required

• 12+ years of software engineering experience, including at least four years in senior technical leadership at a startup, evaluation or audit firm, machine-learning infrastructure company, or research-grade platform team.
• Experience building and operating a platform with a plugin, extension, or component model, and running AI evaluation, red-teaming, machine-learning observability, or model-testing systems in production.
• A record of hiring and retaining senior engineers in a distributed team, including at least one applied machine-learning or agentic-systems specialist.

Strongly preferred

• Prior CTO, VP Engineering, or founding-engineer experience at an early-stage company that raised institutional capital; experience enabling external developers to build on a platform; and prior evaluation-methodology work with academic or civil-society partners.
• A PhD and extensive experience at a frontier AI research laboratory are not required.

Knowledge, Skills, and Abilities

• Ability to design reusable platform components and work with human-in-the-loop (HITL) transcripts, benchmark scores, and large language model (LLM) red-teaming results.
• Ability to communicate technical decisions to enterprise security leaders, foundation program officers, investors, customers, and partners.
• Ability to prioritize competing technical and organizational needs and lead a distributed engineering team through delivery, operational issues, and growth.
• Understanding of the operational separation required between a public benefit corporation, a companion nonprofit, and grant-funded programs.
• Strongly preferred: experience leading implementation of ISO 42001, the NIST AI Risk Management Framework, or SOC 2 requirements, rather than participating only as a reviewer.

Additional Notes

This is a full-time, remote-first role based in the US or EU. New York or the San Francisco Bay Area is strongly preferred for quarterly in-person meetings with the CEO of IAEF, partners, and the board. Travel is approximately 15% for funder meetings, partner sites, conferences, and off‑sites.

Role and Responsibilities

Platform architecture and delivery

• Own the Workbench’s Design → Test → Analyze → Publish (DTAP) workflow, Component Contract, Artifacts API, component registry, and canvas interface.
• Create a persistent, structured, queryable evaluation history rather than one-off engagement exports, and integrate the existing red‑teaming platform, administrative dashboard, and HITL data‑capture architecture instead of rebuilding them from scratch.
• Decide when to build or buy authentication, storage, orchestration, LLM-provider services, and sandboxing for the planned notebook environment.

Engineering leadership

• In conjunction with the Human Resources Department of Digital Harbor Foundation, recruit, develop, and retain the engineering organization against the approved build plan, including Staff/Senior Backend, Senior Frontend, DevOps/site reliability, Applied Machine Learning, and Senior Product Manager roles
• Set planning, code‑review, on‑call, incident‑response, and roadmap‑review practices; provide engineering progress and risks for board and funder reporting.

Evaluation platform strategy

• Partner with the IAFE’s CEO and Evaluation Science Lead to translate HITL red teaming, automated benchmarks, LLM red teaming, LLM-as‑judge, and drift detection into a Component Contract that supports all three Test artifact types and planned agentic components.

Security, compliance, and trust

• Lead vendor risk reviews for LLM providers, cloud infrastructure, and third‑party evaluation tools.
• Coordinate with Digital Harbor Foundation’s Leadership, where applicable, on required approvals, funder restrictions, and separation of PBC and nonprofit resources, data, and intellectual property under approved agreements. Escalate unresolved risks rather than independently determining legal or sponsor policy.

External representation and fundraising

• Represent IAEF in technical conversations with customers, partners, funders, and press, and own the technical content of investor fundraising decks, roadmap materials, demonstrations, and due diligence reviews in partnership with the Executive Director of IAEF and Digital Harbor Foundation’s leadership.
• Lead the developer‑relations narrative for software and agent development kits (SDK/ADK) as the Component Contract opens to outside developers in Phase 1, partnering with the Developer Relations & Ecosystem hire planned for Period 2.

Other Duties

This job description is not a comprehensive listing of all activities, duties, or responsibilities required for this role. Duties, responsibilities, and activities may change at any time with or without notice.

Compensation

Compensation for this full‑time position is $215,000 - $315,000 annually, commensurate with experience.

Digital Harbor provides a best‑in‑class, comprehensive set of benefits to support the team. All regular, full‑time employees are eligible to receive:

Health Benefits & Insurance

• Carefirst Blue Cross Blue Shield - Health, Dental, and Vision Insurance (100% of the premium paid for employees and 85% of dependents)
• Pre‑Tax Health Savings Account (HSA) (with $365 monthly employer contributions)
• Pre‑Tax Flexible Savings Account (FSA)
• Paid Accidental Death & Dismemberment (AD&D) Insurance
• Paid Short‑Term & Long‑Term Disability Insurance
• Paid Basic Life Insurance
• Supplemental Voluntary Life Insurance (Employee, Spouse & Dependent Children)
• Total Pet Plan and Supplemental Wishbone Pet Insurance
• Employee Opportunity Program (EAP) - Health and Wellness
• Wellness Reimbursement Program

Retirement

• 401k Retirement Plan (with 6% matching)

Paid Time Off

• 15 Days Paid Time Off Per Year
• 20 Days Paid Time Off Per Year (after 3rd Anniversary)
• 25 Days Paid Time Off Per Year (after 6th Anniversary)
• 16 Paid Holidays (14 common plus 2 flexible holidays, including Dec 25 - Jan 1)
• Paid Bereavement Leave
• Paid Parental Leave for Moms and Dads (two weeks after first year)

If our mission and vision align with your personal values, please apply!

A cover letter outlining your qualifications for the position, along with your resume, is required. Interviews will be conducted virtually.

Digital Harbor and IAEF are equal opportunity employers.

#J-18808-Ljbffr Jobleads-US