AI/ML Lead

AI/ML Lead

The AI/ML Lead is a senior, hands-on engineer and the first member of ISACA's AI implementation team, a new internal capability that will take on AI engineering work currently delivered by external vendors. Reporting to the VP, Application Engineering, and working within the platform architecture, standards, and governance set by the Senior Director, AI Architecture, Platforms & Agentic Systems, this role builds and hardens the AI solutions that serve ISACA's members and business, both those in delivery today and those that emerge as ISACA's AI portfolio grows. The initial focus is ISACA's personalized member experience, a production RAG and knowledge graph-based recommendation system that integrates ISACA's Body of Knowledge, certification details, workforce skills data, and member profile information. The majority of the role is spent implementing: writing production code, building data and retrieval pipelines, and operationalizing the engineering practices that keep AI systems reliable. Success is measured by production stability, the share of AI engineering work brought in-house, and the implementation team's growing ability to deliver ISACA's current and future AI needs without vendor dependency.

Responsibilities

AI Platform Technical Contribution

• Provide hands-on engineering for the development, integration, and production delivery of ISACA's AI solutions, beginning with the AI-driven member experience, working within the platform architecture and standards set by the Senior Director, AI Architecture. Build and maintain core AI components, including knowledge graph ontology and configuration (Graphwise or a comparable enterprise knowledge graph platform), embedding and retrieval pipelines, recommendation logic, and LLM-powered and agentic application components. Implement the APIs and service contracts that connect AI capabilities to ISACA's digital experiences and business systems. Ensure all components are built for reuse, long-term internal ownership, and maintainability.

AI Vendor Oversight & Insourcing

• Serve as the key internal technical counterpart to external AI development vendors and contractors. Participate in vendor architecture reviews and review vendor code and pull requests, evaluating AI solution designs, including RAG pipelines, knowledge graph implementation, and LLM integration, against ISACA's technical requirements and the engineering standards set by the Senior Director, AI Architecture. Define and maintain acceptance criteria for vendor deliverables, support production readiness validation, and surface technical risk or delivery misalignment to the Senior Director, AI Architecture, and IT leadership. Lead knowledge transfer so that vendor-built components move to internal ownership on a planned timeline, progressively reducing vendor dependency.

AI Engineering Practices & Operations

• Implement and operate MLOps and LLMOps practices in line with the platform standards set by the Senior Director, AI Architecture, including CI/CD for ML and LLM pipelines, model and prompt versioning, evaluation, monitoring, drift detection, and token cost tracking. Establish team-level engineering practices, including code review, documentation, runbooks, and reusable components, that promote quality and consistency across the implementation team and sustain ISACA's AI solutions beyond any single initiative. Foster knowledge sharing so that expertise is built into the organization rather than held by individuals. Implement the security, responsible AI, and auditability controls defined by the Senior Director and Security within the AI solutions the team builds.

Data Infrastructure & AI Data Enablement

• Build and maintain the data foundations that supply ISACA's AI solutions with reliable, well-structured inputs. Implement ingestion and transformation pipelines from source systems such as CRM, third-party content and learning platforms, and external workforce and market trend data. Implement data contracts with source-system owners, plus data quality checks and monitoring aligned to the standards set by the Senior Director, AI Architecture. Identify and document gaps in ISACA's current data architecture that limit AI performance or extensibility, and work with data, engineering, and product teams to close them.

AI Implementation Team Building

• Serve as the first engineer on ISACA's AI implementation team. Recommend roles and hiring sequence to bring outsourced AI engineering in-house, participate in hiring and onboarding, and mentor engineers and contractors.

Stakeholder Communication

• Translate AI system decisions, technical risks, and engineering tradeoffs into clear, actionable communication for product, engineering and IT stakeholders. Collaborate across product, architecture, security, and data teams to ensure AI initiatives integrate coherently into ISACA's broader enterprise.

Other Duties

• Evaluate emerging AI tools, platforms, and architectures through hands-on experimentation and proofs of concept, providing practical recommendations that support the Senior Director's build-versus-buy and platform decisions.

Qualifications

Required Field of Study:

• Bachelor's Degree in Computer Science, Data Science, Statistics, Engineering, or a related field, or equivalent professional experience.

Minimum Years of Experience Required:

• 8+ years of professional software, data, or application development experience, including 2+ years of hands-on experience developing, implementing, or integrating AI, generative AI, LLM-powered, or agentic AI solutions in production environments, beyond proof-of-concept work.

Description of Minimum Relevant Experience Required:

• Production delivery of LLM-powered or AI solutions in cloud environments (Azure strongly preferred or equivalent), including experience with LLM integration, RAG pipeline architecture, and model evaluation beyond proof-of-concept work.
• Proficiency in Python and LLM application frameworks and SDKs; working knowledge of LLM evaluation, RAG pipeline performance tracking, and prompt versioning practices using tools such as Microsoft Foundry, Azure Promptflow, or equivalent evaluation frameworks.
• Hands-on experience with retrieval-augmented generation (RAG) pipelines, prompt engineering, embedding-based semantic retrieval, vector databases (e.g., Azure AI Search, Pinecone, or Weaviate), and LLM orchestration. Familiarity with multi-agent frameworks (e.g., LangChain, Microsoft AutoGen, or comparable) is a plus.
• Hands-on experience with knowledge graph architectures and ontology design; direct Graphwise experience is a strong plus. Working knowledge of Neo4j, Amazon Neptune, or Azure Cosmos DB provides transferable graph reasoning skills and is helpful.
• Foundational MLOps and LLMOps capability: CI/CD for ML and LLM pipelines (Azure DevOps or GitHub Actions), containerized deployment (Docker/Kubernetes), model and prompt versioning, monitoring, token cost tracking, and drift detection. Familiarity with data and workflow pipeline orchestration tools such as Azure Data Factory or Apache Airflow is a plus.
• Demonstrated ability to oversee AI vendor work through technical architecture review, acceptance criteria definition, and delivery validation.
• Strong stakeholder communication skills: proven ability to translate AI architecture decisions, risks, and tradeoffs for non-technical audiences including product and engineering stakeholders.
• Experience working in agile delivery environments with cross-functional product, engineering, data, and security teams.

Preferred Field of Study:

• Masters Degree in Computer Science, Machine Learning, Data Science, or Artificial Intelligence

Preferred Years of Experience:

• 10+ years of professional software or data engineering experience, including 3+ years of hands-on experience developing, implementing, or integrating AI, generative AI, LLM-powered, or agentic AI solutions, ideally including early-stage or greenfield AI systems delivered at enterprise scale.

Description of Preferred Relevant Experience:

• Experience contributing to or helping shape an AI or data practice — establishing engineering practices and delivery patterns. Experience helping build or scale an engineering team is beneficial.
• Domain literacy in IT governance, cybersecurity risk management, or professional certification ecosystems — sufficient to collaborate effectively with ISACA's internal subject-matter experts and align AI outputs to organizational standards.
• Experience with knowledge graph reasoning, or GraphRAG patterns relevant to professional knowledge and competency ontology use cases.
• Familiarity with bias and fairness evaluation for AI-driven recommendation systems, including output-level testing for demographic or professional skew
• Prior experience working in a non-profit, professional association, standards body, or other mission-driven organization is a plus.
• Experience collaborating across hybrid delivery models including in-house staff, contractors, and external development vendors.
• Proficiency with Azure AI platforms, or equivalent services on AWS or GCP. Azure is the primary cloud platform for this role.
• Famili