AL/ML Evaluation Engineer

ML / AI Evaluation Engineer We need an experienced engineer to lead AI evaluation initiatives supporting public health missions. You will translate advanced AI capabilities into mission‑specific workflows, collaborate with ML engineers, data scientists and product teams, and design scalable ML and Generative AI solutions, including AI agents, retrieval‑augmented generation (RAG) pipelines and AI evaluation frameworks. Location: Atlanta, GA. Responsibilities Build and maintain scalable data pipelines using PySpark and Palantir Foundry for AI, analytics, and scientific evaluation workflows. Design and implement ML and Generative AI workflows, including AI agents, RAG pipelines, and AI evaluation frameworks. Integrate advanced AI technologies such as Codex and Claude to support mission‑specific workflows and real‑time decision making. Develop evaluation strategies covering model quality, retrieval optimization, benchmarking, and performance monitoring for deployed AI systems. Design AI architectures supporting memory, state management, structured/unstructured public health data interaction, and scalable agent orchestration. Establish data governance, privacy, anonymization, documentation, and ethical AI standards across AI/ML systems and public health data environments. Qualifications 5+ years of experience with Generative AI, LLMs, AI agents, or RAG applications, and designing, developing, and deploying ML models and AI solutions using Python. 3+ years of experience with AI agents and AI evaluation strategies in enterprise environments. 2+ years of experience with deep research evaluation methodologies and AI evaluation workflows. Experience with ML frameworks such as TensorFlow or PyTorch for production‑grade model development. Experience with data engineering using PySpark, SQL, and Palantir Foundry, including Foundry AIP. Experience with MLOps platforms such as MLflow and cloud environments, including Azure. Knowledge of public health, healthcare, or government data systems and associated governance practices. Ability to design and optimize AI systems involving retrieval workflows, memory or state management, and real‑time decision‑support capabilities. Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements. Bachelor's degree in CS, Engineering, or Data Science. Nice to Have Experience working in healthcare, biomedical, or government public health AI/ML environments. Experience with conversational AI, chatbot systems or full‑stack AI application development. Experience with containerization, CI/CD, orchestration, and production MLOps pipelines. Experience with Agile delivery environments and tools such as Jira. Experience writing technical documentation and presenting AI/ML solutions to various audiences. Experience integrating enterprise AI tools such as Codex, Claude or similar AI enablement platforms. Knowledge of enterprise AI governance, compliance, and ethical AI frameworks. Knowledge of AI systems for retrieval, ranking, and scientific evaluation use cases. Ability to collaborate effectively in matrixed, cross‑functional organizations. Master's degree in CS, Data Science, ML or a related field. Compensation Projected annualized salary range: $128,700.00 to $292,000.00 (USD). Salary is determined by factors such as location and experience. This posting will close within 90 days from the posting date. EEO Statement All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, local, or international law. #J-18808-Ljbffr Booz Allen Hamilton