Machine Learning Engineer

Overview

The Machine Learning Engineer owns the intelligence layer that makes Evergreen's agents clinically meaningful rather than simple rule executors. This person designs, trains, validates, and monitors the ML models that power the Markov state transition logic, clinical decision support, patient risk stratification, adherence prediction, and dosage optimization. They work at the intersection of clinical domain knowledge and applied ML - building models that must be explainable to providers, safe for patients, and auditable by regulators. As the architecture matures, this role expands into the predictive analytics layer (Amazon Forecast DeepAR+ for churn risk, adherence forecasting) and the evaluation frameworks that determine whether agent recommendations meet clinical safety thresholds. This is not a research role - it is an applied ML engineering role where every model must ship, every prediction must be defensible, and every failure mode must be anticipated.

Job Activities and Responsibilities: The responsibilities listed below are subject to change.

• Build the evaluation and validation framework for all agent-driven clinical recommendations - including safety guardrails, confidence thresholds, and human-in-the-loop escalation triggers for the Clinical Protocol Agent
• Develop patient risk stratification models for adherence prediction, adverse event likelihood, dosage titration optimization, and churn/dropout risk using clinical, behavioral, and engagement signals
• Implement and manage the predictive analytics pipeline on AWS - Amazon Forecast (DeepAR+) for time-series clinical predictions, S3 Vectors for embedding-based patient similarity and retrieval, and Bedrock for agent inference
• Design and build the RAG architecture that grounds agent responses in clinical protocols, formulary data
• Own model lifecycle management - training pipelines, feature stores, model versioning, A/B testing, drift detection, and retraining triggers in production
• Build explainability layers for clinical recommendations
• Collaborate with the clinical team to translate clinical protocols and pharmacy domain knowledge into model features, training labels, and validation criteria
• Establish model monitoring and alerting - prediction quality dashboards, distribution shift detection, and automated alerts when model performance degrades below clinical safety thresholds
• Work with the DevOps/AgentOps teammate to ensure all ML decisions are logged, reproducible, and auditable for regulatory review

Required Technical Skills

(Staff-Level Depth Expected)

• Applied ML / Statistical Modeling (Expert): Deep experience with classification, regression, time-series forecasting, and probabilistic graphical models (Markov chains, HMMs, Bayesian networks). Must have shipped ML models to production - not just notebooks. Strong foundation in experimental design, A/B testing methodology, and statistical significance in clinical contexts.
• NLP & LLM Engineering (Expert): Production experience with LLM-based systems - prompt engineering, fine-tuning, RAG architecture design, embedding models, and output evaluation. Must understand hallucination mitigation, grounding techniques, and how to constrain LLM outputs to clinically safe boundaries. Experience with Bedrock (Claude, Titan) preferred; equivalent depth with OpenAI, Vertex AI, or Azure OpenAI acceptable.
• AWS ML Services (Strong): Hands-on experience with at least 3 of: SageMaker (training, endpoints, pipelines), Bedrock (agents, knowledge bases, model invocation), Amazon Forecast (DeepAR+, predictor optimization), S3 Vectors, Comprehend Medical, or HealthLake analytics. Must be comfortable building end-to-end ML pipelines on AWS.
• Feature Engineering & Data Pipelines (Strong): Experience building feature stores and training data pipelines from healthcare data sources. Must understand FHIR resource structures well enough to extract clinically meaningful features from Patient, Observation, Condition, MedicationRequest, and related resources. Experience with data normalization challenges (free-text to structured, terminology mapping) is critical.
• Model Evaluation & Safety (Strong): Experience designing evaluation frameworks for high-stakes predictions - clinical decision support, medical device software, or similarly regulated domains. Must understand sensitivity/specificity tradeoffs in clinical contexts, how to set appropriate confidence thresholds, and when a model should defer to human judgment.
• MLOps & Production ML (Strong): Model versioning (MLflow, SageMaker Model Registry, or equivalent), automated retraining pipelines, drift detection, shadow deployments, and canary rollouts for model updates. Must have experience monitoring models in production beyond accuracy metrics - latency, cost, fairness, and distributional stability.

Required Experience

• 6+ years in ML engineering or applied data science, with at least 3 years shipping ML models to production in a healthcare, biotech, or clinical domain
• Direct experience building clinical decision support, risk stratification, or patient outcome prediction models - must understand the regulatory and ethical implications of ML in healthcare
• Production experience with LLM-based agent systems - must have built or significantly contributed to a system where an LLM makes consequential decisions with safety guardrails
• Demonstrated ability to collaborate with clinical domain experts (physicians, pharmacists, clinical researchers) to translate domain knowledge into model design decisions
• Experience working in HIPAA-regulated environments - must understand de-identification requirements, minimum necessary data access, and audit trail requirements for ML training data
• Track record of building explainable models in regulated contexts - must be able to articulate why a model makes a specific prediction to both technical and clinical audiences

Preferred Qualifications

• Advanced degree (MS or PhD) in machine learning, computational biology, biomedical informatics, statistics, or a related quantitative field
• Experience with GLP-1 / obesity medicine clinical workflows, compounding pharmacy operations, or weight management program analytics
• Familiarity with FDA Software as a Medical Device (SaMD) guidance and how it applies to clinical decision support systems
• Experience with Comprehend Medical for clinical NLP (entity extraction, ICD/RxNorm mapping from unstructured text)
• Prior work with SMART on FHIR or CDS Hooks for integrating ML-driven recommendations into clinical workflows
• Publications or patents in healthcare ML, clinical NLP, or related domains (valued but not required)
• Experience with causal inference methods for treatment effect estimation in observational clinical data

Physical Requirements:

• Proficient in using a computer and related equipment, including printers and fax machines.
• Ability to sit or stand for extended periods.
• Effective communication skills via telephone and email.
• Capable of lifting up to 40 pounds as needed.

Benefits:

• Health care insurance (medical, dental, vision)
• Life Insurance
• Supplemental Insurance
• PTO
• 401K matching
• Sick leave
• Phone/internet reimbursement
• Remote work
• Top of the line machines