Senior AI Engineer, Agentic Systems & Runtime Architecture

Together we fight for everyone’s opportunity for a better financial future. We will do this together — with customers, partners and colleagues. We will fight for others, not against: We will stand up for and champion everyone’s access to opportunities. The status quo is not good enough … we believe every individual and every community deserves access to financial opportunities. We are determined to support both individuals and communities in reaching a better financial future. We know that reaching this future depends on our actions today. Like our Purpose Statement, Voya believes in being bold and committed to action. We are committed to a work environment where the differences that we are born with — and those we acquire throughout our lives — are understood, valued and intentionally pursued. We believe that our employees own our culture and have a responsibility to foster an environment where we all feel comfortable bringing our whole selves to work. Purposefully bringing our differences together to positively influence our culture, serve our clients and enrich our communities is essential to our vision. Are you ready to join a company with a strong purpose and a winning culture? Start your Voyage – Apply Now About the Role We’re looking for a hands-on Senior AI Engineer to lead the design, build, and operation of production agentic AI systems—including multi-agent research assistants that deliver cited, grounded answers via both conversational experiences and programmatic APIs. You’ll own “runtime architecture” decisions (orchestration/routing, retrieval strategy, model serving patterns, and runtime controls) and help evolve our capabilities toward more sophisticated agentic design: planner/supervisor orchestration, advanced retrieval + reranking, evaluation gates (AgentOps), agentic security, and end-to-end observability. What You’ll Do Collaborate with business and technical stakeholders to translate real-world research and workflow needs into AI-powered solutions that are measurable, reliable, and safe in production. Architect and build multi-agent workflows (planner/supervisor + specialist agents) with explicit state management and routing, and interoperability via emerging agent protocols (MCP for tool integration, A2A for agent-to-agent delegation) designed for non-deterministic behavior and real operational constraints. Design and continuously improve retrieval architectures for research assistants (hybrid retrieval + reranking), including advanced strategies such as contextual retrieval / contextual embeddings to reduce retrieval failures and improve grounding coverage. Establish and operationalize AgentOps-style evaluation gates: treat the agent as a versioned artifact (model + prompt + tools + guardrails + eval thresholds), run statistical evaluation suites, and use staged rollout approaches to manage risk while maintaining iteration speed. Implement agentic security controls for systems that ingest external content and use tools/APIs, including defenses against prompt injection and unsafe/over-broad tool execution. Build production-grade observability across multi-step agent executions (traces/metrics/logs), define SLIs/SLOs for reliability and performance, and use telemetry to debug and improve probabilistic runtime behavior. Own reliability outcomes: performance and cost tradeoffs (latency/throughput/cost), failure isolation, and incident response for AI-driven components. Partner effectively with platform, security, and governance functions—ensuring enterprise standards are met while runtime architecture accountability stays with the team operating the production AI behavior. Rapid learner with a hands-on mindset — able to quickly ramp up on emerging AI frameworks and tooling, prototype rigorously, and translate new developments into production-ready implementations with engineering discipline. What You Bring Proven experience designing and building LLM-powered applications in production, including prompt/tool orchestration and grounded response patterns. Hands-on experience implementing multi-agent orchestration (planner/supervisor patterns, tool chaining, state management, and conditional routing. Strong understanding of advanced retrieval for RAG: hybrid retrieval, rank fusion concepts, and reranking, with bonus points for contextual retrieval/contextual embeddings approaches. Demonstrated ability to build evaluation systems for non-deterministic AI/agent behavior (rubrics/metrics, regression suites, and release gates), replacing “vibe checks” with systematic improvement loops. Experience with AgentOps / LLMOps practices, including staged rollout models and continuous monitoring for quality, drift, safety, and cost-per-task. Strong security mindset for LLM applications, including awareness of prompt injection (direct and indirect) and defense-in-depth patterns (input sanitization, structured prompts, output validation, least privilege, HITL where appropriate). Proficiency in Python and modern AI engineering frameworks commonly used for agentic systems (e.g., graph-based orchestration patterns and RAG integration toolkits). Experience designing and managing agent memory systems (working, long-term, episodic) and scalable prompt architectures — including version-controlled prompt libraries, hot-swap update patterns, and persona-specific prompt management across multi-agent systems. Experience building production telemetry and diagnosing distributed, multi-hop workflows using tracing/metrics/logs (OpenTelemetry-style concepts are a plus). Bonus: familiarity with Databricks, Azure Foundry and other cloud AI platform patterns and operational requirements for model/agent lifecycle management (versioning, promotion, rollback, policy enforcement, telemetry). Bonus: experience in regulated or audit-minded environments where governance, traceability, and operational resilience matter. Why Join Us Own end-to-end runtime architecture for production agentic systems where engineering decisions materially drive quality, reliability, and risk outcomes. Build advanced agentic capabilities (planner/supervisor orchestration, contextual retrieval, eval gates, security controls, observability) that distinguish “production-grade” from “prototype Help shape how agentic systems are governed and operated using a purpose-built AgentOps model aligned to regulated-firm needs (evaluation-driven gates, staged rollout, continuous monitoring). Work with a team that treats multi-agent systems with the same discipline as distributed systems: contracts, observability, SLOs, and blast-radius control Ready to build production-grade agentic AI? Apply now or reach out to learn more. #LI-LW1 Compensation Pay Disclosure: Voya is committed to pay that’s fair and equitable, which means comparable pay for comparable roles and responsibilities. The below annual base salary range reflects the expected hiring range(s) for this position in the location(s) listed. In addition to base salary, Voya offers incentive opportunities (i.e., annual cash incentives, sales incentives, and/or long-term incentives) based on the role to reward the achievement of annual performance objectives. Please note that this salary information is solely for candidates hired to perform work within one of these locations, and refers to the amount Voya Financial is willing to pay at the time of this posting. Actual compensation offered may vary from the posted salary range based upon the candidate’s geographic location, work experience, education, licensure requirements and/or skill level and will be finalized at the time of offer. Salaries for part-time roles will be prorated based upon the agreed upon number of hours to be regularly worked. $160,000-$174,000 Be Well. Stay Well. Voya provides the resources that can make a difference in your lives. To us, this means thriving physically, financially, socially and emotionally. Voya benefits are designed to help you do just that. That’s why we offer an array of plans, programs, tools and resources with one goal in mind: To help you and your family be well and stay well. What We Offer Health, dental, vision and life insurance plans 401(k) Savings plan – with generous company matching contributions (up to 6%) Voya Retirement Plan – employer paid cash balance retirement plan (4%) Tuition reimbursement up to $5,250/year Paid time off – including 20 days paid time off, nine paid company holidays and a flexible Diversity Celebration Day. Paid volunteer time — 40 hours per calendar year Learn more about Voya benefits (download PDF) Critical Skills At Voya, we have identified the following critical skills which are key to success in our culture: Customer Focused: Passionate drive to delight our customers and offer unique solutions that deliver on their expectations. Critical Thinking: Thoughtful process of analyzing data and problem solving data to reach a well-reasoned solution. Team Mentality: Partnering effectively to drive our culture and execute on our common goals. Business Acumen: Appreciation and understanding of the financial services industry in order to make sound business decisions. Learning Agility: Openness to new ways of thinking and acquiring new skills to retain a competitive advantage. Learn more about Critical Skills Equal Employment Opportunity Voya Financial is an equal-opportunity employer. Voya Financial provides equal opportunity to qualified individuals regardless of race, color, sex, national origin, citizenship status, religion, age, disability, veteran status, creed, marital status, sexual orientation, gender identity, genetic information, or any other status protected by state or local law. Reasonable Accommodations Voya is committed to the inclusion of all qualified individuals. As part of this commitment, Voya will ensure that persons with disabilities are provided reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please reference resources for applicants with disabilities. Misuse of Voya's name in fraud schemes Voya Financial, Inc., is a leading retirement, employee benefits and investment management company. Voya’s services and solutions help clear the path to financial confidence and a more fulfilling life for individual, workplace and institutional clients, supporting more than 18 million customer relationships. Certified as a “Great Place to Work” by the Great Place to Work® Institute, Voya fosters a culture that values customer centricity, integrity, accountability, agility and inclusivity. Together with customers and partners, Voya employees fight for everyone's opportunity for a better financial future. For more information visit Voya’s values serve as our compass, bringing together our culture, performance, brand, vision, strategy and corporate responsibility activities. Our values guide our decision-making and behaviors allowing us to deliver worth to our customers, distribution partners, shareholders and each other. With a lens on balance, health & welfare, our work environment offers flexibility. The majority of our roles have adopted a hybrid schedule — working some days in the office and some days remotely, providing flexibility for work-life balance and still allowing the benefit of in-person collaboration. Some functions will continue to be 100% on-site and others may be fully remote, depending on business needs. We care We do the right thing We have customer passion We have a winning spirit We are the We Certified by the Great Place to Work® Institute as a Great Place to Work® for the 10th consecutive year, Voya is equally committed to conducting business in a way that is socially, environmentally, economically and ethically responsible. Voya’s culture is proudly recognized by multiple organizations. See a full listing of our awards and recognition.