Staff Software Engineer - Agent Architecture

Job Description

Job Description:\n\n Company Description At PayNearMe, we're on a mission to make paying and getting paid as simple as possible. We build innovative technology that transforms the way businesses and their customers experience payments. Our industry-leading platform, PayXM™, is the first of its kind—designed to manage the entire payment experience from start to finish. Every click, swipe or tap is seamless, fast and secure, helping non-commerce businesses boost customer satisfaction, accelerate payments, and reduce costs. Our single platform handles it all: cards, ACH, digital wallets such as PayPal, Venmo, Cash App Pay, Apple Pay and Google Pay, and even cash at more than 62,000 retail locations nationwide. Today, thousands of businesses across consumer lending, iGaming and online sports betting, property management, and tolling trust PayNearMe to deliver a payment experience that drives real results. In September 2025, we raised a $50 million Series E funding round to accelerate our growth. We're a team of 300+ employees across 41 states, headquartered in Silicon Valley with satellite offices in Dallas, TX and Holmdel, NJ. Join us and be part of a team that's shaping the future of payments—one experience at a time. Responsibilities We build agentic AI products that our customers interact with across different modalities. These agents sit on top of the same money-movement platform that handles real funds for businesses in regulated industries, so they have to be safe, compliant, and predictable in ways most consumer AI products are not. We're looking for a Staff Engineer to own the architecture and implementation of these agents end-to-end. This is a builder role for someone who has shipped agents to production at scale, not just used them. You will define how we build agents at PayNearMe: the frameworks, the integration patterns with our existing systems, the guardrails around money and PII, and the testing/eval/observability loop that lets us improve agents safely over time. Our core platform stack is Ruby on Rails with MySQL (monolith) plus Go microservices on AWS/Kubernetes, with Datadog for observability. Our agents interact with consumers and business partners across a wide variety of use cases. What you'll do You'll define and build the agent platform used across applications at PayNearMe—the architectural patterns, the shared infrastructure—and you'll set the bar for how agents are designed, tested, evaluated, and operated in a regulated, money-movement context.Own the architectural direction for agentic AI at PayNearMe in partnership with other engineering leaders. We are building an agent platform, not a single agent—our business customers have different rules, brand voices, allowed actions, knowledge bases, and compliance postures, and the architecture has to treat per-tenant configuration, isolation, and evaluation as first-class concerns. Produce and maintain architecture documentation (current state, target state, migration plan) and drive alignment across product, engineering, security, and compliance.Design, build, and ship production agents—including voice and chat agents for a wide range of payment-related activities—that integrate cleanly with our Ruby on Rails / MySQL platform and partner services (ElevenLabs, Twilio, and others). Treat tool design as a first-class discipline: tool schemas, descriptions, idempotency, side-effect semantics, and error surfaces directly determine agent quality, and for money-moving tools they determine whether we can stand behind every action the agent took.Make and defend the "what kind of intelligence goes where" decisions: when to lean on a partner's stack vs. orchestrate frontier LLMs directly, when RAG is the right answer vs. tool calls vs. fine-tuning, when a small/fast/cheap model is sufficient vs. when a frontier model is warranted, and where classical ML or deterministic logic is a better fit than an LLM at all. Design the seams that let us swap providers, voice vendors, and models as the landscape shifts—without rewriting the agents that sit on top of them.Design and operate the agent lifecycle as a closed loop: testing, offline evals, online evals, observability, scoring, and a disciplined feedback path from what we observe in production back into the test suite and eval set. Own the rollout discipline for non-deterministic systems: prompt and agent versioning, shadow mode, canary-by-tenant, gradual ramps, and rollback playbooks that account for the fact that the "bad version" may have already taken real payments. The system has to get measurably better over time, not just ship.Own the unit economics of agent interactions. Token budgets, prompt and semantic caching, model cascades (cheap model first, escalate on uncertainty), batch APIs, latency-vs-cost tradeoffs, and per-tenant cost attribution should be instrumented and reasoned about explicitly—at scale, the gap between a well-engineered conversation and a naive one is the business.Build the guardrails that make agents safe in a payments context: scope enforcement, refusal behaviors, deterministic handoffs for anything money-changing, PCI-compliant handling of card data, PII protection, and clear human-in-the-loop or fallback paths when confidence is low. Own the identity and consent model for agent-initiated actions—who the agent is acting as, when step-up authentication is required before a consequential action, and how explicit consent is captured and stored in a form that holds up in a dispute or chargeback. Treat prompt injection and social-engineering of the agent as real attack surfaces; stand up a red-team practice that exercises them continuously, especially against money-moving tools.Treat voice as its own modality, not a text agent with a microphone—design for latency budgets, barge-in and turn-taking, STT/TTS error modes, DTMF fallback, recording and consent, and the operational realities of telephony.Partner with Security, Compliance, and Legal to ensure agent behavior meets PCI-DSS, state-level payments regulations, and our customers' own compliance obligations. Make agent decisions reconstructable: for any consumer interaction we should be able to explain to a regulator, an auditor, or a disputing party exactly why the agent did what it did, on what information, and with what authorization.Raise the bar across the org for agent engineering: define shared patterns for prompts, tools, evals, telemetry, and incident response; serve as a reviewer and approver for architecture decision records (ADRs) and major designs in the agent domain. Teach the rest of engineering how to build, evaluate, and operate agents—most engineers on the team are picking this discipline up for the first time, and the team's velocity depends on how well that knowledge transfers.Partner with the Engineering Managers, Product, and other Staff peers to shape the roadmap—develop deep expertise in both the technical system and the business need (what our customers and their consumers actually want from an agent), and translate that into durable platform capabilities. Minimum requirements8+ years of software engineering experience, with Staff-level scope (cross-team influence, major initiatives, long-term technical direction).Demonstrated experience shipping agentic AI systems to production—not prototypes, not internal copilots, but agents that real users have relied on. You should be able to talk concretely about what broke, what you changed, and how you knew it got better.Hands-on experience with at least one modern agent framework (LangGraph, or comparable). You understand the tradeoffs between graph-based orchestration, ReAct-style loops, and more deterministic state machines, and you have opinions about when each is appropriate.Deep, lived experience with the full agent lifecycle: prompt and tool design, offline and online evaluation, scoring rubrics, observability and tracing, and the discipline of feeding production signals back into your eval set and test suite.Strong system design fundamentals: reliability, consistency, data modeling, and pragmatic API/service boundaries. You can integrate an agent into an existing transactional system without compromising the integrity of that system.Comfortable working in a Ruby on Rails / MySQL environment, or confident in your ability to ramp quickly. You don't need to be a Rails expert, but you need to be able to read the code, work with the team that owns it, and design integrations that fit how the platform actually behaves.Clear communication and strong judgment in high-stakes, cross-functional environments—especially in conversations with Security, Compliance, and Legal where the right answer is rarely the fastest one.Ability to move between high-level architecture and hands-on coding. This role builds. Preferred qualificationsExperience shipping voice agents specifically, and a working understanding of how voice differs from text/chat: latency, turn-taking, STT/TTS failure modes, prosody, barge-in, DTMF, telephony quirks, and recording/consent.Hands-on experience building and shipping voice-enabled applications — including conversational AI, TTS/STT pipelines, or telephony integrations — using any platform or stack (e.g., ElevenLabs, Twilio, Vonage, Deepgram, Vapi, LiveKit, or similar).Experience designing agents that combine multiple styles of intelligence—partner-managed conversational stacks, frontier LLMs, smaller/faster models, RAG, classical ML, and deterministic logic—and choosing among them based on cost, latency, accuracy, and risk.Payments/fintech experience, or other regulated/high-integrity domains (healthcare, lending, insurance). Direct experience with PCI-DSS, TCPA, or similar regimes is a strong plus.Experience building and operating evaluation infrastructure—LLM-as-judge, rubric-based scoring, regression suites, A/B and shadow testing for agents in production.Experience designing agent platforms with multi-tenant configuration—per-customer rules, knowledge, allowed actions, escalation paths, and isolation of evals and guardrails.Experience adversarially testing LLM-based systems (prompt injection, jailbreaking, social-engineering of tool-using agents), particularly where the agent can take consequential actions.Experience optimizing the unit economics of LLM-based products: caching strategies, model cascades, prompt compression, and per-tenant cost attribution.Experience with observability for non-deterministic systems: tracing across LLM calls and tool invocations, capturing the right signals for debugging and eval mining, and turning production traces into test cases.Experience integrating AI capabilities into a Ruby on Rails monolith, or evolving a monolith to support a new class of workload without destabilizing it. The annual base salary range for this role represents PayNearMe's good-faith estimate of the base salary it reasonably expects to offer for this position at the time of hire. Actual compensation may vary based on factors including the candidate's experience, qualifications, skills, and work location. PayNearMe may offer compensation outside of this range in certain circumstances. This position will remain posted until filled. Annual Salary Range $225,000—$285,000 USD Why Join Us?:Competitive salary and benefits with growth-company options grantFast- paced and professional work cultureStock options with standard startup vesting - 1 year cliff; 4 years total$50 monthly communication expense stipend to go towards your phone/internet bill$250 stipend to enhance your WFH setupReimbursement for peripheral equipment: monitor (up to $400), keyboard and mouse (up to $200)Premium medical benefits including vision and dental (100% coverage for employees)Company-sponsored life and disability insurancePaid parental bonding leavePaid sick leave, jury duty, bereavement401k planFlexible Time Off (our team members typically take off ~3-4 weeks per year)Volunteer Time Off13 scheduled holidays PayNearMe strives to create a workplace where all employees thrive. Our core values represent who we are today and we take pride in the way we work with each other as well as with our stakeholders. We're in this together to do the right thing. We deliver real results we are proud of while remaining respectful, transparent, and flexible. PayNearMe is an equal opportunity employer. We are diligently and thoughtfully working towards cultivating a diverse workforce which in turn, enhances our products and services for the communities we serve. Applicants who represent all backgrounds are strongly encouraged to apply. CALIFORNIA CONSUMER PRIVACY ACT: APPLICANT NOTICE Effective Date: January 1, 2020 Last Reviewed on: December 23, 2019 PayNearMe, Inc. (the "Company") is providing you with this Notice ("Notice") to inform you about:the categories of Personal Information that the Company collects and maintains about applicants; andthe purposes for which the Company uses that Personal Information. For purposes of this Notice, "Personal Information" means information that identifies, relates to, describes, is capable of being associated with, or could reasonably be linked, directly or indirectly with, a natural person that the Company may collect in connection with screening applicants for job openings at the Company.Identifiers and Professional or Employment-Related Information. The Company collects identifiers and professional or employment-related information, which may include some or all the following: real name, nickname or alias, postal address, telephone number, e-mail address, membership in professional organizations, professional certifications, language skills, and current and past employment history. The Company collects this Personal Information to evaluate previous job performance and consider applicants for positions, to develop a talent pool and plan for succession, to conduct applicant surveys, to maintain an internal applicant directory and for purposes of identification, to promote the Company as a place to work, and for workforce reporting and data analytics/trend analysis.Personal Information Categories from Cal. Civ. Code § 1798.80(e). The Company may collect categories of Personal Information listed in Cal. Civ. Code §1798.80(e), other than those already listed above, (a) to the extent necessary to comply with the Company's legal obligations, such as to accommodate disabilities; (b) to conduct a direct threat analysis in accordance with the Americans with Disabilities Act and state law; (c) for occupational health and safety compliance and record-keeping; and (d) to respond to an applicant's medical emergency.Characteristics of Protected Classifications Under California or Federal Law. The Company may collect information about race, age, national origin, disability, sex, and veteran status as necessary to comply with legal obligations, including the reporting requirements of the federal Equal Employment Opportunity Act, the federal Office o