Senior Software Engineer II - AI Engineering

Role Description

Smartsheet is building the next generation of AI-powered strategic planning and work execution agents through SmartAssist, our intelligent agent platform. You will work closely with Engineering and AI Platform teams, embedded in a team working in a critical area and building toward a mature Agent Development Lifecycle (ADLC). You will architect how we design, build, and operationalize AI-driven solutions that transform work execution at enterprise scale.

You Will:

• Design and build production-grade LLM-powered agents and workflows within Smartsheet, including architecture decisions on system design, data pipelines, and deployment strategies.
• Develop and optimize prompts, RAG pipelines, and agent reasoning patterns; build evaluation frameworks to measure accuracy, hallucination rates, and performance across model versions.
• Implement and manage end-to-end AI systems on cloud infrastructure (AWS, GCP, or Azure), including monitoring, optimization, and incident response.
• Collaborate with Engineering, Product, and cross-functional teams to translate business requirements into technical AI solutions that drive real impact.
• Mentor and guide other engineers on applied ML best practices, modern AI development patterns, and production considerations.
• Establish best practices and standards for AI development, model governance, and responsible AI within the organization.
• Stay current with the rapidly evolving AI/LLM landscape and evaluate emerging tools, frameworks, and techniques for Smartsheet's needs.

Qualifications

• 8+ years of software engineering experience, with at least 2 years working directly with LLMs in production.
• Deep, hands-on experience with prompt engineering and context engineering; understanding how model behavior changes with framing, structure, and input design.
• Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis.
• Experience building or extending LLM evaluation frameworks; designed scorers, worked with golden datasets, and thought carefully about what good looks like.
• Fluency in agent system design; able to engage as a peer on architectural tradeoffs that affect quality.
• Strong Python skills; comfortable working in data-heavy environments (Databricks, Delta tables, or equivalent).
• Ability to communicate complex quality findings (written and verbal) to both technical and non-technical stakeholders.
• Strong cross-functional judgment; knows when to escalate, when to resolve independently, and how to build credibility across teams.
• A bias for clarity in ambiguous situations; brings structure and a clear point of view rather than waiting for consensus.
• Legally eligible to work in the U.S. on an ongoing basis.
• BS or MS in Computer Science, a related field, or equivalent industry experience.

Requirements

• Experience with MLflow or similar experiment tracking platforms.
• Familiarity with CI-integrated evaluation pipelines.
• Experience with multi-agent orchestration frameworks.
• Prior work in an Applied AI or LLMOps function within a product company.

Benefits

• Employer subsidized medical/vision and dental coverage for full-time employees.
• 401k Match to help you save for your future (50% of your contribution up to the first 6% of your eligible pay).
• Monthly stipend to support your work and productivity.
• Flexible Time Away Program, plus Sick Time Off.
• US employees are automatically covered under Smartsheet-sponsored life insurance, short-term, and long-term disability plans.
• US employees receive 12 paid holidays per year.
• Up to 24 weeks of Parental Leave.
• Personal paid Volunteer Day to support our community.
• Opportunities for professional growth and development including access to Udemy online courses.
• Company Funded Perks, including a counseling membership, local retail discounts, and your own personal Smartsheet account.
• Teleworking options from any registered location in the U.S. (role specific).
• Competitive base salary range for roles that may be hired in different geographic areas.