Senior AI Engineer

Job Description

Job Description

As a Senior AI Engineer , you will be the technical engine behind every AI implementation the company runs, setting up the models, building the safety and reliability infrastructure, and establishing the engineering standards that every future AI project will inherit.

This is a greenfield role with high ownership. You will be designing and building the foundational AI platform that Hotwire's business units depend on. You'll partner closely with the Director of AI Implementation and AI Champions embedded in each business unit, translating validated workflow proposals into production-grade AI solutions.

Duties / Responsibilities:

• Design and build the core AI platform that connects Hotwire's business applications, data sources, and AI models into reliable, production-grade pipelines
• Own the model deployment layer, configure, version, and maintain LLM endpoints across Azure OpenAI and/or AWS Bedrock with environment isolation (dev / staging / prod)
• Implement a model abstraction layer (e.g., LiteLLM) to ensure portability across model providers and avoid hard vendor lock-in
• Build and maintain an internal AI SDK / shared libraries so that future engineers and CoE projects can bootstrap quickly without reinventing plumbing
• Own infrastructure-as-code and CI/CD pipelines for AI services Other duties as required or assigned.
• Actively participate in Steering Committee reviews, translating technical risk and feasibility into language business leaders understand
• Build and enforce input/output security controls for every AI-facing endpoint:
• PII detection and redaction after data reaches external model APIs
• Prompt injection detection, pattern-based and embedding-based classifiers
• Content policy filtering and output moderation for customer-facing AI surfaces
• Role-based access control to AI capabilities across business units
• Partner with IT Security and Compliance to ensure every AI deployment meets Hotwire's data residency, encryption, and access audit requirements
• Maintain a centralized secrets management approach for API keys, model credentials, and third-party integration tokens
• Implement an LLM evaluation framework that every CoE project must pass after production promotion
• LLM-as-judge pipelines for automated output quality scoring
• Regression test suits that protect against model drift when providers update underlying models
• Semantic similarity and coherence metrics for RAG-based applications
• Golden dataset management and versioning for reproducible evals
• Own the eval harness integration into CI/CD, no model change ships without passing eval thresholds
• Track and report quality metrics to the Director and Steering Committee as part of the AI implementation lifecycle
• Build operational safety infrastructure around AI services:
• Rate limiting and token-budget enforcement per business unit and use case
• Circuit breakers to prevent downstream cascades when model APIs degrade
• Iteration caps and wall-clock timeouts on agentic workflows
• Async queue management and retry logic for high-volume pipelines
• Configure private endpoints and VNet integration for model APIs to keep data off public internet paths
• Implement cost allocation and spend controls so that per-department AI usage is visible and accountable
• Set up comprehensive tracing and monitoring across all AI services using tools such as LangSmith, LangFuse, or equivalent
• Build dashboards that surface latency, error rates, token consumption, quality scores, and cost per workflow, visible to both engineering and business stakeholders
• Establish alerting thresholds and on-call runbooks for AI service degradation
• Maintain audit logs of all model inputs and outputs for compliance review
• Serve as the technical reviewer for AI workflow proposals coming from business unit AI Champions after they reach the Steering Committee
• Write engineering standards, integration patterns, and runbooks that AI Champions and future engineers can follow
• Contribute to vendor evaluations, help assess new AI tooling, model releases, and platform options
• Other duties as required or assigned by supervisor.

Minimum Qualifications:

To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required.

• 5+ year for Senior AI Engineer, building and operating production LLM applications, not prototypes, not demos, production systems with real users and real SLAs
• 5+ years for Senior AI Engineer years of software engineering experience with a strong bias toward system design and production-grade architecture
• Expert-level Python, you write clean, tested, maintainable Python, not just scripts
• Deep understanding of API design, microservices patterns, async programming, and distributed system fundamentals
• Hands-on experience with CI/CD pipelines, containerization (Docker), and cloud-native deployment
• Strong debugging instincts, you can trace a failure from a user-facing symptom down to a model API edge case
• Experience deploying and managing LLMs on enterprise cloud platforms: Azure OpenAI Service or AWS Bedrock
• Proficiency with at least one LLM orchestration framework: LangChain, LangGraph, LlamaIndex, or equivalent
• Hands-on implementation of RAG (Retrieval Augmented Generation) pipelines from chunking strategy to retrieval tuning
• Direct experience implementing AI security controls: PII redaction, prompt injection defense, output filtering
• Familiarity with LLM evaluation approaches, you have built or operated an eval pipeline, not just read about

Benefits:

We truly appreciate and value all our employees and show our appreciation by offering a wide range of benefits, including:

• Comprehensive Healthcare/Dental/Vision Plans
• 401K Retirement Plan with Company Match
• Paid Vacation, Sick Time, and Additional Holidays (including your Birthday!)
• Paid Volunteer Time
• Paid Parental Leave
• Hotwire Service Discounts – for employees who live on a property serviced by Hotwire. Discounted service offerings are provided for high-speed internet, video service, phone, and security service
• Employee Referral Bonuses
• Exclusive Entertainment Discounts/Perks

Hotwire provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

#LI-MC1