Senior AI Engineer

Senior AI Engineer at Hermetic

Remote · Full-time

About Hermetic

Restaurants and event venues lose bookings every day to slow replies. A guest asks about a rehearsal dinner on a Tuesday night, and by the time someone answers on Thursday, they’ve booked somewhere else.

Hermetic fixes that with Mia, an AI assistant that picks up inquiries within seconds, answers questions about rooms, menus, and availability, follows up when a guest goes quiet, and hands the conversation to a coordinator at exactly the right moment.

The Role

You’ll own Mia’s conversations end to end: the agent that writes each reply, the models that decide whether a reply can go out on its own or should wait for a human, and the measurement that tells us whether a change actually made Mia better. The work sits across LLM systems, applied ML, and production engineering.

What You’ll Work On

• The conversation agent: Mia’s replies, routing, tool calls for availability and pricing, handoff detection, and guardrails that keep things accurate and on-brand.
• Autonomy: Improve the models and per-location thresholds that predict which replies need an edit, so more can go out safely on their own.
• Evaluation that drives decisions: Build offline eval sets from real conversations, regression tests for prompts and graph changes, and online metrics tied to what customers care about: edit rate, handoff rate, response time, and bookings.
• Voice: Help take Voice Mia from early adopters to every customer, with per-customer rollout flags, safe call transfer, and instant rollback when something sounds off.
• Running it for real: Tracing and cost tracking on every LLM call, prompt versioning, latency budgets, and being the calm person in the room when a model provider has a bad day.
• Raising the bar: Write the design docs others build from, review code thoughtfully, and help engineers new to LLM systems get good at them fast.

What You Bring

• 6+ years building production software, including 2+ years shipping LLM-powered features that real users depend on
• Strong Python, and comfort reading and changing TypeScript
• Real experience with agent frameworks or tool-using LLM workflows, and opinions about where they break
• Experience building evaluation for a model or LLM system and using it to make actual ship or don’t-ship calls
• Practical ML skills: you can train, calibrate, and threshold a classifier, and explain the precision-versus-coverage trade-off to someone who isn’t an ML person
• Solid production instincts around async queues, idempotency, retries, observability, and cost
• Clear writing: your design docs and PR descriptions make it easy for others to act

Nice to Have

• Voice AI or real-time conversational systems
• Experience in hospitality, events, booking, or conversational commerce
• Human-in-the-loop systems, or learning from human edits
• Hands-on time with SageMaker, Langfuse, or LangSmith

Our Stack

• Services: Python, TypeScript, React
• Data: PostgreSQL, Redis, SQS
• Infrastructure: AWS, Grafana, PostHog (analytics and feature flags)
• Integrations: Twilio, plus the CRMs and booking systems our venues use, such as Salesforce and Tripleseat

How We Work

• We’re a small team with a lot of ownership, and you’ll ship to production every week.
• AI coding tools are part of how we build. We expect you to use them well, and to know when not to trust them.