Senior Applied AI Engineer, Agentic Systems

Job Description

SENIOR APPLIED AI ENGINEER, AGENTIC SYSTEMS

Thresh Consulting | Strategy in Action for the AI Era

Salary: $190,000 to $235,000 + up to 25% performance-based bonus

Location: Hybrid (Denver, Chicago, or NYC preferred), with client travel

ABOUT THRESH

Thresh Consulting is a modern digital consulting firm built for the AI era—where strategy, execution, and craft come together to deliver real outcomes with speed. Our specialty is helping organizations design, build, and scale digital products that create exceptional customer experiences and drive value.

Our engineers are builders who work side by side with product & design. We value clarity over complexity, quality over shortcuts, and learning as a core part of growth.

ROLE OVERVIEW

The Senior Applied AI Engineer builds everything around the model: the agents, harnesses, tools, and evals that turn a capable model into something a business can run on. Some of that work faces customers. Some of it rebuilds how client engineering teams deliver software. All of it ships to production.

Most companies bolted AI onto the way they already worked. They bought coding assistants and chat tools, kept the same handoffs and approval gates, and saw little change. The bottleneck didn't go away; it moved. Your job is to build the systems that remove it.

The model is rented. The harness and the context are owned. That's where quality and cost get decided, and it's where most of your time goes.

You'll spend about 70% of your time building and 30% on technical leadership: guiding 1 to 3 engineers, pairing with client teams, and making the case to engineering leaders for what to build, what to buy, and what to retire.

WHAT YOU’LL DO

CHOOSE THE RIGHT SYSTEMS

• Match each problem to the right pattern: a single agent with tools, an orchestrator with subagents, a deterministic workflow with model steps, or no agent at all
• Default to proven platform agents and SDKs (e.g., Claude Agent SDK, Claude Code, AWS Bedrock), and build custom only where they fall short
• Decide where people stay in the loop: fixed gates for security, privacy, and release, plus dynamic risk flags for anything unfamiliar or high-impact

BUILD AGENTS & THE HARNESS AROUND THEM

• Build multi-step agents that take real actions in client systems, including standard task agents for work every team repeats, such as QA, code review, and triage
• Own the harness: system prompts, context assembly, memory, compaction, retries, and sandboxed execution
• Design tools and MCP servers that give agents structured, least-privilege access to internal systems
• Keep harness and context portable, so a model vendor switch costs a few migration sprints, not a rebuild

BUILD THE CONTEXT LAYER

• Build retrieval and context over codebases, documentation, catalogs, and product decisions, pointed at live systems wherever possible
• Capture what subject-matter experts know, and make keeping it current a standing step in delivery

PROVE IT WORKS & KEEP IT WORKING

• Build eval sets that cover edge cases, out-of-scope requests, and prompt injection, and turn them into automated gates on capability and cost
• Build improvement loops that read execution traces, find failure patterns, and feed fixes back into the harness
• Instrument agents for quality, drift, cost, latency, and the business outcome they exist to move
• Build for security, privacy, and compliance review from the first sprint, with permissions, audit trails, and PII handling

WORK AGENT-FIRST

• Use AI coding agents as your default way of building, configured at the team level with custom tools, hooks, and subagents
• Write specs an agent can implement against, verify the output against the spec, and prototype workflows in days with mock data
• Re-evaluate tools, models, and harness design on a regular cadence. The right setup today would have been wrong six months ago

LEAD & DELIVER

• Own agent systems from concept to production, inside each client's existing stack and constraints
• Mentor engineers, and partner with product and design on scope, technical risk, and estimates
• Explain tradeoffs plainly to technical and business leaders, including what agents still do badly
• Contribute reusable agents, tools, and patterns to Thresh's practice and to ProductOS, our shared context layer for builder teams

WHAT YOU BRING

TECHNICAL

• 7+ years shipping production software, including 2+ years building LLM-powered systems for real users.
• Agents and harnesses you've shipped to production on LLM APIs and agent tooling (e.g., Claude, OpenAI, Gemini; Claude Agent SDK, LangGraph, or similar)
• Production experience with tool integration (MCP or comparable), retrieval, and eval frameworks, including evals that decided what shipped
• Working command of LLM risk: prompt injection, data leakage, and permissions, plus the tradeoffs between cost, latency, and quality
• Back-end strength in Python or TypeScript, including APIs, PostgreSQL, and vector stores
• Working knowledge of cloud AI services (e.g., AWS Bedrock, Azure OpenAI, Vertex AI), CI/CD, and Docker
• Bonus: open-source or published agent work, platform or developer-experience engineering, fine-tuning or classical ML, or delivery inside a large enterprise

HOW YOU WORK

• You ship production systems on time and at a high standard, even as requirements shift
• You read traces, not just outputs, and debug across the model, the harness, and the systems behind them
• You tell clients when an agent isn't the answer, or when a workflow is still a demo
• You've helped other engineers grow, formally or informally
• You build agents in your own time because you want to see what they can do
• Professional services experience is strongly preferred

WHY THRESH

• Senior-level compensation with performance bonus upside
• Comprehensive healthcare benefits
• 401(k) with company matching
• High ownership over technical direction, from the first spec to the last release gate
• Agent work that goes to production at enterprise clients
• Hybrid flexibility with client travel and team co-locations
• A low-ego culture built on clarity, craft, and outcomes

AN EQUAL OPPORTUNITY EMPLOYER

Individual salaries that are offered to a candidate are determined after consideration of numerous factors including internal equity, market data, the candidate’s qualifications- including but not limited to specialty skills, prior relevant industry experience, relevant degrees or certifications. The salary range estimate below is inclusive of all departments, and offered salaries may vary within the given range between departments. A reasonable estimate of the base salary range for this level is $190,000 to $235,000 annualized and other competitive benefits.

Must be legally authorized to work in the United States.