Azure OpenAI Engineer

Oaks,PA, Hybrid 3 days onsite

MUST HAVE 3+ years hands-on agent or LLM application development, having personally built and shipped applications using agent frameworks or SDKs such as LangChain, LangGraph, the OpenAI Agents SDK, Microsoft Agent Framework, or equivalent. This is non-negotiable for this position.

6+ years professional software engineering, with production depth in at least two of C#, Python, Java and TypeScript or JavaScript, and the ability to work across all four. There is no one-language-per-engineer model on this team.

2+ years building automated testing or evaluation for LLM or agent applications, using frameworks such as the Azure AI Evaluation SDK, Promptfoo, DeepEval, Ragas, LangSmith evaluation or equivalent. This includes handling non-deterministic output, building golden datasets, and running evaluation as a gate inside CI rather than as a manual exercise.

2+ years working with agent memory and retrieval in application code: conversation and session state, short-term and long-term memory, and retrieval-augmented generation against a vector or search backend. You do not need to build the stores, but you must know what an agent developer needs from them and how these interfaces behave inside an agent loop.

3+ years designing and maintaining libraries or SDKs consumed by other engineering teams: public API design, versioning, backward compatibility, and packaging to registries such as NuGet, PyPI, Maven or npm.

3+ years building CI/CD pipelines using GitHub Actions, Azure DevOps or equivalent, including quality gates that block promotion.

2+ years working with LLM application patterns in production: tool and function calling, structured output, streaming, context management, retries and fallback across providers, and prompt handling.

Fluency with AI-assisted development tooling such as GitHub Copilot, Claude Code or Cursor, used daily as part of how you work. Team sizing on this engagement assumes that leverage.

2+ years working with containerised deployment and a major cloud platform, sufficient to build scaffolds that deploy cleanly into Kubernetes and managed runtimes.

Demonstrable developer experience judgement: building tooling, templates or libraries that other engineers adopted willingly, and iterating based on their feedback.

Working knowledge of OpenTelemetry and how tracing is instrumented inside a client library.

For applications and inquiries, contact:View email address on us.fitly.work