Azure AI Engineer (RAG Optimization)

Job Description

Job Description

Job Title: Azure AI Engineer Retrieval Augmented Generation (RAG)

Job Description

We are seeking a hands-on Azure AI Engineer to optimize and enhance an existing Retrieval Augmented Generation (RAG) application built within the Microsoft Azure AI ecosystem. The core architecture and infrastructure are already in place and are approximately 80% functional. Your primary goal will be to improve retrieval accuracy, response quality, and overall performance for a customer-facing knowledge application used by a product support organization. This is a technical execution role focused on building, tuning, and refining RAG solutions, rather than AI architecture or strategy.

Responsibilities

• Optimize and refine an existing RAG implementation built in Azure AI Foundry to improve performance and reliability.
• Improve retrieval accuracy, ranking, reranking, and context selection to deliver more relevant responses for customer-facing use cases.
• Configure and enhance document ingestion pipelines, including chunking strategies, metadata tagging, indexing approaches, and search optimization techniques.
• Evaluate and tune development and staging environments to improve response quality and system performance.
• Work with large document sets such as manuals, specifications, and technical documentation to improve retrieval effectiveness and coverage.
• Enhance prompt engineering techniques and context window strategies to increase first-response accuracy and reduce the need for follow-up queries.
• Collaborate with business and technical teams to understand requirements and implement a backlog of planned enhancements and improvements.
• Test and validate retrieval and response quality using real-world customer support scenarios and knowledge application use cases.
• Continuously monitor, analyze, and iterate on RAG pipeline performance to ensure ongoing optimization of search relevance and response quality.

Essential Skills

• Hands-on experience with Azure AI Foundry, including building and refining AI applications within the Azure ecosystem.
• Practical experience building, tuning, or optimizing Retrieval Augmented Generation (RAG) solutions in production or near-production environments.
• Experience with Vibe Coding and the ability to apply it in the context of Azure AI and RAG pipelines.
• Strong understanding of document ingestion processes, including chunking strategies, metadata tagging, indexing, ranking, and reranking.
• Proven experience improving search relevance and retrieval accuracy within AI-driven applications.
• Knowledge of vector search, semantic search, and retrieval optimization concepts and how to apply them in Azure-based solutions.
• Experience testing and validating prompt strategies and AI response quality, including iterative prompt engineering and evaluation.
• Ability to work independently within an established environment and deliver incremental enhancements without extensive supervision.
• mid-level Azure AI Engineer, Applied AI Engineer, Machine Learning Engineer, Search/Relevance Engineer, or Software Engineer with direct experience building and tuning RAG solutions in Azure.

Additional Skills & Qualifications

• Experience supporting customer service or knowledge management AI solutions, particularly those used by product support or help desk organizations.
• Experience with Azure Cognitive Search or Azure AI Search and integrating these services into RAG architectures.
• Familiarity with large language model (LLM) evaluation frameworks and methods for testing response quality and robustness.
• Experience working with technical documentation, manuals, and product support content to design effective retrieval and search strategies.
• Comfort collaborating with both business stakeholders and technical teams to translate requirements into concrete AI enhancements.

Work Environment

You will work within an established Azure AI environment where the core RAG architecture and infrastructure are already deployed and largely functional. The role focuses on iterating and improving an existing customer-facing knowledge application used by a product support organization. You will spend most of your time working with Azure AI Foundry, RAG pipelines, Azure-based search technologies, and large sets of technical documentation such as manuals and specifications. The environment emphasizes practical execution, experimentation, and data-driven optimization of retrieval and response quality. Collaboration with cross-functional teams, including product support and engineering stakeholders, will be a regular part of the work, while still allowing for independent, focused development and testing.

This is a Contract position based out of Milwaukee, WI.

The pay range for this position is $65.00 - $85.00/hr.

Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors.

Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave)

This is a fully onsite position in Milwaukee,WI.

This position is anticipated to close on Oct 9, 2026.

About Actalent

Actalent is a global leader in engineering and sciences services and talent solutions. We help visionary companies advance their engineering and science initiatives through access to specialized experts who drive scale, innovation and speed to market. With a network of almost 20,000 consultants and 5,000 clients across the U.S., Canada, Asia and Europe, Actalent serves many of the Fortune 500. We are proud to be an Engineering News-Record (ENR) Top 500 Design Firm for our engineering design services and a ClearlyRated Best of Staffing® winner for both client and talent service.

The company is an equal opportunity employer and will consider all applications withoutregard to race, sex, age, color, religion, national origin, veteran status, disability,sexual orientation, gender identity, genetic information or any characteristic protectedby law.

If you would like to request a reasonable accommodation, such as the modification oradjustment of the job application process or interviewing process due to a disability,please email View email address on ziprecruiter.com for other accommodation options.

San Francisco Fair Chance Ordinance: Pursuant to the San Francisco Fair Chance Ordinance, for all positions located in thecity and county of San Francisco, we will consider for employment qualified applicantswith arrest and conviction records.

Massachusetts Lie Detector: It is unlawful in Massachusetts to require or administer a lie detector test as acondition of employment or continued employment. An employer who violates this lawshall be subject to criminal penalties and civil liability.

Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI) to support parts of our hiring process,including sourcing, screening, and evaluating candidates. AI helps assess applicationsand qualifications, but final decisions are made by our hiring team. By applying, youacknowledge and agree that your application may be reviewed using AI tools.