AI Quality Engineer

Our client is seeking a Quality Analyst (AI Quality Engineer) 2 to join their team.

Job Title: Quality Analyst (AI Quality Engineer)

Location: Remote, however looking for candidates to work PST hours

Pay Range: $ 33/hr. - $36/hr.

What's the Job?

Content Automation Specialist

Overview:

• We are seeking a highly technical and strategic Content Automation Specialist to transform how content quality is validated across Microsoft 365 and Copilot experiences.

• This role focuses on designing scalable evaluation systems, automated testing workflows, and AI-assisted quality frameworks that increase efficiency while maintaining human-centered quality standards. The ideal candidate understands where automation can replace manual review, where human evaluation remains critical, and how to build systems that continuously improve content quality at scale.

• This individual will partner with cross-functional teams to define testing strategies, build evaluation frameworks, develop automation opportunities, and shape the future of content quality operations.

Responsibilities :

Quality Systems Strategy

• Design scalable testing and evaluation frameworks for content, templates, prompt templates, and AI-generated experiences.

• Define quality measurement approaches and establish testing standards across content ecosystems.

• Create recommendations for balancing automation, AI evaluation, and human review.Automation & Tooling.

• Develop and implement automated testing workflows and validation systems.

• Build tools and processes that reduce manual effort while maintaining content quality.

• Identify opportunities to leverage AI for defect detection, content validation, and quality assurance.

Human Evaluation & AI Quality

• Design human-in-the-loop evaluation frameworks for assessing AI outputs.

• Establish quality rubrics, benchmarking criteria, and testing methodologies.

• Determine where human judgment adds measurable value and where automation can scale operations.

Data & Optimization

• Analyze testing metrics and operational data to identify improvement opportunities.

• Develop recommendations that improve quality, efficiency, and testing coverage.

• Measure the effectiveness of testing systems and continuously optimize workflows.

Cross-Functional Leadership.

• Partner with cross-functional teams on automation recommendations and evaluations of quality risks.

• Drive adoption of testing standards and scalable evaluation practices.

• Influence long-term strategy for content quality and AI evaluation programs.

Required Qualifications:

• Bachelor's degree in computer science, Engineering, Data Science, Information Systems, AI, or related field.

• 2-4 years of experience in test automation, quality engineering, AI evaluation, or systems development.

• Strong experience with Python and automation tooling.

• Experience with Copilot, VSCode, and other AI tooling and infrastructure.

• Experience building testing frameworks, quality systems, or workflow automation solutions.

• Experience interpreting data and translating findings into strategic recommendations.

• Strong technical problem-solving and systems-thinking skills.

Top 3 Must-Have HARD Skills & years of experience for each:

• 2-4 years in content testing and automation work (including experience with Python and automation tooling).

• Demonstrated experience translating findings into recommendations, actions, next steps, and milestones - 2 years minimum.

• Demonstrated experience with AI tooling, preferable Copilot and VSCode, but other tooling experience is acceptable so long as transferable - 1 year (min); 2 years (preferred)

Preferred Qualifications:

• Experience evaluating generative AI systems or LLM-powered products.

• Experience designing human evaluation programs or human-in-the-loop workflows.

• Familiarity with content quality measurement and content operations.

• Experience working with machine learning, AI quality, or model evaluation frameworks.

• Understanding of Microsoft 365, Copilot, or productivity software ecosystems.

• Experience building internal tools, dashboards, or quality monitoring systems.

What's in it for me?

• Opportunity to work in a dynamic and innovative environment

• Engagement with industry-leading professionals across IT and Critical Environments sectors

• Contributing to the safety, security, and operational excellence of critical infrastructure

• Professional growth through diverse technical responsibilities and projects

• Supportive team culture that values inclusion and continuous improvement

Upon completion of waiting period, consultants are eligible for:

• Medical and Prescription Drug Plans

• Dental Plan

• Vision Plan

• Health Savings Account

• Health Flexible Spending Account

• Dependent Care Flexible Spending Account

• Supplemental Life Insurance

• Short Term and Long Term Disability Insurance

• Business Travel Insurance

• 401(k), Plus Match

• Weekly Pay

P.S: No third party or C2C. Candidate needs to be based within USA.

If this is a role that interests you and you'd like to learn more, click apply now and a recruiter will be in touch with you to discuss this great opportunity. We look forward to speaking with you!

About ManpowerGroup, Parent Company of: Manpower, Experis, Talent Solutions, and Jefferson Wells

ManpowerGroup® (NYSE: MAN), the leading global workforce solutions company, helps organizations transform in a fast-changing world of work by sourcing, assessing, developing, and managing the talent that enables them to win. We develop innovative solutions for hundreds of thousands of organizations every year, providing them with skilled talent while finding meaningful, sustainable employment for millions of people across a wide range of industries and skills. Our expert family of brands - Manpower, Experis, Talent Solutions, and Jefferson Wells - creates substantial value for candidates and clients across more than 75 countries and territories and has done so for over 70 years. We are recognized consistently for our diversity - as a best place to work for Women, Inclusion, Equality and Disability and in 2022 ManpowerGroup was named one of the World's Most Ethical Companies for the 13th year - all confirming our position as the brand of choice for in-demand talent.

ManpowerGroup is committed to providing equal employment opportunities in a professional, high quality work environment. It is the policy of ManpowerGroup and all of its subsidiaries to recruit, train, promote, transfer, pay and take all employment actions without regard to an employee's race, color, national origin, ancestry, sex, sexual orientation, gender identity, genetic information, religion, age, disability, protected veteran status, or any other basis protected by applicable law.