Application Engineer (3-6 Years)
$52 per hourVeterans Sourcing Group
Job Description
Job Description
AI Engineer (Generative AI & Agent Quality)
Pay Rate: $52.00 Per Hour
Duration: Contract through 12/31/2026 with possible extension
Location: Remote
Business Hours: 8:00 AM – 5:00 PM local time, with some flexibility
- Two rounds of Zoom interviews (webcam required)
There is potential for both extension and conversion . Candidates must reside near an Osaic hub location to be considered for conversion opportunities.
Position SummaryWe are seeking an AI Engineer to help design, test, optimize, and support AI-powered agent experiences across customer and internal business workflows. This role combines prompt engineering, agent orchestration, quality assurance, evaluation framework development, and AI observability.
The ideal candidate is curious, analytical, and hands-on, with experience working with Large Language Models (LLMs), prompt design, testing methodologies, and modern software development practices. You will partner closely with product managers, software engineers, architects, and business stakeholders to improve agent behavior, reliability, accuracy, and user experience.
Key ResponsibilitiesPrompt Engineering & Agent Design
- Design, develop, and iterate on AI agent logic for business use cases.
- Create, test, and optimize prompts, system instructions, tools, and workflows to improve agent performance.
- Conduct prompt experiments and A/B testing to measure effectiveness.
- Analyze agent responses and refine prompting strategies for accuracy, consistency, groundedness, and user satisfaction.
- Document prompt patterns, reasoning frameworks, reusable templates, and agent design decisions.
- Collaborate with engineering teams to deploy prompt and agent updates through established SDLC processes.
AI Quality Assurance & Testing
- Design and execute comprehensive test plans covering AI agents, web applications, APIs, and backend services.
- Develop evaluation frameworks and scoring methodologies for AI output quality.
- Create test cases covering accuracy, hallucination detection, safety, tone/style adherence, instruction following, context retention, and response consistency.
- Perform regression testing for prompt and model changes.
- Track, prioritize, validate, and retest defects using Azure DevOps.
AI Evaluation & Observability
- Utilize Arize or similar observability platforms to monitor model and agent performance.
- Analyze user interactions and evaluation results to identify trends and improvement opportunities.
- Develop quality metrics and dashboards for accuracy, relevance, latency, user satisfaction, defect rates, and prompt performance.
- Investigate production issues and recommend prompt, workflow, or architectural improvements.
Software Engineering & Integration
- Support integration of AI capabilities into enterprise applications and workflows.
- Collaborate with developers to validate API integrations, retrieval pipelines, and agent workflows.
- Assist in CI/CD deployment and release validation activities using Azure DevOps.
- Create and maintain automated testing assets where appropriate.
Documentation & Collaboration
- Maintain documentation for prompts, evaluation methodologies, testing procedures, and agent configurations.
- Participate in sprint planning, backlog refinement, and Agile ceremonies.
- Communicate findings and recommendations to technical and non-technical stakeholders.
- Share best practices and lessons learned across the team.
- Bachelor's degree in Computer Science, Engineering, Information Systems, or related field required .
- Minimum 4–5 years of relevant experience in software engineering, QA engineering, automation testing, business systems analysis, or related technical roles.
- 1 years of hands-on experience with Generative AI, LLMs, prompt engineering, or conversational AI solutions.
- Experience working with Azure DevOps or similar Agile delivery platforms.
- Experience designing and executing structured test plans.
- Strong analytical and troubleshooting skills.
- Experience working with APIs and JSON data structures.
- Excellent written and verbal communication skills.
- AI Agents
- Strong engineering and software development practices
- Azure experience
- Experience with Arize AI or comparable AI observability tools.
- Experience with Azure OpenAI, OpenAI, Anthropic, Gemini, or other LLM platforms.
- Knowledge of Retrieval-Augmented Generation (RAG) architectures.
- Experience evaluating AI systems using human and automated scoring methods.
- Familiarity with vector databases and semantic search.
- Experience creating automated test frameworks.
- Understanding of responsible AI principles and AI governance.
- Experience with Python, SQL, JavaScript, or C#.
- Knowledge of prompt versioning and experimentation practices.
AI & GenAI: Prompt Engineering, Agent Design, AI Evaluation Frameworks, RAG Concepts, LLM Testing, AI Quality Assurance, AI Observability, Prompt Optimization
Tools: Arize AI, Azure DevOps, Azure OpenAI, Git, Postman, Jira (optional), Power BI (optional)
Development: Python, SQL, REST APIs, JSON, Test Automation Frameworks
$140k - $150k
...Application Security Engineer Vestwell is the financial technology company powering the new savings economy. The New York City-based fintech platform... .... Requirements The Necessities ~2-4 years of professional experience in application security, product...SuggestedFull timeWork at office- ...Guidance & Assistance, Inc., (SGA), is searching for an AI Application Engineer for a contract assignment with one of our premier Infrastructure... ...Engineering, Information Systems, or related field. ~5-10 years of experience in software engineering, QA engineering,...SuggestedContract workRemote work
$155k - $248.1k
...market.The Teradyne Defense and Aerospace Field Applications team is looking for an energetic, driven Field Application Engineer who will work collaboratively with the team... ...fields OR equivalent experience.Minimum of 5 years of experience of test engineering or similar...SuggestedPermanent employmentRemote workWorldwideRelocationFlexible hours- ..., developing, and maintaining banking applications and services using .NET technologies.... ...owners, business analysts, and other engineers to translate banking requirements into... ...rails (ACH, Wires, RTP, FedNow...etc) ~6+ years of C# ~3 years messaging experience (MSMQ, RabbitMQ...SuggestedFull timeRemote work
- ...Enterprise, Song, Supply Chain and Engineering, and Talent, with advanced... ...decision logic where applicable.Contribute to robot training... ...Here’s what you need: Minimum 3 years of relevant experience in robotics... ...degree, must have a minimum of 6 years’ work experience)Bonus...SuggestedWork experience placementLive inWork at officeLocal area
- ...Manager, Forward Deployed Engineer (FDE) who helps clients... ...solution architecture.3. Data Cloud & Enterprise... ...What You Need:Minimum of 4 years of experience of... ...Degree, must have minimum 6 years’ work experience)... ...discriminate against employees or applicants because they have...Work experience placementLive inWork at officeLocal area
- ...Operations - Senior Salesforce Developer (3–6 Years) @ BugendaiTech -A Summit Partner... ...your clock. What You will Do Engineer Excellence: Develop high-performing LWC... ...Eligibility Requirements Location: Applicants must be currently residing in the United...Work visa
$110k - $130k
....What youll doSignificant technical contributor to market and application initiatives. Direct and deploy cross functional team to execute... ...Along with Product Development and Partner Applications Engineer, the role will require management and maintenance of laboratory...Local areaFlexible hours$142.7k - $158.3k
...Bachelor's degree in Software Engineering, or related Science, Technology... ...Mathematics field, plus a minimum of 8 years of relevant experience; or Master's degree, plus 6 years relevant experience.... ...time based on requirements. Applicants selected will be subject to a U...Full timeFlexible hours- ...this role, you will be expected to develop and maintain web applications across multiple platforms and user experiences (websites, apps... ...priorities at once WEB DEVELOPER KNOWLEDGE AND SKILLS ~3-6 years' experience in web programming Proven experience with...Temporary workWork at officeFlexible hours
- ...Job Description Lead Test Engineer Employment Type: Contract Labor... ...Mathematics field, plus a minimum of 8 years of relevant work experience; or... ...'s degree, plus a minimum of 6 years of relevant experience.... ...test scripts using Python 3.9+ following GDMS coding standards...Contract workWork experience placement
$100.22k - $111.18k
...Embedded Software Engineer Requires a Bachelor's degree in Software... ...Mathematics field. Also requires 2+ years of job-related experience, or a Master's degree and 6 months of job-related... ...amount of time from date of hire. Applicants selected will be subject to a U...Second jobFlexible hours$100.22k - $111.18k
...Bachelor’s degree in Software Engineering, or a related Science,... .... Also requires 2+ years of job-related... ...a Master's degree and 6 months of job-related... ...time from date of hire. Applicants selected will be subject... ...national assets. Preferred: 3+ years of experienceWe...Second jobFlexible hours$124.4k - $138k
...Bachelor's degree in Electrical Engineering, or a related Science,... ...Mathematics field, plus a minimum of 5 years of relevant experience; or... ...Master's degree plus a minimum of 3 years of relevant experience.... ...required at time of hire. Applicant must be able to acquire clearance...Flexible hours- ...Enterprise, Song, Supply Chain and Engineering, and Talent, with advanced... ...’s What You Need:Minimum 8+ years of pre-sales, solution consulting... ...’s Degree, must have minimum 6 years work experience)Bonus... ...discriminate against employees or applicants because they have inquired...Work experience placementLive inWork at officeLocal area
- ...transformation expertise.The LearnVantage Engineering team builds and operates the... ...and operate containerized applications in cloud environments.Build... ...'s What You NeedMinimum of 3 years of professional software... ...'s Degree, must have minimum 6 years work experience.Professional...Work experience placementLive inWork at officeLocal area
$120k - $130k
...Job Profile: Applications Developer 4 Job Family:... ...'s degree and seven (7) years of experience appropriate... ...an experienced software engineer who excels at creating and... ...development experience with Vue 3 or a comparable modern... ...format, for example 6/04 to 8/14, along with...Full timeContract workPart timeInternshipWork at officeLocal areaRemote workRelocation packageFlexible hours- ...match Talent Skills: Senior backend engineer: ~8+ years of overall IT experience ~4+ years of... ...~4+ years of overall IT experience ~3+ years of experience implementing REST... ...~2+ years of experience with Angular 6+ Mid-level full stack engineer:...Contract work
- ...opportunities, we’ll reach out directly when formal applications become available. About Software Engineering Roles at Danaher Are you passionate about... ...field Preference for students who have completed 3+ years of coursework by the internship start date Experience...Remote jobInternship
- ...AABG leverages cutting-edge engineering, high-performance... ...deploy custom cloud-native applications and services. Our teams help... ...or equivalent (minimum 12 years) work experience. (If... ...Degree, must have minimum 6 years work experience)Minimum of 3 years of full-stack engineering...Work experience placementLive inWork at officeLocal area
- ...confidence.You AreA front-end engineer passionate about creating... ...customize, and integrate front-end applications for Agentic AI-as-a-Service... ...'s what you needMinimum of 3 years of experience as a front-end,... ...’s Degree, must have minimum 6 years work experience)Bonus...Work experience placementLive inWork at officeLocal area
- ...to make real industrial applications more efficient, safer,... ...collaboration with experts in engineering, AI, and strategy. In... ...delivery teams (3-10 people) across PoC,... ...You Need:Minimum of 3 years of relevant experience... ...Degree, must have minimum 6 years work experience)...Work experience placementLive inWork at officeLocal area
$112.92k - $125.28k
...Bachelor's degree in Software Engineering, or related Science, Technology... ...Mathematics field, plus a minimum of 5 years of relevant experience; or Master's degree, plus 3 years relevant experience.... ...of time from date of hire. Applicants selected will be subject to a U...Flexible hours- ...Dynamics Mission Systems (GDMS) engineers a diverse portfolio of high... ...Mathematics field, plus a minimum of 5 years of relevant experience; or... ...'s degree plus a minimum of 3 years of relevant experience.... ...required at time of hire. Applicant must be able to acquire clearance...Flexible hours
$118.52k - $131.48k
...Bachelor’s degree in Electrical Engineering, or a related Science,... ...Mathematics field. Also requires 5+ years of job-related experience, or a Master's degree plus 3 years of job-related experience... ...amount of time from date of hire. Applicants selected will be subject to a U...Immediate startFlexible hours- ...and client requirementsWhat you Need:Minimum 3 years of SAP supply chain consulting focused on SAP... ...work experience (If an, Associate’s Degree with 6 years of work experience)Bonus Points If:You are a SAP Certified Application Associate — SAP EWM (C_EWM_2020 or equivalent...Work experience placementLive inWork at officeLocal area
$135k - $140k
...seeking a Senior Digital Design Engineer in Scottsdale, AZ. The... ...Engineering, or equivalent* 12 plus years of circuit design and a wide range... ...of circuit card design and 3 years of FPGA design... ...with FPGA VHDL design for space applications, SystemVerilog and advanced testbench...- We AreIn the last 30 years, no technology has promised... ...a Snowflake Advanced AI Engineer, you will design, build,... ...simulations along with their applications to solve complex... ...Cortex SearchMinimum of 3 years of experience as... ...Degree, must have minimum 6 years work experience)Professional...Work experience placementLive inWork at officeLocal areaShift work
- ...programs through your combined SAP application and functional process... ...requirementsThe Work: Minimum of 5 years SAP functional and technical experience... ...intercompany movements.Minimum 3 years of experience in SAP... ..., must have equivalent minimum 6-year work...Work experience placementLive inWork at officeLocal area
- ...and client requirementsWhat you Need:Minimum 3 years of SAP supply chain consulting focused on SAP... ...work experience (If an, Associate’s Degree with 6 years of work experience)Bonus Points If:You are a SAP Certified Application Associate — SAP EWM (C_EWM_2020 or equivalent...Work experience placementLive inWork at officeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Application Engineer (3-6 Years). Be the first to apply!
- senior application support engineer Scottsdale, AZ
- application performance engineer Scottsdale, AZ
- senior application security engineer Scottsdale, AZ
- app developer Scottsdale, AZ
- software applications developer Scottsdale, AZ
- oracle apps technical consultant Scottsdale, AZ
- director enterprise applications Scottsdale, AZ
- vice president of application development Scottsdale, AZ
- application security lead Scottsdale, AZ
- application development Scottsdale, AZ





