Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Evaluation Engineer

Deposco

AI Evaluation EngineerDeposco is seeking an experienced AI Evaluation Engineer to join our innovative quality assurance team. This role focuses on ensuring the accuracy, reliability, and performance of AI-driven applications, models, and automation workflows. The ideal candidate will be highly analytical, detail-oriented, and passionate about shaping the future of AI testing practices. You will work closely with product managers, developers, and data scientists to validate AI features, services, and integrations that power our products. Reporting to the Vice President of Quality, this role will be instrumental in shaping the team's future and supporting Deposco's growth.What You'll Do:Engage in requirements and project meetings to fully understand goals and deliverablesReview and analyze project requirements to pinpoint gaps and ensure they are suitable for creating test casesDesign and maintain automated tests using Java or Python for our omnichannel solutionsEmploy automated testing tools for RESTful services connected to marketplace integrationsIdentify marketplace-specific use cases, performing manual tests when necessary, while progressing toward automationDevelop automated acceptance and unit tests within an agile environmentUtilize Jenkins and other CI tools for automation of development, build, deployment, and testing workflowsCollaborate closely with product owners as part of a Scrum team to clarify requirementsExamine test results from a technical perspective and provide detailed feedback to the development teamAssist in troubleshooting production issues through coordinated testing and support effortsCreate essential QA documentation, including test plans, scripts, strategies, and requirement traceability matricesNitty Gritty (What You'll Need):Four-year degree in Computer Science/Engineering preferred2+ years experience in software quality testing2+ years of Agile Experience2 years Python Experience preferredExperience in integration platforms preferredThorough understanding of the Software Development Life Cycle (SDLC), encompassing requirements gathering, design, and testing, with hands-on experience in modern testing methodologies and toolsProficient in working with JSON and XML message formatsFamiliarity with Atlassian products such as JIRA and ConfluenceExperience working in agile environments and using continuous integration tools such as GitHubSolid knowledge of relational databases, with the ability to write MySQL queries to validate test resultsPractical exposure to RESTful automation toolsCompetence in at least one object-oriented programming language; scripting skills are an added advantageCapable of working independently or collaboratively in a fast-paced, dynamic environment under tight deadlinesStrong analytical and organizational abilities, with aptitude for handling multiple projects simultaneouslyDemonstrates initiative, accountability, and ownership of projects and quality assurance activitiesPerksBe part of an energetic team that is serious about delivering software to customers and cultivating our culture. We work awfully hard and are having a blast doing it. This team believes that what we are doing is a game-changer in the industryCoffee bar with cold brew on tap and a full time baristaStanding desk (if you're into that sort of thing)Corporate events including food trucks, Top Golf and other shenanigansCorporate office with co-working spaces, ping-pong tables, corn-hole boards, and Avalon around the cornerBenefits package including health care, paid time off, life insurance, 401k plan (with match), and gym reimbursementBe part of a team recognized 6 years running as a USA Today Workplace

Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the AI Evaluation Engineer in Alpharetta, GA vacancy
  •  ...The AI Engineering team at Global Payment Holding Company is building and improving ML/DL and Generative AI systems to support automation...  ...management. Stay current with AI research and industry trends, evaluating emerging tools and techniques for enterprise adoption.... 
    Suggested
    H1b

    Global Payment Holding Company

    Alpharetta, GA
    21 hours ago
  • $137k - $263k

     ...#VTeamLife.What you'll be doing…As a Distinguished Forward Deployed AI Engineer within Verizon’s Transformation Command Center (TCC), you...  ..., and serverless compute across AWS, GCP, or Azure).Evaluation Harness Design: Architect offline and online evaluation frameworks... 
    Suggested
    Full time
    Temporary work
    Part time
    Work experience placement
    Work at office
    Work from home
    Shift work
    3 days per week

    Verizon

    Alpharetta, GA
    1 day ago
  •  ...26-09-09Travel: Up to 10%Worker Category: HybridThe Senior AI Software Engineer to design, develop, and deploy AI-powered solutions that enhance...  ...Generation (RAG), AI agents, prompt engineering, model evaluation, and AI governance principles. Demonstrated ability to... 
    Suggested
    Full time
    Work at office
    Remote work
    Worldwide
    Monday to Thursday

    Rheem Manufacturing Company

    Roswell, GA
    16 hours ago
  • $101k - $194k

     ...Want in? Join the #VTeamLife.What you’ll be doing...As a Senior AI Engineer within our team of propensity modelers, you will be...  ...Employment OpportunityVerizon is an equal opportunity employer. We evaluate qualified applicants without regard to veteran status, disability... 
    Suggested
    Full time
    Temporary work
    Part time
    Work experience placement
    Work at office
    Work from home
    Shift work
    3 days per week

    Verizon

    Alpharetta, GA
    1 day ago
  •  ...Description Job Description We are looking for a Senior AI Engineer We are seeking a hands-on Senior AI Engineer to help design...  ...and agent behavior. Define and implement approaches for LLM evaluation, AI testing, debugging, and performance optimization.... 
    Suggested

    Advanced Tech Placement

    Alpharetta, GA
    23 days ago
  • $155.4k - $261.1k

     .... No relocation is offered.We’re looking for a Principal AI-Native Software Engineer who can turn complex business needs into intelligent, production...  ...rate limits, error handling, and access controls.Develop evaluation strategies for AI and agentic systems, including success... 
    Full time
    Temporary work
    Local area
    Relocation

    AT&T

    Alpharetta, GA
    16 hours ago
  • $196.1k - $326.9k

     ...stakeholder groups.Key Responsibilities: Proven leadership of AI/ML engineering orgs through a management layer — direct experience managing...  ..., experience and skills, equity, regular job market evaluations, and geographical markets. The pay range shown below is aligned... 
    Full time

    McKesson

    Alpharetta, GA
    2 days ago
  • $155.3k - $258.8k

     ...want to hear from you.About the TeamOur AI Enablement team drives enterprise-wide adoption...  ...of generative AI, spanning platform engineering, governance, and hands-on delivery of AI...  ...(RAG), tool/connector integration, and evaluation/governance tooling, in a highly... 
    Full time

    McKesson

    Alpharetta, GA
    1 day ago
  • $109k - $182.4k

     ...a difference at Fiserv.Job TitleAI - Cyber Security Engineer - IIAbout your role:As an Advisor II, AI - Software Engineering, you will design, build, and...  ...automation, monitoring, observability, performance evaluation, and lifecycle management for artificial intelligence... 
    Full time
    Temporary work
    H1b
    Work at office
    Monday to Friday

    Fiserv

    Alpharetta, GA
    1 day ago
  • $131k - $218.3k

     ....We are seeking a highly skilled and experienced Senior AI Full Stack Software Engineer to join our innovative team at CMM. This role involves designing...  ...(e.g., clinical notes, imaging, lab results). Model Evaluation and Compliance: Apply rigorous evaluation metrics (e.g.,... 
    Full time

    McKesson

    Alpharetta, GA
    3 days ago
  •  ...from you.We are seeking a highly skilled and motivated AI Full Stack Software Engineer to join our innovative team at CMM. In this role, you will...  ..., or equivalent experienceCritical SkillsAI System Evaluation & Optimization: Proficient in sensitivity metrics (precision... 
    Full time

    McKesson

    Alpharetta, GA
    1 day ago
  • $90k - $110k

     ...develop, and operationalize secure agentic AI capabilities in risk & security...  ...oversharing, and unauthorized accessEstablish evaluation practices for agent accuracy, hallucination...  ...leading enterprise-scale security engineering initiatives5+ years of progressive experience... 
    Full time

    Axis Capital

    Alpharetta, GA
    2 days ago
  • Principal Agentic AI Security EngineerThis role has been designed as 'Hybrid' with a...  ...with HPE.Job Description:The Agentic AI Engineer is responsible for designing, building, and...  ...asynchronous and distributed agent execution.Design evaluation and testing capabilities for agent... 
    Full time
    Work experience placement
    Work at office
    Local area
    Immediate start
    2 days per week

    Hewlett Packard Enterprise

    Alpharetta, GA
    3 days ago
  • $120.5k - $231k

     ..., and execution of cutting-edge agentic AI/ML solutions within the Verizon Consumer...  ...standards of data privacy, compliance, and engineering rigor.Partnering with Customer Service...  ....Experience developing automated AI evaluations and regression tests, including pytest,... 
    Full time
    Temporary work
    Part time
    Work experience placement
    Work at office
    Work from home
    Shift work
    3 days per week

    Verizon

    Alpharetta, GA
    1 day ago
  • $126k - $242k

     ...#VTeamLife.Why Verizon? Most enterprise AI leaders are still helping teams build human...  ...At Verizon, you’ll define the technical engineering strategy and architectural blueprint for...  ...architect the systems that validate, observe, evaluate, and optimize our autonomous voice and... 
    Full time
    Temporary work
    Part time
    Work experience placement
    Work at office
    Work from home
    Shift work
    3 days per week

    Verizon

    Alpharetta, GA
    4 days ago
  • $68 per hour

     ...US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other Job Title: AI Engineer Location: Alpharetta, GA Duration: 14 Months + Extension Bill Rate: $68/hour (C2C) Job Type: W-2, 1099, C2C Client: To Be Discussed Later Work Authorization:... 
    H1b
    Local area

    Valiant Technologies LLC

    Alpharetta, GA
    3 days ago
  • $92.6k - $173.1k

     ...the design, development, and deployment of advanced generative AI models, including large language models (LLMs), driving innovation...  ...data ~ Experience with generative LLM fine-tuning and prompt engineering. ~ Proficiency with data wrangling, visualization, and... 
    Summer holiday
    Local area
    Flexible hours

    Ernst & Young

    Alpharetta, GA
    a month ago
  •  ...Role- Senior AI Engineer Location- Alpharetta, GA Fulltime Key Responsibilities Design, build, and deploy Python-based AI/LLM applications using FastAPI/Flask. Develop REST APIs and scalable enterprise solutions. Build AI-... 
    Full time

    Futran Tech Solutions Pvt. Ltd.

    Alpharetta, GA
    21 hours ago
  •  ...people-first culture focused on work-life balance Location Alpharetta, GA Position Overview The Senior Data & AI Engineer is responsible for designing, building, and scaling Meadows & Ohly's enterprise data platform and AI capabilities. This hands-on... 
    Work at office

    Meadows and Ohly , LLC

    Alpharetta, GA
    20 hours ago
  •  ...Model SDKs used by data scientists and ML engineers • Develop SDKs that support seamless...  ...experimentation, validation, and deployment across the AI/ML lifecycle • Apply deep Python...  ...SDKs that simplify AI/ML model training, evaluation, and deployment workflows • Build SDKs... 
    Contract work

    Purple Drive

    Alpharetta, GA
    2 days ago
  •  ...role: We are seeking a highly technical and innovative AI Software Engineer in Test (AI SEIT) to help modernize quality engineering through...  ...and business rule compliance. Create automated evaluation datasets prompt test suites golden answer sets and scoring... 
    Full time
    Local area
    Shift work
    3 days per week

    GreenSky Administrative Services

    Alpharetta, GA
    5 days ago
  •  ...The Senior AI Software Engineer to design, develop, and deploy AI-powered solutions that enhance how Rheem builds and operates digital products...  ...cloud-based applications that leverage AI technologies. Evaluate model performance, prompt effectiveness, accuracy, and... 
    Full time
    Work at office
    Remote work
    Monday to Thursday

    hdde

    Roswell, GA
    15 days ago
  • Job Description Job Description Required Skills " 7 years of experience in software development. " Strong experience with C#, ASP.NET MVC, ASP.NET Core, .NET Framework/.NET Core. " Expertise in Angular 10 , TypeScript, JavaScript, HTML5, CSS3. " Experience in...

    Diverse Lynx

    Alpharetta, GA
    2 days ago
  •  ...Job Description Job Description AI /ML Engineer Alpharetta, GA Contract to Hire Seeking an AI Engineer with strong ML/DL, Generative AI, and MLOps experience to build a scalable AI System for our client. Must have Python, TensorFlow/PyTorch, cloud experience... 
    Contract work

    Universal Business Solutions

    Alpharetta, GA
    23 days ago
  •  ...Proven experience (2+ years) supporting Microsoft 365 technologies, including hands-on experience with Copilot Premium or equivalent AI-driven platforms.Strong understanding of how Copilot integrates with SharePoint Online, OneDrive, Exchange, Microsoft Search, and related... 
    Full time

    Axis Capital

    Alpharetta, GA
    3 days ago
  •  ...and deploy enterprise-grade Generative AI and Machine Learning solutions using Python...  ..., multi-agent orchestration, and LLM evaluation frameworks. Architect and productionize...  ...ML/AI development lifecycle from feature engineering and model validation to MLOps and production... 
    Full time

    Pyramid Consulting

    Alpharetta, GA
    8 days ago
  • $24 - $31 per hour

     ...science, machine learning, and generative AI to real business challenges across...  ...Write SQL and Python code to support data engineering, analytics, and machine learning projects...  ..., or PyTorch. Understanding of model evaluation, validation, feature engineering, and data... 
    Hourly pay
    Internship
    Summer internship
    Local area

    Siemens

    Alpharetta, GA
    4 days ago
  • $126k - $210k

     ...McKesson Pharmacy Systems R&D Senior Software Engineer McKesson is an impact-driven, Fortune...  ..., Spring Boot, SQL, Azure, Docker, and AI-assisted engineering tools. Limited...  ...experience and skills, equity, regular job market evaluations, and geographical markets. The pay range... 
    Full time
    H1b
    Work at office
    Remote work
    Work from home
    2 days per week

    McKesson

    Alpharetta, GA
    4 days ago
  •  ...technical, systems-oriented, and operationally sophisticated  AI Workflow Engineer / Automation Architect  to help design and implement the...  ...system logic, integration maps, and operational standards. Evaluate emerging automation technologies, orchestration frameworks,... 
    Full time
    Temporary work
    Work at office
    Shift work
    Afternoon shift

    Modern Family Law

    Alpharetta, GA
    1 day ago
  •  ...a difference at Fiserv.Job TitleSenior AI Data ScientistAbout your role:The Senior...  ...optimization, partnering with product, engineering, architecture, and business teams to move...  ...optimization.Create high-quality training, evaluation, and synthetic datasets while... 
    Full time
    Temporary work
    H1b
    Work at office
    Monday to Friday

    Fiserv

    Alpharetta, GA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Evaluation Engineer. Be the first to apply!