Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Application QA Specialist

$85k - $115k

Virtue AI

Job Title AI Application QA Specialist Job Description FortiOS Release QA team in Burnaby has over 90 staff and are responsible for ensuring Fortinet products meet the high standards we expect before release to the market. The AI Application QA Specialist is responsible for testing and validating AI-powered applications which support FortiOS verification. Job Responsibilities Design and execute test plans for AI features, including functional, regression, and edge-case testing across models and surrounding application logic. Perform data verification on training, validation, and production datasets to ensure data quality, integrity, and correct labeling. Review new AI feature requirements to better support FortiOS testing. Set up automated tests, monitor test failures and maintain test scripts. Identify, document, and track software defects throughout the testing lifecycle, ensuring timely resolution. Required Skills and Qualifications Open to both recent graduates and candidates with experience in testing AI applications or network-related products. Familiarity with AI-assisted workflows (LLMs, prompt engineering, evaluation pipelines) and common data tools (for example, SQL, Excel, or notebooks). Solid knowledge of QA processes, testing methodologies, and software development fundamentals. Practical knowledge of machine learning concepts and evaluation metrics, plus experience with at least one programming language such as Python. Familiarity with TCP/IP fundamentals and networking technologies, including routing, DNS, DHCP, VPN, and switching. Education Requirements Bachelor's degree in Computer Science, Math, Data Science, Engineering, or a related field (or equivalent experience). Salary and Benefits The Canada base salary range for this full-time position is expected to be between $85,000 - $115,000 annually. Wage ranges are based on various factors including the labour market, job type, and job level. Exact salary offers will be determined by factors such as the candidate's subject knowledge, skill level, qualifications, and experience. Fortinet strives to provide you and your family with a comprehensive benefits package. Benefits eligibility starts on your first day of hire and comprises of 100% company paid medical, dental, and vision coverage, including a Health Spending Account and a Personal Spending Account that gives you flexibility to spend where you need it the most. Our Employee & Family Assistance Plan (EFAP) offers you and your family access to various services like counseling, legal advice, mental health resources etc. We also provide critical illness, disability, and life insurance, as well as a Group Registered Retirement Savings Plan (RRSP) with a company match to help you save faster for retirement. We offer competitive Paid Time Off and flexible leave policies, including paid health days, to help you take care of yourself and your family members. All roles are eligible to participate in the Fortinet equity program. Bonus eligibility is reviewed at time of hire and annually at the Company's discretion. About Us Fortinet (NASDAQ: FTNT) secures the largest enterprise, service provider, and government organizations around the world. Fortinet empowers its customers with intelligent, seamless protection across the expanding attack surface and the power to take on ever-increasing performance requirements of the borderless network - today and into the future. Only the Fortinet Security Fabric architecture can deliver security without compromise to address the most critical security challenges, whether in networked, application, cloud or mobile environments. Fortinet ranks number one in the most security appliances shipped worldwide and more than 500,000 customers trust Fortinet to protect their businesses. We are committed to providing reasonable accommodations for all qualified individuals with disabilities. If you require assistance or accommodation due to a disability, please contact us at View email address on click.appcast.io. Fortinet is an equal opportunity employer. We value diversity in our company, and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age, military/veteran status or any other applicable legally protected characteristics in the location in which the candidate is applying. Virtue AI

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the AI Application QA Specialist in New York, NY vacancy
  •  .... Through modern vertical software, embedded payments, and AI-powered capabilities, we help businesses in fitness, recreation...  ...Service team in our Enterprise Fitness vertical as an Application Support Specialist, working remotely from Spain. The team provides 3rd level technical... 
    Suggested
    Local area
    Remote work
    Flexible hours

    Xplor Inc

    New York, NY
    5 days ago
  • Company Overview NWN is the leading AI-powered technology solutions provider for North America’s most innovative public and private...  ...more information, visit: NWN.ai. Responsibilities The Cash Application Specialist is responsible for the accurate and timely application,... 
    Suggested
    Work experience placement
    Local area
    Remote work
    Flexible hours

    Carousel Industries of North America Inc.

    New York, NY
    2 days ago
  • $30 - $36 per hour

     ...reviewed and diagnosed by radiologists. These specialists are the central hub of diagnostic...  ...has allowed us to build RadOS —a unified, AI-powered operating system powering the...  ...expectations in every interaction. As a Sirona Application Support Specialist , you will play a... 
    Suggested
    Hourly pay
    Full time
    Remote work
    Monday to Friday
    Flexible hours
    Shift work
    Night shift

    Sirona Medical

    New York, NY
    1 day ago
  •  ...Consulting is looking for a Systems Operations Specialist that would like to collaborate with one...  ...to the Grants Funding and Early Access applications. Skills & Experience Fluent English...  ...SQL. Experience with writing prompts in AI (Artificial Intelligence). Degree (... 
    Suggested
    Permanent employment
    Full time
    Contract work
    Remote work
    Home office
    Flexible hours

    Dentsu International Americas, LLC

    New York, NY
    3 days ago
  • $85 per hour

     ...$85 / hr · Location: Remote — US-eligible · Contractor QA & Source Inspection Specialist — AI Quality Review (Remote) is a remote review track for evaluating...  ...specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US. Compensation Hourly... 
    Suggested
    Remote job
    Hourly pay
    For contractors
    Work experience placement
    10 hours per week

    AuraOne

    New York, NY
    3 days ago
  • Obsidian is seeking expert Evaluators in Document/deck production QA, focusing on reviewing AI-generated documents for quality and accuracy. This role requires a minimum of 5 years of relevant experience and fluency in English.The position is remote and hires on an hourly... 
    Remote job
    Hourly pay
    Work at office

    Obsidian

    New York, NY
    1 day ago
  • $30 - $34 per hour

    Role Overview Help improve an AI-driven video generation product through rigorous manual testing, detailed issue reporting, and responsive...  ...platforms is preferred. Work Terms Remote contract engagement. Applicants must be located in the United States and available to work... 
    Hourly pay
    Contract work
    Remote work

    SaidGig

    New York, NY
    4 days ago
  • AuraOne is seeking a QA & Source Inspection specialist to evaluate AI outputs across QA workflows. This remote contractor role focuses on grading workflow correctness, policy adherence, and stakeholder fit, and documenting the right next step for model training. Review... 
    Remote job
    For contractors
    10 hours per week

    AuraOne

    New York, NY
    5 days ago
  • AuraOne in the United States is seeking a VA Eligibility Specialist for a Remote AI Review role on a contractor basis. You will evaluate AI outputs against VA workflows, apply policy guidelines, and document next steps for the modeling team.The ideal candidate demonstrates... 
    Remote job
    For contractors
    10 hours per week

    AuraOne

    New York, NY
    5 days ago
  •  ...Important Eligibility Notice Location: Applications are strictly restricted to individuals currently...  ...for a detail-oriented Data Labeling Specialist to review, annotate, and structure...  ...according to strict project guidelines. Review AI model outputs for factual accuracy,... 
    Remote job
    Full time
    Contract work
    Home office

    Quik Hire Staffing

    New York, NY
    3 days ago
  • AI Trainer Jobs is seeking a remote reviewer with hands-on experience in GTM/RevOps and sales operations. You will evaluate AI outputs...  ...flexible hours, US-eligible, and a focus on consistent rubric application across multi-page tasks. Prior calibration experience is a plus... 
    Remote job
    For contractors
    Flexible hours

    AuraOne

    New York, NY
    3 days ago
  • $85 per hour

    AI Trainer Jobs is seeking a remote Food Delivery App User Tester contractor to evaluate AI outputs across operations workflows. The role pays $85/hr and is remote with US eligibility. You will review, grade, and document actionable next steps to help training the modeling... 
    Remote job
    For contractors
    Flexible hours

    AuraOne

    New York, NY
    3 days ago
  • Obsidian is hiring expert Evaluators in Software, AI, IT, and data to review AI-generated work products for accuracy and quality. The role requires over 5 years of relevant experience and proficiency in both Microsoft Office and Google Workspace. This is a remote, hourly... 
    Remote job
    Hourly pay
    Work at office

    Obsidian

    New York, NY
    1 day ago
  • $30 - $50 per hour

    A tech company specializing in AI solutions is searching for a remote, full-time candidate to support AI/ML training workflows. In this role, you will perform high-accuracy data labeling, execute RLHF evaluation, and ensure training data quality. Candidates should have... 
    Remote job
    Hourly pay
    Full time

    REX

    New York, NY
    4 days ago
  •  ...В составе Muse Group — Hal Leonard, публикующий нотные издания и образовательные материалы по всему миру. Задачи включают работу по QA моделей и разметке данных в области нотной записи и аудио, а также помощь в улучшении моделей и создании качественных золотых наборов... 

    Muse Group

    New York, NY
    1 day ago
  • $85 per hour

    AuraOne is seeking a Medicare Enrollment Specialist - Remote AI Review to join as a contractor, enabling rapid evaluation of AI outputs across enrollment workflows. You will review against policies, grade tone, and escalation paths while noting gaps for improvement in... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne

    New York, NY
    1 day ago
  • AuraOne, Inc. seeks a remote Photographer reviewer to evaluate AI outputs across photography review workflows. You will grade craft, constraints, and tooling adherence, and document corrections for model training. The role is a CONTRACTOR, US-eligible, with an hourly rate... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    1 day ago
  • Supporting and enhancing R&D product and specification applications, the full-time IT Applications Specialist will partner with global business teams to translate...  ...product data specifications, R&D applications, and AI tools is preferred Familiarity with testing techniques... 
    Full time
    Work experience placement
    Remote work
    Flexible hours

    Virtual Vocations Inc

    New York, NY
    3 days ago
  • Invisible Technologies Inc. seeks Korean translators to shape the future of AI. You will assess AI-generated translations for grammar, fluency, culture, and context, working with Korean and English. As a contractor, you’ll translate in both directions, annotate errors,... 
    Remote job
    For contractors

    Invisible Technologies Inc. Defunct

    New York, NY
    2 days ago
  • Lifted is seeking Italian-speaking coding experts to support the evaluation and improvement of AI-generated audio responses. You will review live transcripts, assess language quality, technical accuracy, and JSON structure, and provide clear written justifications. The... 
    Flexible hours

    Lifted (Upwork)

    New York, NY
    1 day ago
  •  ...Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying...  ...P&L. Role Overview We are looking for experienced Video Data QA Specialists to support the OpenAI Egocentric project. The selected professionals... 
    Remote work
    Flexible hours
    Weekend work

    Turing

    New York, NY
    1 day ago
  • TSMG is seeking Quality control specialists to support an AI language project focused on improving speech recognition systems. You will review completed language annotation tasks and ensure AI-generated transcripts are natural and readable according to guidelines. You... 
    Flexible hours

    TSMG

    New York, NY
    5 days ago
  • TSMG is seeking Quality control specialists to support an AI language project focused on improving speech recognition systems. You will review completed language annotation tasks and ensure AI-generated transcripts are natural, readable, and aligned with project guidelines... 

    TSMG

    New York, NY
    5 days ago
  • SuperAnnotate AI, Inc is seeking a Punjabi Language Specialist (SME) for an hourly, remote contractor role to review AI-generated Punjabi content and create high-quality Punjabi reference content. You will evaluate outputs for accuracy, fluency, grammar, spelling, tone... 
    Remote job
    Hourly pay
    For contractors

    Brain Trust Inc

    New York, NY
    1 day ago
  •  ...individuals to join our team as Data Labeling Associate for Tamil speakers. This is not a traditional annotation role; you will evaluate AI outputs, flag gaps, and help improve system performance in real-world scenarios, where data quality, model evaluation, and human... 
    Full time

    Welocalize, Inc.

    New York, NY
    3 days ago
  • AuraOne is seeking a remote contractor to review AI outputs for Electrical Hardware Engineering Experts workflows. You will grade tone, policy adherence, and stakeholder fit against established playbooks and capture the right next steps for the modeling team. This role... 
    Remote job
    For contractors
    10 hours per week

    AuraOne

    New York, NY
    3 days ago
  • Web Build & QA Specialist Location: Remote Employment Type: Full-Time We are looking for a detail-oriented Web Team member to join a US digital...  ...experience. Ability to work scheduled US Central Time hours. Interview Process: AI Screening Client Interview Offer Stage Pavago
    Full time
    Remote work

    Pavago

    New York, NY
    2 days ago
  •  ...Data Labeling Associate. This 100% onsite role focuses on evaluating AI outputs, identifying gaps, and improving model behavior in real-...  ...and attention to detail will excel in this human-in-the-loop QA environment. Location-friendly sites include NYC, Seattle, and other... 
    Full time
    Remote work

    Welo Data

    New York, NY
    1 day ago
  • SaidGig is seeking a QA tester to help improve an AI-driven video generation product through thorough manual testing, issue reporting, and responsive client support. You will contribute valuable real-world input to train and improve AI systems, with prior AI experience... 
    Remote job

    SaidGig

    New York, NY
    4 days ago
  • $30 per hour

    Prolific is seeking Customer Support Representatives to join our Expert Network to train and evaluate AI models using real service expertise. You will complete a short test and, if selected, join Prolific as a participant and get paid for AI tasks. Expected pay up to $... 
    Remote job
    Hourly pay

    Prolific

    New York, NY
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Application QA Specialist. Be the first to apply!