Senior QA Engineer — ML/LLM End-to-End Testing & Eval
$160k - $300kSolstice
Solstice is seeking a dedicated QA lead to manage quality for AI-powered pharmaceutical marketing content. This senior engineering role demands expertise in building test infrastructure and conducting both automated and manual tests.
You will work on developing evaluation systems, ensuring product reliability, and implementing CI/CD practices. Other responsibilities include designing tests for non-deterministic systems and running exploratory QA.
Benefits include health insurance, equity opportunities, 401(k) with match, and competitive NYC compensation ranging from $160,000 to $300,000.
#J-18808-LjbffrVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior QA Engineer — ML/LLM End-to-End Testing & Eval in New York, NY vacancy
- Senior QA Engineer We are building the next generation of data-driven... ...standards, drive test automation, and ensure... ...regression, performance, and end-to-end test plans.... ...Experience testing AI/ML-powered applications, Generative... ...Large Language Model (LLM) outputs,...SeniorFull timeRemote work
$152.93k - $254.88k
...Quality Automation Engineer - USA (B) Country... ...rubrics; automated eval harnesses in CI;... ...methods (including LLM-as-a-judge) so quality... .... Design testing strategies for multi... ...relevant. Evaluate end-to-end agent... ...applications or AI/ML systems and quality...SuggestedLocal areaRemote workBMC Software, Inc.
New York, NY1 day ago- ...efficient large language model (LLM) applications turning real-... ...systems Optimization: From prompt engineering to fine-tuning and... ...Experimentation: Built-in A/B testing, smart routing, and fallback capabilities... ...Staff with expertise in front-end or design engineering. Why not...Senior
$90k - $155k
...This is a foundational QA position, and the person... ...will be hands-on manual testing, workflow validation, and... ...and sign-off Document end-to-end business... ...technical direction to the QA Engineer on the team Contribute... ...Experience testing AI or ML outputs is a strong plus...SeniorTemporary workImmediate startRemote workFlexible hoursShift workNight shiftAlteva RCM
New York, NY17 days ago$160k - $300k
...first dedicated QA lead to own quality... ..., with a solid end-to-end suite in Playwright... ...hands‑on manual testing that catches what... .... This is a senior, hands‑on engineering job. Most of your... ...‑deterministic, ML, or LLM‑based systems, or... ...Experience with eval and LLM‑...SuggestedPermanent employmentH1bVisa sponsorshipRelocation packageNight shiftSolstice
New York, NY1 day ago$160k - $300k
...Senior QA Lead (AI Systems) About the Company... ...This is a hands-on engineering role focused on building testing infrastructure, evaluation... ...for everything from LLM evaluations and... ...and Playwright Own end-to-end testing across... ...Familiarity with AI, ML, or LLM-based...Visa sponsorshipRelocation packageStorm3
New York, NY4 days ago- ...Job Title: Full Stack Software Engineer (Front-End Focus) Location: New York City, NY (Hybrid or... ...cutting-edge technologies, including AI/ML, they aim to improve patient outcomes,... ...practices. Participate in code reviews, testing, and debugging to ensure quality and...SeniorPermanent employmentFlexible hours
- Senior QA Engineer Join a forward-thinking technology team building AI-powered... ...practice, combining modern test automation with AI-driven testing... ...quality leadership: Own the end-to-end quality assurance... ...privacy. Hands-on experience with LLM evaluation and AI...SeniorContract workRemote work
- ...Employment Type: Contract Experience: 8+ Years About the Role We are looking for an experienced Senior QA Engineer (8+ years) to lead end-to-end regression testing across our enterprise MarTech ecosystem. In this role, you will own the validation of interconnected...SeniorFull timeContract work
Geysers International
New York, NY1 day ago - ...collaborative work environment. Job Summary As a Senior Creative Front‑End Engineer, you will build premium, highly animated, design‑driven web... ...work independently and write clean, maintainable, well‑tested code. Optimize animation‑heavy pages for performance (...SeniorRemote work
Truelogic Software LLC
New York, NY1 day ago - ...software, data infrastructure, and AI/ML with deep agricultural expertise and hands... ...from the ground up and looking for engineers who want meaningful ownership over product... ...The Role: We’re looking for a QA Engineer who tests data as rigorously as features - and keeps...SeniorFull time
NIBBANA
New York, NY2 days ago - ...guidance and subject matter expertise to engineers on testing and Quality Assurance (QA) methodologies and processes• Works... ...with Web Services testing and back-end service/API tools such as SoapUI ,... ...TechnologyExperience level: Mid-Senior LevelIndustry: Computer SoftwareSeniorWork experience placement
EROS Technologies
New York, NY5 days ago $184.9k - $250.2k
...exploration to mission-critical pipelines. The Engine Front End team is responsible for the query... ...that customers depend on every day. As a senior engineer on the team you will own... ...services end to end: shape designs, build, test, ship through our release pipelines, and...SeniorInternshipFlexible hoursAmazon
New York, NY3 days ago$184.96k - $246k
The Bank of New York Mellon seeks a Senior Vice President, Front-End Engineer for its New York, NY location.DUTIES: Consult with internal business groups... ...testable, efficient/ code. Analyze, design and develop tests and test-automation suites. Develop flowcharts, layouts...SeniorTemporary workRemote workFlexible hoursThe Bank of New York Mellon
New York, NY5 days ago$188k - $242k
...Senior Software Engineer, Front End - NY New York, New York Who We Are At Justworks, you’ll enjoy a welcoming and casual environment, great... ...technical solutions to common web UI/UX problems Coding, testing, and deploying modern web apps Help hire and develop...SeniorCasual workWork at officeLocal areaJustworks
New York, NY15 hours ago- ...We are looking for a Senior Quality Engineering Lead to help build an enterprise... ...This is not a role focused on testing individual applications. Instead... ...a scalable, reusable QA capability that serves the whole... ...across unit, integration, front-end and platform testing. A key...Senior
Oliver James
New York, NY2 days ago - ...Senior QA Engineer Dili builds software that helps construction and clean-energy companies manage... ...into deterministic, executable tests. What You'll Own Build and maintain... ...Build and improve integration, API, end-to-end, and regression test suites Establish...SeniorPermanent employmentApprenticeship
dili
New York, NY4 days ago $130k - $160k
...About the role We're looking for a Senior QA Engineer to join a strong, collaborative QA team... ...We want someone who is exceptional at testing: sharp enough to learn a complex product... ...process tooling. Experience testing AI/ML systems, NLP outputs, transcription accuracy...SeniorFull timeWork at officeHealthie Inc
New York, NY4 days ago- ...are hiring for the below, Role: Senior QA Engineer Location: NYC, NY Hybrid 3 days... ...focus on automation strategy, scalable test frameworks, and release quality. This role... ..., regression, integration, system, and end-to-end testing Design, build, and...Senior
SWITS DIGITAL Private Limited
New York, NY2 days ago - ...Senior QA Engineer Location - NYC Mode - Hybrid Key Responsibilities Lead end-to-end QA efforts across multiple projects and Agile teams Define and implement QA... ..., develop, and maintain scalable automated test frameworks Oversee functional, regression...Senior
Georgia IT Inc
New York, NY2 days ago $110k - $130k
...with pre-vetted, remote software engineers through its Intelligent Talent... ...~ We are looking for a Senior QA Engineer to lead quality engineering... ...automation strategy, scalable test frameworks, and release... ...regression, integration, system, and end-to-end testing Design,...SeniorRemote workApTask
New York, NY4 days ago$32 - $38 per hour
...has a client that is seeking a QA Engineer in New York, NY.... ...develop, and maintain automated test suites for AI platforms and services... ...functional, integration, and end-to-end testing frameworks* Collaborate... ...& Data* Generative AI* LLM Evaluation* Prompt Testing* Golden...KForce
New York, NY3 days ago- Somnio Software is seeking a Senior QA Engineer to ensure our products work well and feel right for... ...exploratory and user-centric testing rather than automation, with a focus on... ...and business rules to think like both an end user and an admin. This is a part-time,...SeniorRemote jobPart time
- ...client that is seeking a QA Engineer in New York, NY.... ...and maintain automated test suites for AI platforms... ...functional, integration, and end-to-end testing... ...* Generative AI * LLM Evaluation * Prompt Testing... ...certifications, experience, skills, seniority, location, performance,...Hourly payContract work
- As a Senior QA Engineer at Somnio Software, you’ll make sure our products work well and also feel... ...focus will be exploratory and user-centric testing rather than running test cases. Your... ...rules well enough to think like an end user and like an admin. Flag issues even...SeniorPart timeImmediate startRemote workFlexible hours
$190k - $220k
...powerful vector-based matching engines, and delivering AI-driven... ...engineering team, we’re looking for a senior technical leader who wants to... ...and mentor a team of front-end and full-stack engineers; set... ...notifications. Jest + React Testing Library for testing. Who We Looking...SeniorFull time$127k - $235k
...'re looking for a Senior Software Engineer - AI to join the CoCounsel... ...workflows, LLM orchestration, and... ...LLM-as-judge or eval-driven quality gates — serving real end users at scale. Technical... ...with AI/ML infrastructure including... ...opinions on testing, automation, and operational...SeniorFull timeWork at officeLocal areaWorldwideFlexible hoursThomson Reuters
New York, NY1 day ago- ...AI Agents Applied Research/Engineering Senior Associate in our The... ...conversations. You'll lead the end-to-end lifecycle of LLM-based agents: defining research... .... Strong foundation in ML, deep learning,... ...monitoring, versioning, A/B testing, and rollbacks. ~ Track...Senior
Testing Site - JPMC
New York, NY15 hours ago - ...of the future! Electric Power Engineers, Inc. (EPE) is a rapidly... ...for a motivated and talented Senior QA Engineer to join our growing software... ...build, and maintain automated test frameworks for our web... ...frameworks and methodologies for front-end development. Familiarity with...SeniorRemote workFlexible hours
$100k - $143k
...company built to last.Role SummaryWe are seeking an experienced QA Test Engineer/Lead to join our Service Technology QA team for our Customer... ...organization. You will collaborate with project stakeholders and end-to-end testers, ensure adherence to standards and guidelines,...SeniorLocal area3 days per weekNew York Life Insurance Company
New York, NY6 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior QA Engineer — ML/LLM End-to-End Testing & Eval. Be the first to apply!
Related searches
- sdet qa automation engineer New York, NY
- senior quality assurance engineer New York, NY
- qa automation engineer New York, NY
- senior automation test engineer New York, NY
- qa automation engineer remote New York, NY
- junior software test automation engineer New York, NY
- software quality engineer New York, NY
- junior qa automation engineer New York, NY
- senior software test automation engineer New York, NY
- qa engineer New York, NY