Founding AI Quality & Release Engineer
$170k - $200kWintermeyer Ventures
Job Description
Job Description
About the Role
This is the first dedicated quality-infrastructure hire at a well-backed AI/data startup, and it is a software engineering role through and through. You will own the full verification and release system, from agentic test harnesses and LLM graders to CI pipelines and deployment gates, making every green build trustworthy and every release uneventful.
What You'll Do- Design, build, and maintain agentic test harnesses and LLM-based evaluation systems that run and grade tests automatically.
- Own regression coverage and release gates, ensuring unsafe builds are stopped with evidence rather than intuition.
- Develop and improve CI pipelines, artifact publication workflows, and container build automation using GitHub Actions.
- Maintain reliable demo and staging environments that faithfully represent production.
- Drive down bug-escape rates by closing gaps in the automated signal, not by adding manual test runs.
- Participate in business-hours incident escalations and make incidents rarer and easier to diagnose over time.
- 7 to 10 or more years of overall software or quality engineering experience.
- At least 1 year of hands-on experience building, testing, or evaluating agentic AI systems and LLM-based applications, applied within the last year.
- Deep proficiency with GitHub Actions and GitHub-based CI/CD, including container builds and pipeline automation.
- Strong command of Python and TypeScript; comfort with Docker, Kubernetes, and Helm.
- Familiarity with at least one major cloud or data platform such as AWS, Snowflake, Databricks, or Azure.
- Background in early-stage startups or in modernizing legacy QA and DevOps practices inside larger organizations.
- Demonstrated ability to take full ownership of systems and processes, make independent decisions, and execute without close guidance.
- Fully authorized to work in the US or Canada without visa sponsorship now or in the future.
Base salary range: $170,000 to $200,000 USD annually. Visa sponsorship is not available.
LocationFully remote, open to candidates based anywhere in the United States or Canada. The company's primary hub is New York, NY.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Founding AI Quality & Release Engineer in New York, NY vacancy
$170k - $200k
...Description About the Role This is the first dedicated quality-infrastructure hire at a well-backed AI/data startup, and it is a software engineering role through and through. You will own the full verification and release system, from agentic test harnesses and LLM graders...SuggestedRemote workWintermeyer Ventures
New York, NY1 day ago- ...looking for an experienced Senior QA Engineer to own quality for our AI voice virtual agent. This role is... ...into the CI/CD pipeline to serve as release gates. Define and track core quality... ...have. The defects that matter here are found by someone who imagines the caller...SuggestedRemote work
- ...thinking bigger - and moving faster. We're a family-founded company on a mission to create the world's first AI-powered Personal & Entrepreneurial Resource... ...cases.Work with cross-functional teams to define quality standards and best practices.Stay up to date with...SuggestedWorldwide
$125.92k - $155.29k
ABOUT THE ROLEAs a Senior Product Quality Engineer at Peloton, you will be responsible for ensuring... ...software, and exclusive content. Founded in 2012 and headquartered in New York City... ...Peloton, we embrace technology, including AI, to enhance productivity and accelerate...SuggestedLocal area- ...Release Train Engineer Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where... ..., all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry...SuggestedPermanent employmentFull timeContract workLocal area
Capgemini
New York, NY5 days ago $147.5k - $211k
Location Designation: Hybrid - 3 days per week Release Train Engineer (RTE) - Agentic AI Web Application Role OverviewWe are seeking an experienced Release... ...and readiness criteria for AI capabilities, including quality, accuracy, latency, reliability, safety, and cost.·...Local area3 days per week- ...Release Train Engineer Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where... ..., all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry...Full timeLocal area
Capgemini
New York, NY3 days ago $170k - $190k
...portfolios, value streams and Agile Release Trains. We foster a... ...delivery while embedding innovative AI-enabled practices and... ...sessions to improve predictability, quality, transparency, and execution alignment... ...significant Release Train Engineer experience across product...Live inFlexible hoursS&P Global
New York, NY4 days ago$150k - $350k
A technology startup is seeking a Founding Engineer to lay the technical groundwork for their AI-powered platform. The role involves developing end-to-end features, with a focus on Python and full-stack experience. Ideal candidates will thrive in ambiguity and be able...Remote job- iFindTech Ltd. is seeking a Founding Engineer to join an early-stage team and build the core platform from the ground up. You will shape architecture... ...strategy, with ownership of systems that push boundaries in AI-driven products and large-scale operations. #J-18808-Ljbffr...
- Scend is hiring a Founding Engineer to join our NYC-based team and help shape the product and architecture from the ground up. You will work directly with founders and have outsized impact on system design and engineering culture, with a preference for candidates interested...Remote job
$100k - $120k
...companies and law firms. Since its founding in 2009, Burford has worked... ...and technically capable QA Engineer to support the quality of complex, data-driven... ...role will also use modern AI-assisted development tools... ...quality, reliability and release confidence of its data-driven...Full timeWork at officeShift work- ...- and moving faster. We're a family-founded company on a mission to create the world's first AI-powered Personal & Entrepreneurial Resource... ...lives along the way.The RoleAs a QA Engineer at, you will play a key role in ensuring the quality, reliability, and performance of our...Work at officeWorldwideRelocation
- Success Matcher Recruitment is seeking a Founding Engineer to architect the core systems of an AI agent platform for e‑commerce and fintech. This role lets you own the technical roadmap, build reliable infrastructure from scratch, and shape the engineering culture as the...
- ...things they had to do. Powerful AI will be the biggest lever for... ...builds without losing engineering intent between revisions.... ...structural subsystems into a single releasable product definition that manufacturing, supply chain, and quality can all build against. Close...
FluidStack
New York, NY4 days ago $125k - $135k
QA Test Engineer Application Development QA Test Engineer Ashburn, VA / Remote Current FDA... ...test strategy, test data, test scripts, and release artifacts for development and maintenance... ...using tools such as Selenium and AI-assisted development tools. Validate data...Contract workRemote work- ...seeking a highly skilled QA Automation Engineer with strong expertise in Core Java, Selenium... ...execution, and contribute to enterprise-quality engineering initiatives. Experience in... ...by its market leading capabilities in AI, generative AI, cloud and data, combined...Full timeLocal area
Capgemini
New York, NY3 days ago - Talan Group is seeking a Founding Software Engineer for its NYC office, offering a hybrid working model. This role will involve designing and scaling an AI platform while collaborating closely with AI/ML experts. The ideal candidate will have over 6 years of full-stack...Work at office
- Transformcap is seeking a Founding AI Design Engineer to redefine how pharmaceutical brands create content. You will combine classic design tools with modern AI workflows to build innovative solutions for pharma marketing. The ideal candidate has over 4 years of experience...
- ...Job Description Job Description QA Engineer Role Type: Contractor Location: Remote... ...to support technical evaluation and AI training projects. You'll apply your testing... ...outputs, identify issues, and maintain high-quality standards. Key Responsibilities...Remote jobFor contractors
YO AI Labs
New York, NY25 days ago $94k - $110k
...Role, Mission or Department OverviewThe Quality Engineering team at The New York Times is hiring a... ...Product Engineering to guide our weekly release schedule without sacrificing excellence... ...automate everything and use modern AI-assisted development tools, such as Cursor...Work at officeLocal areaFlexible hours2 days per week$113.4k - $221.75k
The OpportunityAdobe is reimagining the future of creativity, where generative AI and human imagination work together. As a Quality Engineer on the Agentic Harness team, you’ll help the harness behave the way creators expect. You’ll verify that the agent’s behavior, tool...Full timeTemporary workLocal areaWorldwide$113.4k - $221.75k
The OpportunityWe’re looking for a technically strong, AI-augmented QE/SDET who can build and scale quality engineering for a rendering-heavy web application. The ideal candidate brings strong QA judgment, understands product risk, and knows how to use AI tooling to accelerate...Full timeTemporary workLocal areaWorldwideShift workNight shift- Job Title: QA Automation Test Engineer II (100% Remote - USA) 6 Month Contract to Hire Job... ...and agile team focused on delivering high-quality automation for web portals and API components... ...directly into each article, started with the help of AI. #J-18808-Ljbffr ZeektekFull timeContract workRemote work
- Job Title QA Automation Engineer (Selenium, Playwright & Performance Testing) Experience... ...Playwright, and Apache JMeter to ensure the quality, reliability, and performance of web... ...cross-platform testing. Familiarity with AI-assisted testing tools. Technical Skills...Shift work
- Role: Senior QA Engineer AI & Automation Testing Location: Remote Duration: FTE Job Summary QA Engineer - Python & Playwright... ...products using Python and Playwright, ensuring high-quality, scalable, and reliable releases across web applications and APIs. Demonstrate hands-...Remote work
- Get AI-powered advice on this job and more exclusive features. Greetings from Cogent... ...LLC , we are looking for a QA Automation Engineer with one of our direct clients in Austin,... ...Contract Job function Information Technology, Quality Assurance, and Administrative Industries...Contract workRemote work
- Position: Senior QA Automation Engineer Location: Hybrid in NY, NY or... ...for Capital Markets ensuring quality, reliability, and performance.... ...accuracy and integrity. Demonstrate AI fluency by leveraging AI-... ...proficiencies Excellent understanding of release testing, sign-off and...Contract workWork experience placement
$45 - $50 per hour
Job Title: Senior QA Automation Engineer Location: Hybrid - New York,... ...frameworks, and ensuring the quality and performance of enterprise... ...Capital Markets, FinTech, and AI-powered testing tools is highly... ...environments, builds, deployments, and release testing activities....Permanent employmentContract work- Senior QA Automation Engineer Location: Remote U.S. Role Overview This role involves performing... ...(ADO) Test Plans, Agile/SAFe practices, AI-driven test case generation, and Gherkin-... ..., and continuous learning. We offer quality career resources, training,...Remote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Founding AI Quality & Release Engineer. Be the first to apply!
Related searches
- ai engineer New York, NY
- ai research engineer New York, NY
- ai engineer remote New York, NY
- ai prompt engineer New York, NY
- ai developer New York, NY
- senior ai engineer New York, NY
- machine learning ai engineer New York, NY
- ai ml engineer New York, NY
- advanced quality engineer New York, NY
- quality engineer- qms New York, NY