Senior AI Test Engineer: Build Scalable AI & LLM Evaluations
TWG Global AI
TWG Global AI in New York is seeking a Senior or Staff AI Software Engineer in Test to develop test automation frameworks for AI products. The ideal candidate will have significant experience in Python and Java, along with a strong software engineering background focused on test automation. This role is a great opportunity to work collaboratively with AI engineers and contribute directly to deploying high-quality AI solutions in a rewarding environment. #J-18808-Ljbffr TWG Global AI
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior AI Test Engineer: Build Scalable AI & LLM Evaluations in New York, NY vacancy
- A technology company in New York is seeking a skilled AI Engineer to enhance AI capabilities, evaluate model performance, and implement testing practices. Candidates should have a Bachelor's degree in a relevant field, with substantial experience in software engineering...Senior
$229.9k - $262.4k
...Senior Lead AI Engineer (LLM Gateway, FM Hosting) Overview: At... ...committed to continuing to build world-class applied... ...experiences and scalable, high-performance AI... ...Design, develop, test, deploy, and support... ...search, guardrails, model evaluation, experimentation, governance...SeniorFull timePart timeLocal areaCapital One
New York, NY11 hours ago- ...scaling reusable AI-driven... ...agents. You will build foundational components... ...reliability, evaluation rigor, and... ...broadly across engineering teams. Key Responsibilities... ...patterns for LLM-powered agents.... .... Build testing frameworks,... ...into scalable services. Identify...Senior
Calliere Group
New York, NY6 hours ago $95k - $130k
Cognizant is seeking an AGENTIC AI DEVELOPER in New York. This remote role will involve building AI systems, productionizing LLMs, and collaborating with cross-functional teams. The ideal candidate should have 5+ years of software development experience, particularly with...SuggestedRemote job- A leading tech firm in New York is seeking a Senior AI Engineer to join its Equities Technology AI group. This role entails managing the design... ...will have a strong background in Python, experience with LLM frameworks, and a PhD or Master's with substantial work experience...SeniorWork experience placement
$229.9k - $262.4k
Senior Lead AI Engineer (AI Foundations, LLM Customization and Finetuning)... ...committed to continuing to build world-class applied... ...product experiences and scalable, high-performance... ...Design, develop, test, deploy, and support... ...guardrails, model evaluation, experimentation, governance...SeniorFull timePart timeLocal area$229.9k - $262.4k
...Overview Senior Lead AI Engineer (AI Foundations, LLM Core and Agentic AI) Overview... ...to continuing to build world-class applied... ...experiences and scalable, high-performance... ...Design, develop, test, deploy, and support... ...guardrails, model evaluation, experimentation, governance...SeniorFull timePart timeLocal area$178k - $267k
AlphaSense is looking for a passionate software engineer to join their AI & Search mission in New York. In this role, you'll architect and implement scalable services for AI and search products, improving reliability and cost efficiency. You should have solid back-end engineering...Senior- ...Feedinkoo is searching for a skilled software engineer to develop scalable, AI-powered applications utilizing models like GPT‑4 and Claude. The role involves building internal tools and APIs, integrating AI models into real workflows, and owning the lifecycle of features...Remote work
Feedinkoo
New York, NY1 day ago - ...A tech-driven company is seeking an AI Engineer to design, build, and operate machine learning systems in a remote role based in the United States... ...for developing end-to-end ML pipelines, implementing evaluation metrics, and optimizing production services. Strong Python...Remote work
Rex USA
New York, NY3 days ago - ...A leading technology research organization is seeking an AI Data Engineer to build scalable data pipelines and collaborate with data scientists on impactful AI initiatives. This is a fully remote role ideal for someone with at least 4 years of Data Engineering experience...Remote work
Pro Search
New York, NY3 days ago $197.3k - $225.1k
...Lead AI Engineer (AI Foundations, LLM Customization and Finetuning) Overview... ...to continuing to build world-class applied... ...experiences and scalable, high-performance AI... ...Design, develop, test, deploy, and support... ..., guardrails, model evaluation, experimentation, governance...Full timePart timeLocal areaCapital One Financial Corp
New York, NY4 days ago$30 - $50 per hour
...A leading tech company is seeking an AI Engineer for a remote role. You will build, evaluate, and deploy production AI systems, focusing on NLP, LLMs, and computer vision. The ideal candidate will have mid-senior experience in delivering ML systems, strong Python skills...SeniorHourly payRemote workRex USA
New York, NY2 days ago- ...Texas Sports Academy Main is seeking a Software Engineer (AI-Forward) to build software for managing student records and AI tools. The role requires... ..., proficiency in AI coding tools, and involvement in LLM-powered features. You'll work closely with founders, moving...SeniorRemote work
Texas Sports Academy Main
New York, NY2 days ago - ...Action1 Corporation is seeking an Applied AI Engineer to build LLM-powered internal solutions and customer-facing AI features. This fully remote position allows for flexibility and ownership over technical decisions within a collaborative environment. The ideal candidate...Remote work
Action1 Corporation
New York, NY3 days ago - ...Yuno is looking for a Senior AI Engineer to build innovative AI-driven solutions. In this remote full-time position, you'll develop scalable AI products using Large Language Models (LLMs) to enhance payment systems. Key responsibilities include designing and optimizing...SeniorFull timeRemote workWork from homeHome office
Yuno Inc
New York, NY2 days ago - ...Senior AI Engineer Washington D.C. / New York... ...functionally to deliver scalable AI-powered... ...design, development, evaluation, and deployment of... ..., including back-testing, rejection inference... ...AI/ML, GenAI/LLM technologies, staying... ...experience building and deploying machine...SeniorFlexible hours
VantageScore®
New York, NY3 days ago - ...seeking a highly skilled Senior AI Engineer with deep expertise in... ...-tuning techniques. Build RAG Pipelines:... ...optimization, and model evaluation. Work with AWS Services... ...to deploy and manage scalable AI solutions. Contribute... ...ML roles, focusing on LLM agent development, data...SeniorLocal area
Quantanite
New York, NY3 days ago - ...world. Cellebrite’s AI-powered Digital... ...the development of scalable, cloud-native AI solutions... ...role focuses on building and maintaining... ...data, and evaluating algorithm performance... ...comprehensive unit testing, performance testing... ...caching, and LLM integration.Knowledge...Senior
Cellebrite
New York, NY3 days ago - ...United States is looking for a Full Stack AI Engineer to develop innovative solutions in a remote environment. The role entails building customer-centric user experiences and... ...engineers and scientists to enhance product scalability. Ideal candidates will have over 5 years...Remote work
Reveleer
New York, NY3 days ago - ...Framework Ventures is looking for a Security Engineer to build AI-driven security infrastructure for LLM applications. The role involves designing multi-agent systems and integrating security methods into development pipelines. Candidates should have at least 3 years of...Senior
Framework Ventures
New York, NY3 days ago - ...Conversica is seeking a Senior AI Software Engineer to design, build, and scale... ...maintainable, and scalable AI-driven solutions... ...engineering, including agent evaluation, interpretability,... ..., regression tests for prompts/agents,... ...experience deploying LLM-powered systems/applications...SeniorFull timeRemote work
Conversica
New York, NY3 days ago $142.32k - $213.48k
...Senior AI Engineer, Banking Technology - C13 (VP Level... ...to design, build, and deploy cutting... ...on implementing scalable, agentic AI frameworks... ...integrating LLM-driven workflows... ...improvement. Evaluation & Optimization:... ...including Git, CI/CD, testing, code reviews, agile...SeniorFull timeCitigroup Inc
Jersey City, NJ11 days ago$160k - $170k
...more. Role As a Senior AI Engineer focused on... ...state management Build and maintain MCP... ...Integrate with managed LLM services across... ...balancing quality, scalability, latency, cost,... ...implement automated evaluation frameworks to... ...around reliability, testing, and operational...SeniorOctus Inc
New York, NY3 days ago$187k - $240k
...representation, and more. We're building a new wave of AI-powered capabilities... ...-end development of LLM- and Agent-based... ...product-minded engineering role: you'll work... ...the way to reliable, scalable production systems.... ...collection, modeling, evaluation, deployment,...SeniorDatadog
New York, NY2 days ago$144.5k - $230k
...Senior AI Engineer We are the better way to work in finance... ...want to be part of building something great. Come... ...and RAG pipelines to evaluation frameworks and production... ...findings into scalable, reliable tools that... ...development, or applied LLM engineering ~ Hands...SeniorWork at officeLocal areaRemote work2 days per weekAccordion USA
New York, NY2 days ago$74.1k - $147.8k
...Crowe, you can build a meaningful and... ...ideas, such as AI-enabled... ...Description: Senior AI Engineer 1 About Crowe... ...performance, and ensure scalable, cloud-native... ...and model evaluation workflows that... ...advanced automated testing, including... ...and integrating LLM APIs (OpenAI, Azure...SeniorLocal areaRemote workWorldwideFlexible hoursCrowe
New York, NY3 days ago$150k - $200k
...located in New York City, is looking for a Software Engineer to join the founding team. You will work on building the action layer of Hazel, connecting to systems... ...ideal candidate has experience shipping production LLM features and ownership of API design. The role offers...Senior- ...Dsilo Ai Technologies is seeking a Senior AI Engineer with over 5 years of experience to design and develop AI solutions utilizing Large Language Models. The ideal candidate will integrate cutting-edge technologies into enterprise applications and ensure robust performance...SeniorRemote work
Dsilo Ai Technologies
New York, NY3 days ago - ...A global technology firm seeks a Senior AI Engineer for remote work. This position requires 4-7 years of experience in AI or machine learning. Responsibilities include designing and deploying AI systems, leading model monitoring, and developing data pipelines. Candidates...SeniorRemote work
Traackr
New York, NY3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Test Engineer: Build Scalable AI & LLM Evaluations. Be the first to apply!
Related searches
- ai research engineer New York, NY
- machine learning ai engineer New York, NY
- ai engineer remote New York, NY
- ai prompt engineer New York, NY
- ai developer New York, NY
- ai engineer New York, NY
- ai ml engineer New York, NY
- senior ai engineer New York, NY
- system test engineer New York, NY
- lead performance test engineer New York, NY