AI Evaluation Architect for Sales Engineering
Mercor
Mercor is partnering with a leading AI research organization to engage experienced sales engineers for a project focused on evaluating how well AI systems perform real-world technical sales work. You will define what excellent work looks like by designing task-specific grading criteria and scoring completed work samples with rigorous, well-reasoned written justifications. You will design precise evaluation criteria for real-world pre-sales deliverables, score AI-generated and human samples with #J-18808-Ljbffr Mercor
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI Evaluation Architect for Sales Engineering in New York, NY vacancy
- Artemis Arc is seeking an Evaluation Specialist Sr to design and implement end-to-end evaluation systems for the VHA AI Education Hub. You will measure adoption, impact, and learning outcomes across AI education initiatives for the Veterans Health Administration. You will...Suggested
- Mercor is partnering with a leading AI research organization to engage experienced UI/UX and product designers for a project that evaluates how well AI systems perform real-world digital product design work. You will define what excellent work looks like by designing task...Suggested
$122k - $240.5k
Position Summary Google AI Architect/AI and EngineeringJoin our AI & Engineering team in transforming technology platforms, driving innovation, and helping... ...reliability, security, and cost.Design, fine-tune, evaluate, and govern LLM solutions with Gemini on Vertex AI...SuggestedLocal areaVisa sponsorshipFlexible hours$222.5k - $261.9k
...West Monroe is seeking a Senior Architect to lead the technical vision, architecture, and engineering strategy for the firm's Asset... ...of proprietary software assets, AI capabilities, and reusable platforms... ...AI platforms and frameworks. Evaluate emerging technologies and...SuggestedLocal areaImmediate startFlexible hours- ...your goals. EisnerAmper is seeking an AI Platform Architect, Strategy & Enablement who will serve... ...The role will build model lifecycle, evaluation, and multi-vendor orchestration as... ...intersection of platform architecture, AI engineering, and strategic operations. The...SuggestedFull timeLocal areaRemote workWork visa
$215k - $275k
...journeys. We blend design, product engineering, analytics, automation, and AI-native delivery to build... ...at enterprise scale. As an AI/ML Architect, you’ll design and deliver production... ...orchestration, agentic workflows, evaluation, deployment, monitoring, optimization...Temporary workLocal area$190k - $199k
...unsure how we would tailor the above?What You Will Get To Do:Own the AI vision for the contact centre: Define and lead the global AI... ...outcomes, operational priorities and regulatory requirements. Evaluate opportunities based on value, feasibility and customer impact.Select...Ongoing contractFull timeWork at office$187.6k - $220.7k
...West Monroe is looking for an Advisory Architect, Data/AI due diligence to join our Technology &... ...; M&A Transaction Services-Data Engineering & Analytics and deliver on technology... ...Individual salaries are determined by evaluating a variety of factors including geography...H1bLocal areaImmediate startFlexible hours- ...our team.KPMG is currently seeking an AI Context Architect to join our Technology Organization (ATO... ...consumableCollaborate closely with AI engineers, content owners, and product managers... ...a rigorous experimentation and evaluation framework to measure retrieval quality...Local area
$230k
...Transformation, Data & Digital Platforms, AI at Scale, Agile,... ...What You'll DoPrincipal AI Tech Architects at BCG Platinion are:... ...system design with hands-on AI engineering - ideal for someone who can envision... ..., and agent orchestration.Evaluate emerging AI frameworks and...Work at officeLocal area$150k
...platforms, at scale.What You'll DoSenior AI Tech Architects at BCG Platinion are:Collaborative.... ...modules in IT implementation projects.Evaluating products, SW packages, and solutions,... ...information systems, or system controls & engineering or a related field (Master’s degree is...Work at officeLocal area$187.6k - $220.7k
Are you ready to make an impact?Agentic AI Architect-Google Cloud West Monroe is seeking an... ...will lead the strategy, architecture, and engineering delivery of enterprise agentic AI... ...clients, exercising sound judgment in evaluating outputs. Apply AI technologies (e.g., generative...Local areaImmediate startFlexible hours- ...A tech-focused research lab is seeking Property Manager Experts to design and evaluate property management challenges for training AI models. This remote role allows you to leverage your expertise in real estate operations while making a significant impact on AI solutions...Remote work
AfterQuery Experts
New York, NY5 days ago $118k - $157k
...seeking a motivated and experienced Security Sales Engineer to join our dynamic enterprise sales... ...sales cycle, including during customer evaluations and proof-of-value initiatives.Develop and... ...and security platform for the AI era, providing businesses with unified visibility...Work at office$220k - $290k
Harness is the AI Software Delivery Platform company, led by technologist and entrepreneur... ...185M deployments, 82M builds, 18T flag evaluations, 8M security scans, 9.1B optimized tests... ...even faster.Position SummaryThe Senior Sales Engineer lives and works in the intersection...Local areaImmediate startFlexible hoursShift work$190k
...Transformation, Data & Digital Platforms, AI at Scale, Cybersecurity and... ....What You'll DoLead AI Tech Architects - AI Platforms at BCG... ...working relationships across engineering, data, product, and client... ..., and operational complexity.Evaluate emerging Ai frameworks and tooling...Work at officeLocal area$134k - $348.5k
...Description & SummaryThe OpportunityAs an AI Architect Director, you will lead the design and... ..., Computer Applications, Computer Engineering, Engineering & Business, Information Technology... ...- Establishing AI design patterns, evaluates platforms and tools, and guides...Full timeH1b$119k - $299.93k
...Description & SummaryThe OpportunityAs an AI/Blockchain Architect Senior Manager, you will design and... ..., Computer Applications, Computer Engineering, Engineering & Business, Information... ...collaborating closely with team members. We evaluate these factors thoughtfully to...Full timeH1b- ...our team.KPMG is currently seeking an AI Architect to join our Audit Technology Alliance organization... ...the product visionPartner with senior engineering managers, product leaders, and data... ...engineering standards and advanced AI evaluation frameworks like LangSmith, taking...H1bLocal area
$165k - $175k
.... Role Overview The Enterprise Sales Engineer is a senior pre-sales and solution-design... ...just answering questions, but shaping evaluation criteria Capture detailed client... ...and deal strategy Ability to leverage AI-tools effectively to drive operational...RIMES Technologies
New York, NY2 days ago- ...human customer experiences with AI. We partner with industry... ...the end-to-end technical pre-sales process by scoping customer use... ...our Sales, Product and Agent Engineering teams to both drive our GTM function... ...job description. We strive to evaluate all applicants consistently...Full timeFlexible hours
Sierra
New York, NY3 days ago $180k - $240k
As a Director, Enterprise Sales Engineering, you will lead a team of frontline leaders, driving... ...world increasingly dominated by Agentic AI.Partner confidently with senior Sales leadership... ...sales cycles in the region, including evaluations, customer engagements, and deal...Work at office- ...human customer experiences with AI. We partner with industry... ...the end-to-end technical pre-sales process by scoping customer use... ...our Sales, Product and Agent Engineering teams to both drive our GTM function... ...job description. We strive to evaluate all applicants consistently...Full timeFlexible hours
Sierra
New York, NY7 hours ago - ...our team.KPMG is currently seeking an AI Architect to join our Audit Technology Alliance organization... ..., automated testing, and AI model evaluation using LangSmith, directly contributing to the continuous improvement of our engineering standardsLead the detailed technical...Local area
$110k - $120k
...BCG Platinion | AI Tech Architect (Internship) - New York, United States of America Locations... ...across BCG Platinion, including engineering, data, AI, cybersecurity, product and... ...decisions across AWS, Azure, and GCP Evaluate scalability, resilience, performance,...Full timeSummer workInternshipWork at officeLocal areaBoston Consulting Group
New York, NY10 days ago- ...Job Description Role: AI Architect (Agentic AI / RAG) Location: NY and NJ Position... ...development; agent harness design and evaluation; multi-agent orchestration patterns;... ...knowledge graph integration; prompt engineering; identity-as-code and policy-as-code; design...Full time
TalentOla
New York, NY4 days ago - Mercor is building high-fidelity simulated work environments to evaluate and improve AI agents on real marketing work. You will help design the... ..., triage, and multi-tool orchestration across marketing and sales stacks. Applicants should have hands-on marketing operations...
- Mercor is partnering with a leading AI research organization to engage experienced brand designers for a project focused on evaluating how well AI systems perform real-world brand and design work. Rather than producing deliverables yourself, you will define what excellent...
- Obsidian in New York builds high‑fidelity simulated work environments to evaluate AI agents on real marketing tasks. You will design creator and influencer scenarios, including briefs, artifacts, and the judging criteria that separate strong responses from plausible but...
$55 per hour
A leading technology firm is seeking an Evaluation Scenario Writer - QA to ensure the quality of evaluation scenarios for AI projects. This part-time, flexible role involves reviewing tests, spotting inconsistencies, and collaborating on automation. Ideal for those with...Remote jobPart timeFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Architect for Sales Engineering. Be the first to apply!
Related searches
- pre-sales engineer New York, NY
- sales engineer New York, NY
- sales engineer remote New York, NY
- director sales engineering New York, NY
- senior sales engineer New York, NY
- sales engineer technical sales New York, NY
- internal sales engineer New York, NY
- sales strategy New York, NY
- global sales New York, NY
- appliance sales New York, NY