Lead, Legal AI Benchmarking & Evaluation
Newcode.ai
Newcode.ai in New York is seeking a highly analytical professional with a strong statistical background to join our Head of Evaluations. In this role, you will design, implement, and scale the testing frameworks used to evaluate our platform, ensuring our AI products meet the highest standards of legal reasoning, factual accuracy, and regulatory compliance while maintaining a near-zero hallucination rate. You will design benchmarks for contract drafting, information extraction, legal research, #J-18808-Ljbffr Newcode.ai
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Lead, Legal AI Benchmarking & Evaluation in New York, NY vacancy
- NewtonX is seeking an ML Lead to serve as the primary technical point... .... In this role, you will design benchmarks in high-value verticals such as finance, legal, and healthcare while collaborating... ...enable you to deliver precise evaluation systems, fostering growth and...Legal
- Who are we? At Newcode.ai, we're transforming how law firms and legal professionals harness AI for real-world impact... ...to join our Head of Evaluations. In this role, you will design, implement... ...Key Responsibilities Design Legal Benchmarks for: Contract Drafting, Information...LegalContract workShift work
$130k - $160k
...OpportunityRole Value PropositionAs Lead, AI-First Delivery Practice, you... ...delivery environment and evaluating AI output quality at scale through test harnesses, benchmarks, or review sampling.Experience... ...company-paid life insurance and legal services, a retirement pension...LegalFull timeTemporary workWork experience placementWork at officeLocal areaWorldwideRelocation package3 days per week- Mercor is partnering with a leading legal AI company to benchmark how AI systems answer real questions of Australian law. We are hiring experienced... ...outputs, scoring responses, and providing concise written feedback for each evaluation, all under #J-18808-Ljbffr DoradoLegal
- Mercor is seeking legal experts to author and review high-quality academic... ...assessment content for an AI research initiative. You will... ...questions across core law domains, evaluate solution quality, and help establish gold-standard benchmarks used to advance AI capabilities...LegalRemote jobFlexible hours
- ...business professionals with AI-powered research tools... ...running large-scale evaluations, the question is no... ...data and domain-specific benchmarks. Almost none of them can... ...the opportunity as ML Lead. Rolling up directly to... ...value verticals (finance, legal, healthcare, and others...LegalFull timeFor contractorsImmediate startFlexible hoursShift work
- Obsidian seeks legal experts to author and review high-quality academic assessment content for an AI research initiative. You will write and verify... ...across core law domains, evaluate solution quality, and help establish gold-standard benchmarks used to advance AI...LegalRemote job
$119.56k - $182.18k
...the forefront of piloting cutting-edge AI technology that is already impacting... ...seeking a highly motivated and skilled Legal AI Solutions Lead to join our Innovation and AI Department... ...business professional teams directly.Evaluates AI use cases and makes recommendations...LegalRemote workWeekend workAfternoon shift$140 - $150 per hour
...consulting opportunity for experienced corporate and M&A lawyers qualified to practise in France. The role supports an AI legal evaluation project benchmarking AI responses to French law questions. You will review AI-generated answers, assess legal reasoning, and provide...LegalRemote jobPart time$116k - $154k
...best fits them.What You'll Do:Lead strategic sourcing for enterprise... ...a documented, collaborative evaluation process, with action plans... ...with Procurement Operations, Legal, Security, IT, and Office Operations... ...Specifically around adopting AI to build repeatable workflows...LegalWork at office- ...machine learning, and generative AI in a highly regulated... ...requests.As an AI/ML Governance Lead - Vice President within the Commercial... ...partner closely with risk, legal, compliance, and technology... ...changing AI landscapeResearch, evaluate, and stay current on new AI tools...Legal
- Generative AI is making machine learning model development faster... ...AI ML Governance & Automation Lead in Consumer & Community... ...science, engineering, product, legal, risk, controls, compliance, and... ...advanced analytical reasoning to evaluate program performance, identify...Legal
- ...emails , and pull in Finance, Legal, Security, and IT just to get... ...and Accel to change that. Our AI-native platform connects every... ...entity. As Omnea's AI Solutions Lead, you own deployment of the... ...clear, specific instructions and evaluate what comes back Client-facing...LegalContract workWork at officeShift workDay shift
Omnea
New York, NY3 days ago $165k - $220k
...MAIA is our next leap in AI, an always-available... ...closely with engineering to evaluate feasibility, shape... ..., and release quality. Lead day-to-day execution of... ...and reliability. Build benchmark sets, red-team scenarios... ...Work with IT, security, legal, and business...LegalWork at office3 days per week$130.97k - $183.68k
...Apply NowGet to Know the Opportunity:The Lead AI Product Manager is a senior individual... ...language models (LLMs), agentic systems, AI evaluation methodologies, and regulated product... ...across business, technology, compliance, legal, and risk functions, the Lead AI Product...LegalFull timePart timeWork experience placementLocal areaWork from homeFlexible hours- ...attorneys, leadership, clients, and tech teams to identify, evaluate, and implement tools that improve how legal work gets done. This is not a traditional attorney... ...; you will pilot, test, and integrate new tools and AI solutions, translating needs into practical...Legal
$130k - $160k
...OpportunityRole Value PropositionAs Lead, Operating Model Design, you... ...for Global Technology’s AI-First operating model. You author... ...incorporate field feedback, evaluate model effectiveness, and align... ...company-paid life insurance and legal services, a retirement pension...LegalFull timeTemporary workWork experience placementWork at officeLocal areaRelocation package3 days per week- Cincinnatus LLC is hiring a senior legal domain expert to collaborate with an AI lab on evaluating and improving legal reasoning in frontier AI models. You will... ...instruction specs, generate golden solutions, and design benchmarks while working closely with research teams. The...Legal
$141.2k - $278.3k
Position Summary Lead Solution Architect - AI & Engineering/EaasJoin Deloitte’s AI & Engineering... ...industries/sectors you serve.Must be legally authorized to work in the United States... ...environmentsExperience with model evaluation, observability, guardrails, and security...LegalLocal areaFlexible hours$185k - $200k
...strategies. Ascot offers clients leading financial security while... ...working confidently alongside AI‑enabled tools, applying sound... ...Serve as the primary liaison with Legal, Compliance, and Risk on AI... ...templates, and self‑service guidance Evaluate governance maturity of AI...LegalTemporary workWork at officeFlexible hours$60 per hour
A leading project-based AI consultancy is seeking legal consultants to evaluate AI systems and improve their reasoning. This non-permanent role requires a law degree and at least 2 years of experience in US law. Candidates must have strong written English skills and a stable...LegalPermanent employmentPart timeFreelanceFlexible hours$130.7k - $205.2k
AI Platforms Analytics and Governance Lead The rapid adoption of AI platforms such as GitHub Copilot, Azure OpenAI... ...and model consumption guidelines. Evaluate emerging AI technologies and... ...strategies. Partner with security, legal, privacy, and compliance teams. Develop...LegalFull timeTemporary workWork experience placementLocal areaRelocationFlexible hoursShift work$151.6k - $205k
...the ASV team. It is Amazon's AI-powered content-to-commerce second... ...streaming. As the lead UX Designer for Shop-the-Show,... ...innovations through data-driven evaluation and A/B testing.Basic qualifications... ...operations, product development, legal, etc.- Experience defining...LegalFlexible hours$200k - $250k
About Distyl AI Distyl is an applied AI technology company partnering... .... Distyl is backed by leading investors including Lightspeed... ...dynamics — procurement, security, legal, and competing internal... ...equal opportunity employer and evaluate all applicants without regard...LegalTemporary workWork at office3 days per week$176.72k - $265.08k
AI Transformation Group Manager The AI Transformation... ...responsible for leading and transforming the technology... ...and resilience of the legal‑entity, counterparty,... ...; own performance evaluation and management, talent... ...analytical tools, and benchmarks and to use them to drive...LegalImmediate startShift work$180k - $235k
AI Enablement Lead, External Relations Location New York Business Area Marketing and Public Relations Ref # 10052528 Description... ...connector to firmwide AI, engineering, product, operations, legal, and compliance partners.Help colleagues navigate access,...LegalTemporary workFor contractorsWork experience placementWork at officeShift work$100 - $150 per hour
...Overview Our client is supporting the development of advanced AI systems for professional legal use, with a focus on improving the quality of legal... ...review and improve AI-generated legal content. You will evaluate briefs, arguments, memoranda, and other litigation...LegalHourly payContract workPart timeFor contractorsRemote work$147.6k - $274.2k
Lead Applied Scientist, Search & Information RetrievalAbout the RoleThis... ...optimization for complex legal, tax, and accounting content.... ...retrieval, ranking, relevance evaluation, and production deployment.You... ...on search, retrieval, and AI strategyMentor applied scientists...LegalFull timeLocal areaFlexible hours$225k - $250k
...within this exciting and growing industry.Job DescriptionSUMMARYThe Lead, AI Governance and Portfolio Management will lead the operational... ...meet enterprise standards for risk, compliance, security, legal, privacy, and architecture. The role serves as the central point...LegalFull timeTemporary workWork experience placementFlexible hours$165k - $185k
...Global Workplace Experience Lead sets the vision,... ...utilization trends, market benchmarking, root-cause analysis,... ...workplace changes. Identify, evaluate, and pilot new... ...artificial intelligence (AI) to efficiently accelerate... ...legitimate business or legal reasons. We will then...LegalFull timeWork at officeLocal areaRelocation
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead, Legal AI Benchmarking & Evaluation. Be the first to apply!
Related searches
- summer legal New York, NY
- legal nonprofit New York, NY
- legal document specialist New York, NY
- legal researcher New York, NY
- summer associate legal New York, NY
- legal compliance analyst New York, NY
- legal reporter New York, NY
- legal billing coordinator New York, NY
- junior legal New York, NY
- asistente legal New York, NY