Speech AI Evaluation Lead: Metrics & Benchmarks
Sanas
Sanas is building a full Speech AI suite and seeks an Evaluations Lead who designs evaluation frameworks and benchmarking systems. This role sits at the intersection of research, product, and infrastructure to build metrics and studies that hold models accountable. You will shape how Sanas evaluates models across products, ensuring progress is measured by understanding, naturalness, and adaptability in real-world interaction. This is an engineering role with hands-on pipelines and tooling. #J-18808-Ljbffr Sanas
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Speech AI Evaluation Lead: Metrics & Benchmarks in Palo Alto, CA vacancy
- NVIDIA seeks a highly analytical Product Evaluations Lead to own evaluation, measurement, and go-forward analysis of GenAI software including... ...enterprise ISV partners. You will drive experiments, benchmarks, and translate results into insights and prioritized next steps...Suggested
- NVIDIA Corporation in Santa Clara is seeking a Senior Research Manager to lead world-model evaluation and benchmarking across the NVIDIA Physical AI portfolio. This role will build the team and research agenda for evaluating world models through closed-system evaluations...Suggested
- NVIDIA is seeking a Senior Research Manager to lead world-model evaluation and benchmarking for Physical AI. This position involves developing evaluation methods and driving model improvement through rigorous scientific standards. The ideal candidate will have a PhD in...Suggested
- ServiceNow in Santa Clara, CA is seeking a leader for the Build Agent evaluation framework. You will own eval strategy, roadmap, telemetry, and... ...team to deliver scalable, data‑driven improvements. You will benchmark models, decide on model support, and communicate results to...Suggested
- ...ModelMesh(TM), reliable model evaluation through LLM-IQ(TM), and... ...expertise to deliver AI that meets the accuracy... ...advantage. You will lead research across the... ...evaluation, competitive benchmarking) that enables every researcher... ...beyond leaderboard metrics — domain‑expert...SuggestedShift work
- ...build the next generation of AI: autonomous agents that can reason... ...problems for an industry leading financial institution. We are... ...shaping how applied AI is designed, evaluated, and deployed at scale. You... ...and engineering plans, success metrics, and scalable solution...
- Treasure AI:Treasure AI is the agentic experience platform built... ...Your Role:We’re looking for a Lead Product Manager to define and... ...insights into agent workflows.Define metrics for success: adoption,... ...ahead of AI trends: Continuously evaluate emerging LLMs, multimodal...Contract workTemporary workWork at officeFlexible hoursNight shift
- ...more than 120 currencies, we are a leading processor of USD payments with daily... .... As a Vice President, Applied AI/ML Lead (Sr Level IC role) within JPMorgan... ...constraints. Define rigorous evaluation and measurement: offline metrics, calibration, robustness testing,...
Next Frontier Capital
Palo Alto, CA2 days ago $140.4k - $175.5k
...People Operational Excellence & AI team sits at the intersection... ...global organization. You’ll lead process mapping and redesign across... .... Define baseline and target metrics - cycle time, cost, volume,... ...teams to define requirements, evaluate options, and prioritise work...Hourly payFull timeContract workTemporary workPart timeLocal areaImmediate startShift workDay shift$200k - $247k
...the strategic engine driving the democratization of AI across Waymo G&A. Operating at the unique intersection... ...prompts, guiding system design decisions, and defining evaluation methodologies and performance benchmarks. Partner with business units to identify manual...Full timeRemote work$115.2k - $172.8k
...with an expanded portfolio of leading-edge technologies that include... ...& DesignArchitect end‑to‑end AI solutions, defining patterns for... ...considerations.AI Technology Evaluation & VettingEvaluate emerging AI... ...results.Support development of metrics and reporting to track AI adoption...Permanent employmentFull time- ...models. Extract and compute graph connectivity metrics such as PageRank, centrality scores,... ...ensuring model interpretability and compliance. Lead model development lifecycle, cross-... ...initiatives, and research efforts focused on AI and ML innovation in the trust and safety...Full time
- Namely is seeking a Senior Reporting Analyst who will take ownership of OTC KPI reporting and related SaaS metrics. This role requires building and enhancing dashboards in Tableau and/or Power BI and involves validating data accuracy while resolving any discrepancies. The...
$220k - $250k
...and high‑impact talent who see AI as a teammate - leveraging it... ...every day. We’re looking for a Lead Product Manager to own the... ...identity risks Build & own an evaluation framework: define what good looks... ...responses, instrument quality metrics, design human and automated...Flexible hours- About PoshmarkPoshmark is the leading fashion marketplace where style... ...initiatives, from defining metrics and building foundational dashboards... ...inform strategic decisions.AI-Empowered Productivity:... ...tests and quasi-experiments to evaluate the impact of new features, algorithms...
- ...actionable insights for leadership and policy-making. You will work across biological risk evaluation, safety frameworks, and model development lifecycles to ensure responsible AI progress. The role involves close collaboration with researchers, engineers, and ethics...
- ...channel strategies and execution. The role involves creating on-brand content, managing community engagement, and tracking performance metrics across platforms. The ideal candidate will have over 5 years of experience in social media management within technology-related...
- Gordon and Betty Moore Foundation seeks an Adaptive Management and Evaluation Officer (Science) and Manager, External Evaluations to lead design, planning, and day-to-day management of external evaluations within the AME department. Based in Palo Alto, CA, this hybrid...
- ...hiring for a position focused on developing innovative speech algorithms and systems as part of the Pixel team. The ideal... ...background, preferably with a PhD. You will lead research projects that integrate AI technologies into Google products, making a significant...
- ...Corporation in Santa Clara, CA seeks a Product Evaluations Lead to own GenAI software evaluation,... ...partners. You will drive experiments, benchmarks, and translate results into actionable... ...years in data science or analytics for AI/GenAI, advanced degree, and strong Python...
- ...seeking a proactive project planning and management professional to lead AI/ML initiatives in Mountain View, CA. You will develop detailed... ...produce technical deliverables, oversee data pipelines, model evaluation, and governance. This role emphasizes clear communication with...
$165k - $248k
...compute infrastructure and platforms, including AI/ML specific products. Preferred... ...of GHG accounting and emissions reductions metrics and standards. Ability to discern key inputs... ...globe. As a Program Manager at Google, you’ll lead complex, multi-disciplinary projects from...Full time$200k
...are seeking an experienced RTL Lead to help drive the... ...Velaura’snext-generation Physical AI SoC. This role combines architectural... ...and transparency and regularly benchmarks compensation to ensure we remain... ...accommodation requests only. Evaluation of requests for...Flexible hours- Voltai is developing world models and intelligent agents to learn, evaluate, plan, experiment, and interact with the physical world. We are... ...with hardware, electronics systems and semiconductors where AI can design and create beyond human cognitive limits. The team collaborates...
$141k - $205k
Google’s Education unit aims to democratize learning, leveraging artificial intelligence to empower people with knowledge and skills. The Learning Science team collaborates with Gemini, Search, YouTube, and Classroom to build high-quality educational products grounded in...$210k - $315.05k
...want to be at the forefront of AI technology exploration and... ...visionary AI Strategy Leader to lead our organization’s AI journey... ...quantify the value, define success metrics, and secure executive... ...POC to enterprise deployment Evaluate and negotiate with external AI...Work at officeLocal area- ...is seeking a highly analytical Product Evaluations Lead to own the evaluation, measurement, and go-forward analysis of our Generative AI Software including the Nemotron family as... ...partners. You will drive experiments and benchmarks, translating results into insights that...
- Crossover is seeking a Reading Program Coordinator to lead on-site K-3 structured-reading workshops across multiple Alpha campuses... .... You’ll deliver daily motivation sessions, analyze AI-driven metrics, and ensure weekly progress targets. This is a full-time, on-...Full time
$175.5k - $237k
Overview Come join Intuit as an IT SOX Lead Risk Advisor within the SOX Risk and Compliance... ...with risk. Manage the deficiency evaluation process including root cause analysis, impact... ...monitoring, and validation. Apply AI‑assisted workflows to perform SOX readiness...Work at office3 days per week- ...role involves performance analysis and total cost of ownership evaluation for AMD’s GPU products. Candidates should have at least 5... ...relevant field, and strong proficiency in performance modeling and benchmarking. The position is essential for driving product strategy and...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Speech AI Evaluation Lead: Metrics & Benchmarks. Be the first to apply!
