Product Evaluations Lead - Gen AI Software
$224k - $356.5kNVIDIA
- # Product Evaluations Lead - Gen AI SoftwareApplylocations: US, CA, Santa Claratime type: Full timeposted on: Posted Todayjob requisition id: JR2021025NVIDIA is seeking a highly analytical Product Evaluations Lead to own the evaluation, measurement, and go-forward analysis of our Generative AI Software including the Nemotron family as they are adopted by our most strategic enterprise ISV partners. As the evaluation's driver for our most strategic partners and products, you will help drive the experiments, benchmarks, and translate results into crisp insights and prioritized next steps that guide Product, Research, and leadership. This role operates at the intersection of rigorous data science, GenAI product strategy, and executive communication, staying ahead of the rapidly advancing reasoning and agentic AI landscape. You will directly influence model release criteria, product direction, and the roadmap for how NVIDIA's GenAI software is measured and improved!**What You Will Be Doing:*** Drive end-to-end evaluation and analysis of NVIDIA's GenAI libraries, primarily Nemotron models for a focused set of strategic ISV partners, translating complex model behavior into clear go/no-go recommendations and prioritized next steps.* Develop and own rigorous evaluation frameworks, partner experimentation, and benchmarks that measure model quality, reasoning, and agentic capabilities against partner requirements and real-world enterprise use cases.* Share learnings and impact. Deliver regular readouts, dashboards, and progress tracking on evaluation results, adoption status, and recommended next steps to Product, Engineering, Research, and leadership stakeholders, consistent with NVIDIA's culture.* Provide analytical judgment under high uncertainty, balancing model quality, risk, and impact from incomplete or noisy signals to inform high-stakes release and integration decisions.* Partner with research scientists and product teams to translate emerging model capabilities into measurable release criteria, and identify the data and signals that feed the model-improvement flywheel.* Represent partner and product evaluation needs to internal teams. Contribute to the product roadmap by synthesizing cross-partner and cross-industry patterns captured from strategic engagements.**What we need to see:*** 12+ years of experience in data science, model evaluation, experimentation, or analytics roles, with a focus on measuring AI/ML or GenAI systems.* Master's or PhD in a quantitative field (e.g., Data Science, Statistics, Computer Science, Economics) or equivalent experience.* Proven track record leading model evaluation and experimentation that directly informed high-stakes, go/no-go product or model-release decisions.* Strong hands-on skills in Python and statistical/experimental methods, including A/B testing, causal measurement (e.g., org-level holdouts), and metric design and failure analysis.* Direct experience evaluating and benchmarking LLMs or GenAI systems including reasoning, agentic, and/or multimodal capabilities, and building high-quality evaluation datasets and human-evaluation strategies.* Ability to translate complex, ambiguous model behavior into clear narratives, reporting, and recommendations for research, product, and executive audiences.* Proven ownership operating cross-functionally across research, engineering, product, and leadership to drive alignment and decisions in a high-velocity environment.**Ways to Stand Out from the Crowd:*** Founding or early-team experience standing up a GenAI evaluation or analytics function defining north-star metrics and rapid-experimentation frameworks that accelerated and de-risked release decisions.* Track record of influencing complex model and product decisions through positive relationships, showing empathy for partner needs and an instinct for translating them into measurable improvements.* Hands-on experience with reward models, data flywheels, or evaluation signals that directly improved model performance.* Published or applied work in LLM capability evaluation or benchmarking (e.g., peer-reviewed research), paired with agility in a high-velocity GenAI landscape.* Highly collaborative standout colleague, able to build deep trust with engineers, researchers, executives, and multi-functional teams at both NVIDIA and partner organizations.We're entering a defining chapter in NVIDIA's history! This role offers an outstanding opportunity to build how our Enterprise Generative AI Software is measured, trusted, and adopted by the world's leading partners. You'll have the support of Engineering, NVIDIA Research, Solutions Architects, and Product teams, and your analysis will directly influence which models ship and how our Enterprise AI business grows. This is an unparalleled opportunity for professional growth and impact in a high visibility team that’s at the forefront of innovation. NVIDIA is widely considered one of the technology world’s most desirable employers. We have some of the world's most forward-thinking and hardworking people on our team. If you're creative and autonomous, we want to hear from you! NVIDIA benefits is available online at Benefits and Support Programs | NVIDIA Benefits#LI-HybridYour base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD.You will also be eligible for equity and benefits.Applications for this job will be accepted at least until August 7, 2026.This posting is for an existing vacancy.NVIDIA uses AI tools in its recruiting processes.NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
- J-18808-Ljbffr NVIDIA Corporation
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Product Evaluations Lead - Gen AI Software in Santa Clara, CA vacancy
- NVIDIA in Santa Clara, CA, is seeking a Product Evaluations Lead - Gen AI Software to drive end-to-end evaluation and analysis of GenAI libraries for strategic ISV partners. You will develop rigorous evaluation frameworks and benchmarks to measure model quality, reasoning...Software
- ...accelerated computing. Today, our AI infrastructure powers global... ...within the Enterprise Generative AI Software team which focuses on evaluation and measurement of models like... ...partners. What You Will Do As a Product Evaluations Lead - Gen AI Software at NVIDIA, you will...Software
$224k - $356.5k
NVIDIA is seeking a highly analytical Product Evaluations Lead to own the evaluation, measurement, and go‑forward analysis of our Generative AI Software including the Nemotron family as they are adopted by our most strategic enterprise ISV partners. As the evaluation’s...Software$215.2k - $245.6k
Lead AI Engineer (Gen AI Platform Services) Overview... ...capabilities with breakthrough product experiences and scalable, high... ...test, deploy, and support AI software components including foundation... ...search, guardrails, model evaluation, experimentation,...SoftwareFull timePart timeLocal areaCapital One Financial Corporation
San Jose, CA1 day ago$229.9k - $262.4k
{"description": "Senior Lead AI Engineer (MLX, Agentic AI, Gen AI platform Services)... ...capabilities with breakthrough product experiences and scalable,... ..., deploy, and support AI software components including... ...search, guardrails, model evaluation, experimentation, governance...SoftwareFull timePart timeLocal areaCapital One Financial Corporation
San Jose, CA1 day ago$197.3k - $225.1k
...Overview Lead AI Engineer (MLX, Agentic AI, Gen AI platform Services) At Capital One... ...with breakthrough product experiences and scalable,... ..., deploy, and support AI software components including foundation... ...search, guardrails, model evaluation, experimentation,...SoftwareFull timePart timeLocal areaCapital One
San Jose, CA4 days ago$250.8k - $286.2k
...Overview Senior Manager, AI Engineering (People Leader) (Gen AI Platform Services... ...our industry leading capabilities with breakthrough product experiences and scalable... ..., and support AI software components including... ..., guardrails, model evaluation, experimentation,...SoftwareFull timePart timeLocal areaCapital One
San Jose, CA4 days ago$190.9k - $334.1k
...Today, ServiceNow is the AI control tower for... ...through a new AI-Native Product-Led Motion. We're looking... ...scale Product Qualified Lead (PQL) models and product... ...cohort analysis, evaluate funnels, and use data to... ...(ITOM), infrastructure software, or enterprise workflow...SoftwareWork at officeImmediate startRemote workFlexible hours$207k - $301k
...and leader within the Cloud AI Quality Evaluation team.Drive high-quality, scalable... ...for GenAI developers and lead features to enable seamless... ..., design infrastructure and product features of the evaluation... ...experience. 8 years of experience in software development.5 years of...Software$129.4k - $198.4k
...the Organization:The Evaluation team builds and evolves... ...for autonomous vehicle software validation. Develop... ...best practices. Leverage AI-assisted development... ...platforms to enhance productivity, improve code quality,... ...the responsibility to lead the change that will make...SoftwareFull timeLocal areaRemote workWork from homeFlexible hours- ...make us more connected, productive, and understanding of the... ...performance computing, cloud, and AI. Whether you’re designing next-gen processors, enabling AI... ...programs move from early evaluation through development and... ..., and hardware/software optimizationHands-on experience...Software
$240k - $333k
...Cloud (GDC) Hardware portfolio including AI Infrastructure General Purpose Compute (GPU... ...engineering, and sales teams to define product requirements, prioritize features, and oversee... ...wide spectrum of solutions from managed software on your own hardware, fully managed...Software- ...us more connected, productive, and understanding... ...computing, cloud, and AI. Whether you’re designing next-gen processors,... ...development, testing, evaluation or design of system... ...engineering or technology lead role.Alternate... ...following:Performing software verification;Unix...Software
$117k - $234k
...handle high impact, critical software/systems monitoring issues, troubleshoot business and production issues. As a member of this fast... ...this role''What you'll do:• Lead by example a team of 3-4 members... ...prompts and send them to the Gen AI model.• Integrate a safety module...SoftwareFull timeTemporary workPart timeWork at officeFlexible hours- ...Position: Tech Lead Santa Clara, Onsite --Locals are... ...Summary Python /.Net and software development using Gen AI Tools We are seeking... ..., and enhance engineering productivity in a global delivery... ...observability insights Evaluate and recommend Agentic AI...SoftwareLocal area
ECHO IT SOLUTIONS INC .
Santa Clara, CA4 days ago - ...builds the world's largest AI chip, 56 times larger... ...to deliver industry-leading training and inference... ...will own day-to-day production support, user administration... ...controls.Assist in evaluating new ERP functionality... ...who are serious about software make their own...Software
- At Fortinet, we believe AI will fundamentally... ...Automation Analyst to help lead the transformation of the... ...redesign to improve productivity, scalability, and the employee... ...-service capabilities.Evaluate emerging AI... ...Experience partnering with software engineering or technical...SoftwareWorldwide
$152k - $241.5k
...tapping into the unlimited potential of AI to define the next era of computing... ...on the world.We are seeking a Software Engineer - Scientific Evaluation to own a shared platform for classical... .... This hands-on role combines production software engineering, rigorous measurement...SoftwareFull time- ...President, IT Data, Analytics, AI & Innovation About the... ...provider of enterprise software support services... ...operational efficiency, and productivity. The role involves leading the development and implementation... ...of the position is to evaluate and scale emerging technologies...Software
- ...Job Title: Tech Lead Job Summary We are seeking... ...high-quality software solutions. The ideal candidate... ...Management Collaborate with product managers, designers, QA... ...have understating of Gen AI based code generation... ...All applicants will be evaluated solely on the basis of...Software
Yantran LLC
Santa Clara, CA2 days ago - ...insights. Experience with modern AI/ML, Generative AI, LLMs, and AI-assisted... ...for business problems. Build and evaluate predictive models using appropriate ML... .... Collaborate with data engineers, software engineers, product teams, and business stakeholders. Communicate...SoftwareFull timeRelocation
Hudson Manpower
San Jose, CA8 days ago - ...make us more connected, productive, and understanding of the... ...performance computing, cloud, and AI. Whether you’re designing next-gen processors, enabling AI... ...AMD, Inc., is hiring MTS Software Development Engineer to... ...or data requires an evaluation of intangibles. Drive projects...Software
$110k - $120k
...and motivated Generative AI Engineer with at least... ...language processing, Gen AI components, computer... ...training. Model Training and Evaluation: Develop and train AI... ...models into existing products and systems, ensuring... ...team, data scientists, software engineers, and product...SoftwareFull timeRelocation package$70 - $80 per hour
...seeking a highly skilled Software Development Engineer to... ...to the adoption of AI-driven solutions in the... ...Collaborate with architects and product managers to prototype... ...Services We’re a leading provider of business and... ...sourcing, screening, and evaluating candidates. AI helps...SoftwareContract workTemporary work$70 - $80 per hour
...seeking a highly skilled Software Development Engineer to... ...to the adoption of AI-driven solutions in the... ...Collaborate with architects and product managers to prototype... ...Services We’re a leading provider of business and... ...sourcing, screening, and evaluating candidates. AI helps...SoftwareContract workTemporary work- Fortinet, Inc. in Sunnyvale, CA, seeks a Senior Software Developer to shape GenAI products across the full development lifecycle, from debugging to design and implementation. You will contribute to Fortinet's GenAI initiatives such as FortiGPT, applying cutting-edge GenAI...Software
- ...Responsibilities Design and ship production agents that understand enterprise landscapes... ...search, reranking, grounding, and evaluation. Manage large context windows by deciding... ...building and operating production software, including meaningful recent experience...SoftwareFull time
Tessera Labs
San Jose, CA6 days ago $121.13k - $247.93k
...Marketing Strategist to lead co-marketing... ...with our world-class AI ecosystem partners,... ...and AI Independent Software Vendors (ISVs).In this... ...accelerators, next-gen data centers, and... ....Brand Integrity: Evaluate partner executions... ...Intel's AI solutions, product portfolios, data...SoftwareFull timeLocal areaImmediate startWorldwideShift work- ...us more connected, productive, and understanding... ...computing, cloud, and AI. Whether you’re designing next-gen processors,... ...silicon, firmware, and software to deliver reliable... ...impact role, you will lead third-party IP... ...including technology evaluations, commercial negotiations...SoftwareContract work
$160k - $250k
...with the world’s most advanced AI-native platform. We work on... ...CrowdStrike is seeking an experienced Software Development Engineer to join... ...root cause analysis for production issues and implement... ...managementExperience with ML model evaluation frameworks and metrics (precision...SoftwareFull timeWork experience placementWork at officeLocal areaWorldwide2 days per week3 days per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Product Evaluations Lead - Gen AI Software. Be the first to apply!
Related searches
- product lead Santa Clara, CA
- software support Santa Clara, CA
- software Santa Clara, CA
- software technical support Santa Clara, CA
- entry level software sales Santa Clara, CA
- software intern Santa Clara, CA
- software technical support engineer Santa Clara, CA
- id software Santa Clara, CA
- software quality assurance Santa Clara, CA
- government software Santa Clara, CA