Senior SDET: Automated Model Evaluation Lead
Socket.dev
Apple is seeking a Senior SDET to lead automated model evaluation across the Intelligence Platform. You will own eval automation, build LLM-as-a-judge in pipelines, and catch regressions before they reach live use.
You will design, implement, and maintain evaluation coverage for generative features, including image, text, and summaries, partnering across modeling and infrastructure teams.
#J-18808-LjbffrVacancy posted 15 hours ago
Similar jobs that could be interesting for youBased on the Senior SDET: Automated Model Evaluation Lead in Cupertino, CA vacancy
- NVIDIA is seeking a Senior Research Manager to lead world-model evaluation and benchmarking for Physical AI. This position involves developing evaluation methods and driving model improvement through rigorous scientific standards. The ideal candidate will have a PhD in...Senior
- ...Experience Validation team builds the tooling and automation that keeps Apple Intelligence features high-... ...before they ship. We are looking for a Senior SDET to lead the design and implementation of automated model evaluation: standing up LLM-as-a-judge in existing and...Senior
Socket.dev
Cupertino, CA15 hours ago $224k - $356.5k
...outstanding opportunity to join NVIDIA, a company at the forefront of AI and high-performance computing. As a Senior / Principal Deep Learning Engineer — Model Evaluation & AI Systems, you will play a meaningful role in crafting the future of AI. Your work will have a direct...SeniorFull time$272k - $431.25k
...building the future, we’re generating it! Our world model team is pushing the boundaries of multimodal AI, robotics... ...models for Physical AI. We are looking for a Senior Research Manager to lead world-model evaluation and benchmarking across NVIDIA’s Physical AI model portfolio...SeniorFull time- NVIDIA Corporation in Santa Clara is seeking a Senior Research Manager to lead world-model evaluation and benchmarking across the NVIDIA Physical AI portfolio. This role will build the team and research agenda for evaluating world models through closed-system evaluations...Suggested
- ...NVIDIA Corporation in Santa Clara is seeking a Senior Software Development Engineer in Test to design, implement, and maintain automated tests for NVIDIA App and related gaming technologies. You will drive Python-based automation, integrate AI-assisted workflows, and collaborate...Senior
NVIDIA Corporation
Santa Clara, CA16 hours ago - ...experienced platform SWQA engineer to own testing for HGX/DGX/MGX servers and CUDA stacks. You will automate server and OS tests, investigate reliability failures, and lead bug lifecycle management in an agile environment. Ideal candidates have 5+ years in OS/server...Senior
NVIDIA Gruppe
Santa Clara, CA16 hours ago - ...NVIDIA is seeking a Senior Software Development Engineer in Test (SDET) to join the Compute CUDA Quality Assurance team in Santa Clara. You will automate testbenches and build scalable, high‑performance validation systems for silicon designs, spanning cloud and on‑premise...
NVIDIA Corporation
Santa Clara, CA15 hours ago - Netskope is seeking a senior QA/SDET with deep PKI expertise to build and test secure cloud services. You will create scalable Python-based automation, design test strategies for certificate lifecycles and APIs, and collaborate with PMs, architects, and SREs to ensure...Senior
- ...ServiceNow in Santa Clara, CA is seeking a leader for the Build Agent evaluation framework. You will own eval strategy, roadmap, telemetry, and... ...scalable, data‑driven improvements. You will benchmark models, decide on model support, and communicate results to...
ServiceNow
Santa Clara, CA15 hours ago - NVIDIA Corporation in Santa Clara, CA seeks a Product Evaluations Lead to own GenAI software evaluation, measurement, and go-forward analysis for strategic enterprise ISV partners. You will drive experiments, benchmarks, and translate results into actionable insights guiding...Senior
- A global IT solutions firm is seeking a Senior Consultant - HP OO in Sunnyvale, CA. The... ...of IT experience, expertise in HP OO and automation workflows, along with strong SQL skills.... ...an IT operations framework, and leading automation design workshops. This is a full...SeniorPermanent employmentFull time
- HKA Enterprises in Santa Clara, CA is seeking a career-level Controls Technician to lead design, development, and implementation of advanced automation systems, owning projects from concept through commissioning. You will mentor engineers, set programming standards, and...Senior
$184k - $287.5k
...application is built. We are seeking a senior vision language model engineer to design and build agentic... ...with our researchers to develop and evaluate prototypes of our latest models, such... ...implement agentic data workflows that automate data discovery, labeling, evaluation,...SeniorFull time- NVIDIA is seeking a Senior Corporate Accountant in Santa Clara, CA. This role involves leading automation projects and ensuring compliance with accounting policies, particularly in the context of growing US federal government business. The ideal candidate will have at least...Senior
$168k - $270.25k
...Nvidia Corporation is seeking a Senior Software Development Engineer in Test to join their Confidential Computing team. This role involves developing test infra and enhancing capabilities using AI tools for more accurate results. The ideal candidate will have over...SeniorNvidia Corporation
Santa Clara, CA16 hours ago- ...semiconductor production facility. You design, develop and lead the implementation of advanced automation systems, including PLCs, HMI/SCADA, and full lifecycle... ...from concept through commissioning. The role offers senior-level responsibility, mentoring for engineers, and...Senior
- HKA Enterprises in Santa Clara, CA seeks a career-level Controls Technician to design, develop, and lead the implementation of complex industrial automation systems—from concept through commissioning—with full ownership. This technical leadership role requires setting...Senior
$148.5k - $223.9k
...your career at the company leading workforce transformation in... ...Salesforce Software Quality Senior Member of Technical Staff (SMTS... ...testing frameworks and automation suites to validate complex PLG... ...flakiness.Human-in-the-Loop & Model Evaluation: Implement and support evaluation...SeniorFull time$182.36k - $273.2k
...place to thrive, learn, and lead. Your Team, Your ImpactMarvell... ...Argentina — and we are looking for a senior experienced technical leader... ...like ChatGPT or Copilot, or automated note-taking bots) during... ...interviews are designed to evaluate your individual experience, thought...SeniorPermanent employmentFull timeWork from home$140k - $224.25k
Are you passionate about automation, AI-assisted development, gaming technology, and improving software quality through thoughtful testing... ..., including how agents are created, configured, prompted, evaluated, and integrated into engineering workflows.Prior experience using...SeniorFull time- ...Cerebras to deliver industry-leading training and inference speeds;... ...Cerebras works with the leading model labs, global enterprises, and... ...looking for an Inference Platform SDET to join the Inference Service... ...who enjoy infrastructure, automation, and debugging across the full...SeniorWork at office
$168k - $270.25k
We are looking for Senior Software Development Engineer in Test to join our Confidential... ...passionate about new feature development, automation development, test and validation infrastructure... ...teams to identify new features and lead developers in definition, automation implementation...SeniorFull timeWorldwide- Waymo is seeking a Senior Tech Lead for ML Infrastructure to drive efficient deployment of large-scale models across diverse hardware. You will work across data engineering, model development, and on-device deployment to enable scalable AI solutions. You will guide optimization...Senior
$272k - $431.25k
...for a new generation of interactive world-model systems. With this release, NVIDIA has... ...engineering leader to transform this technology lead into an industry standard platform and... ...experience building, adapting, or deeply evaluating world models, video generation systems,...SeniorFull time$140k - $215k
...the Role:CrowdStrike is seeking a Software Developer in Test (SDET) to join our AI Detection and Response (AIDR) team. In this role... ...quality improvements to prevent future outagesIncrease and improve automated test coverage across both backend services and UI...SeniorFull timeContract workWork experience placementWork at officeLocal areaWorldwide2 days per week3 days per week$192.2k - $260k
...Amazon's Delivery Foundation Model team, where you'll work alongside... ...We are seeking an exceptional Senior Applied Scientist to help... ...and our extensive training and evaluation infrastructure- Guide and support... ..., evaluation, and inference- Lead focused technical initiatives...SeniorLocal areaWorldwideFlexible hours$265k - $331k
...generations. Role Summary The Perception Adaptation and Integration Lead will oversee the deployment of our core Perception system into... ...on driving data-driven enhancements to machine learning models. Recruit, manage, and mentor individual contributors to support...SeniorFull timeContract workTemporary workPart timeLocal areaImmediate startShift work$174.72k - $295.68k
XPENG is a leading smart technology company at the forefront of innovation, integrating... ...Engineer / Research Scientist to drive the modeling and algorithmic development of XPENG’s... ...DDP, etc.).Conduct systematic ablation, evaluation, and visualization of model behavior across...SeniorFull time$174.72k - $295.68k
...XPENG is a leading smart technology company at the forefront of innovation, integrating... ...Engineers with strong expertise in generative modeling and large-scale deep learning systems,... ...role, you will research, implement, and evaluate world models that learn the dynamics of...SeniorFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior SDET: Automated Model Evaluation Lead. Be the first to apply!
Related searches
- senior lead project manager Cupertino, CA
- senior robotics software engineer Cupertino, CA
- senior devops engineer remote Cupertino, CA
- senior windows systems engineer Cupertino, CA
- senior principal engineer Cupertino, CA
- senior network engineer remote Cupertino, CA
- senior manager executive compensation Cupertino, CA
- senior cloud security engineer Cupertino, CA
- senior power electronics engineer Cupertino, CA
- senior manager accenture Cupertino, CA