AIML Sr. LLM Inference Quality Engineer

Apple is building the next generation of AI data center infrastructure to power large language model (LLM) services at scale. We are looking for a senior engineer to own the correctness consistency and stability of LLM inference in production. You will design and build the automation frameworks evaluation methodology and qualification strategy that give us confidence in every model prompt serving stack and infrastructure change before it reaches our users. Your work is the safety net that lets Apple ship frontier AI quickly without sacrificing quality safety or trust.

In this role you will lead the qualification of an LLM inference output-correctness automation framework for large-scale AI data center deployment and platform validation. You will architect qualification systems for generative AI including agentic workflows and tool-using models running at data center scale validating that outputs remain correct consistent and stable as models weights prompts serving runtimes quantization schemes and accelerator hardware change underneath is a hands-on AI systems engineering role at the intersection of machine learning distributed systems and quality engineering. You will write production-quality software in Python C and/or Swift apply statistical and ML-evaluation methods to validate both data and model behavior and build AI-assisted QA tooling that makes testing itself smarter and faster. You will partner closely with ML engineers inference/performance engineers distributed systems engineers and platform teams and you will be a trusted Quality voice on what correct and stable mean for LLM inference at scale including the inherently non-deterministic probabilistic nature of generative output. You should be comfortable operating in ambiguity defining quality bars where none exist yet and delivering rigorous results on tight schedules.

Design and develop qualification frameworks for LLM inference systems deployed at data center scale validating output correctness consistency and stability across model weight prompt runtime and infrastructure the LLM inference output-correctness automation framework used for large-scale deployment and platform the end-to-end test and qualification strategy for distributed ML serving systems spanning unit integration regression and end-to-end harnesses that enable validation of ML/LLM services independently of upstream model and service dependencies (mocking trace replay and synthetic traffic generation).nnGate releases in CI: every change to a model weight prompt tool definition or runtime runs against the relevant evals producing a before/after comparison a reviewer can read in one glance. nCreate prompt-level and scenario-based regression suites and track quality across model prompt and infrastructure versions over tool-use function-calling and multi-turn agentic workflows for correct invocation argument formation task completion and multi-turn coherence; flag output that is wrong empty or silently safety refusal and policy-compliance coverage verifying guardrail and alignment behavior holds across AI-specific risk: prompt injection data leakage across users and permissions hallucinated or ungrounded output and cost/latency adversarial and robustness tests long-context ambiguous and underrepresented inputs to surface failure modes before users encounter inference-optimization techniques (quantization KV-cache speculative decoding batching) for output parity against a full-precision output equivalence across serving backends and accelerators ensuring hardware and runtime swaps preserve generation functional performance fault-injection and reliability testing benchmarking quality-vs-latency/throughput trade-offs to surface the correctness cost of performance and use AI-powered/agentic QA tooling that improves test-case generation defect detection flaky-test analysis and failure triage automating any check performed manually more than and implement metrics logging pipelines distributed tracing and observability tooling to monitor output correctness model quality data quality drift latency and system health in log analysis using Splunk and Apple internal debugging tools to root-cause failures in complex LLM serving regression performance and integration testing across platforms including iOS and macOS for on-device and server-backed AI production AI infrastructure software in Python C and/or Swift applying modularity scalability testability and performance principles.

5 years of experience with hardware and/or software development lifecycle processes.n5 years of experience in one or more compiled languages (e.g. C C Objective-C/Swift).nProficiency in one or more scripting languages (e.g. Python Go or JavaScript).nExperience using AI/LLMs to improve testing itself test-case generation defect detection or reducing manual QA with LLM serving and inference optimization (batching KV-cache quantization speculative decoding distributed/multi-accelerator serving) and GPU-accelerated with CI/CD Kubernetes containers and cloud-native systems and integrating automated evals into release with machine learning and LLMs their common practical applications and commonly used frameworks (e.g. PyTorch Hugging Face Transformers or comparable inference/serving stacks).nExperience with observability and distributed tracing for ML systems (e.g. Splunk Prometheus Grafana LangSmith or similar) and monitoring for model .S. M.S. or Ph.D. in Computer Engineering Electrical Engineering Computer Science or equivalent experience.

5 years using one or more of the following scripting languages: Python Go or JavaScriptn5 years of experience working with and building operating systemsn2 years of experience in LLM testing/development.

Required Experience:

Senior IC